Project Completed. Well done everyone!

Last month we successfully finished a major six month project for one of our clients. The project was very exciting, using Ruby 1.9.2, Rails 3.2, HAML, SASS and CoffeeScript. I would like to share two areas that presented interesting challenges:

Payment Integration

Our client uses the SagePay Payment Gateway, so we had to integrate this with our solution. Unfortunately this cannot be integrated using ActiveMerchant, however there is an open source gem sage_pay by Graeme Mathieson that gave us a great start.

Initially this gem was using Ruby 1.8.7, Rspec 1.3 and Rails 2.3. Our first task was to upgrade the gem to support Ruby 1.9.2, Rspec 2.7 and Rails 3.2. This version is going to be merged into the main branch by Graeme and it is also available here: sage_pay_rails_3.2. The new version also supports the new SagePay Token System, which is a safe way of keeping card details without actually storing them in your own system.

The gem implements the SagePay Server 2.23 Protocol, but additional models must be built to track transactions and other relevant data. Graeme has also published most of the necessary boiler plate code here. We upgraded the code to the latest gems and included additional models for security audits.

Finally, we implemented models related to our specific payment flows for plans, subscriptions and purchasable content. We tested different scenarios with VCR, Sinatra and, to simulate different conditions, we mocked valid and invalid responses from the payment server. **

Search Optimization

In our first iteration we implemented the search with Metasearch and Metawhere, which provided a way to do complex find conditions without SQL. ­These gems were later replaced by Ransack and Squeel which provide a number of improvements.

Initiallly the number of search parameters required by the interface was just two, however, as the project progressed this grew to twenty.

Because of this, our approach stopped being effective, since Metawhere was generating non-optimal SQL strings that took minutes to execute when a lot of search parameters were requested. To attempt to solve this issue, we manually wrote the SQL for complex searches, which worked fine until we needed to add multiple highly-responsive search filters. These filters were generating hundreds of small SQL searches every time the user touched an array of checkboxes on the UI, slowing down the search performance once again and therefore requiring a different approach.

Finally, we decided to use Sunspot to make searching through Solr easy. We were satisfied with the performance of Solr and the built-in facet functionality for search filters. There were some new elements to consider, such as handling error conditions when the Solr server was down and mocking up Solr for the tests to run fast (we used the sunspot-rails-tester for this).

Naturally with a project of this size there are many other areas of interest, such as making the code base clean or breaking large classes into smaller ones. I am interested in exploring these topics in more detail, so I will be attending the Software Craftsmanship 2012 event at Bletchley Park Trust.

See you there if you are around!