Improving application throughput 9x with asynchronous responses in Rails 3

Update: A quick overview of the changes needed to setup Ajax-Gist to use asynchronous responses is available here: commit summary at GitHub

Several weeks ago when writing Ajax-Gist.com one aspect of the project I neglected was performance. Because every un-cached request requires that we do a HTTP call to gist.github.com, application throughput was very low, on the order of 5 requests per second.

For the most part this was 'good enough' and logs showed that very few people were being left waiting on the queue because of my fairly aggressive HTTP caching. But when it came down to it I wanted to be able to offer a more reliable and more performant service, ideally without shelling out the extra clams for access to more application instances on Heroku.

While thinking about the problem I remembered that Thin (the web-server that Heroku uses) is capable of serving asynchronous requests using EventMachine. I knew that asynchronous Rack applications were fairly easy to write, but I didn't want to abandon Rails because, while this is a fairly simple application, keeping the simplicity of Rails and getting the advantage of higher application throughput in this and other more complex applications would be a big win.

Luckily this is one of those occasions where other, much smarter, people already had the same idea, albeit fairly recently. My research turned up a lot of work by Ilya Grigorik and Mike Perham that pushed toward this goal. Rack/Fiber-Pool by Mike is a piece of Rack middleware that runs each request in it's own Fiber, allowing the possibility of easy cooperative scheduling in Rack applications. While EM-Synchrony provides a set of Fiber-aware EventMachine clients for common things like HTTP requests, Memcached, MySQL and Mongo.

Mike was kind enough to do all of the leg-work in creating a guide for creating asynchronous Rails 2.X applications and only 2 days ago Ilya provided an example for achieving the same results in Rails 3.

In the end implementation turned out to be relatively simple, and the results speak for themselves:

 1 # Synchronous:
 2 
 3 Benchmarking 192.168.1.3 (be patient).....done
 4 Document Path:          /gist/1010.js
 5 Concurrency Level:      30
 6 Time taken for tests:   18.312 seconds
 7 Complete requests:      100
 8 Requests per second:    5.46 [#/sec] (mean)
 9 Time per request:       5493.543 [ms] (mean)
10 Time per request:       183.118 [ms] (mean, across all concurrent requests)
11 
12 # Asynchronous:
13 
14 Benchmarking 192.168.1.3 (be patient).....done
15 Document Path:          /gist/1010.js
16 Concurrency Level:      30
17 Time taken for tests:   1.923 seconds
18 Complete requests:      100
19 Requests per second:    52.00 [#/sec] (mean)
20 Time per request:       576.883 [ms] (mean)
21 Time per request:       19.229 [ms] (mean, across all concurrent requests)

Now all I have to do is wait for Heroku to update Bundler so I can deploy the new version!

Update 2: Heroku updated their version of bundler to be compatible with Rails3, so the new version of the application is live at http://ajax-gist.com/

The application source is available here: http://github.com/aarongough/ajax-gist/tree/development

My thanks to Mike and Ilya for all their awesome work!