Introducing Rails Metal

Rails Edge adopted Rack a while back and we’ve been exploring ways to expose that better. The first thing we did was to make it really easy to hook up any piece of Rack middleware in front of a Rails request. In your config/environment.rb file, you can do:

config.middlewares.use(Rack::Cache, :verbose => true)

This will make your application use Ryan Tomayko’s excellent Rack::Cache middleware for better HTTP caching. There are lots of other great generic, reusable Rack middleware pieces. Do go shopping.

But there’s another use case where Rack also comes in handy. If you have a Rails application that has service end points that need to be really, really fast. So fast that the few milliseconds that a trip through the Rails router and Action Controller path is too much.

For this scenario, we’ve built a thin wrapper around the generic Rack middleware and given it a place in the hierarchy along with the name “Metal”. Rails Metal is meant for things like the Campfire poller service, which is doing hundreds of requests per second and needs to be really fast and is at the same time really simple (2 database calls and returning a string). This service is currently written in C, but we’ll look into converting it to Rails Metal instead.

Rails Metal is a subset of Rack middleware
Think of Rails Metal as a subset of middleware intended for application-specific end points that need the extra speed (“write to the metal”, hence the name). Thanks to recent adoption of Rack sessions, these pieces of metal even have access to the same session as Rails. You can either run them as part of the Rails process or run them as individual services through rackup.

We’ve even included a generator for Metal, which by default will create a stub ala:

class Poller def if env[“PATH_INFO”] =~ /^\/poller/ [200, {"Content-Type" => “text/html”}, “Hello, World!”] else [404, {"Content-Type" => “text/html”}, “Not Found”] end end end

As you can see, all this does is capture requests to /poller and return “Hello, World!” (and the request is not for /poller, it’ll let the Rails router handle it — that’s what the 404 return does). It will do that just about as fast as is technically possible in Rack (cue useless metric of how Rails now has the fastest, most meaningless “hello world” stat!!).

Note that this is a pretty specialized tool. While you could write an entire Rails application using just metal, it would be a terrible idea in most cases. The vast majority of applications out there will never do what they do so fast that the overhead of the already heavily optimized Rails router and Action Controller execution path will matter one bit (pick up Michael Koziarski’s talk on Rails Performance through torrent for more on this).

Also, if you’re writing a generic filter in the vein of Rack::Cache, Metal is not the right choice either. You should just be creating regular middleware which can be shared across all Rack applications.

But for those few, specialized cases where you just need as much raw speed as possible, Metal can be exactly what the doctor ordered. It allows you to have the best of both worlds in one package. The lowest possible overhead for a Rack application when that matters and the full-featured goodness of Action Controller when it doesn’t.

We hope you like this. Josh Peek has putting tireless hours into getting Rails on Rack and this is just the latest product of his efforts. Thanks a bunch, Josh!

Update: Jesse Newland wrote up a great introduction to Metal as well. It includes benchmarks and even more examples!

Update 2: Changed code example to match latest style.

Hosting Ruby on Rails with Passenger

Phusion Passenger aka mod_rails has been on a tour de force lately and rightfully so. It makes Rails deployment so much simpler and combined with REE faster and with less memory overhead. So I’m really happy to see that lots of the hosting companies in the Rails world are adopting it and making it available to their customers.

Rails Machine recently announced that Passenger is now part of their standard stack and that they’ve made it silly easy to switch from a Mongrel-based setup to Passenger. The latest Accelerator from Joyent has Passenger preconfigured as well. At the shared hosting end, Dreamhost has been supporting Passenger for a while (nice tutorial using Passenger at Dreamhost).

I’ve personally been setting up Passenger at Slicehost with Ubuntu and having great results with that. At 37signals, we’re already running Ta-da List (on EC2) and Backpack (at Rackspace) on Passenger and plan to move over the rest of the applications shortly. Our system administrators certainly appreciate not having to funk with Mongrels any more.

Lots of other hosters are in advanced testing with Passenger as well. Brightbox has been building Ubuntu packages for Passenger and is putting one together for REE. They should have complete Passenger support shortly. Most other Rails hosters I’ve talked to are at least looking into it as well.

But just because Passenger is a big step forward for Rails hosting, it doesn’t mean that other approaches are suddenly useless. There may still be situations where a traditional proxy/Mongrel setup would be relevant. For example, if you for some reason are unable to use Apache, that’s still the way to go. Rails will continue to support both FCGI, proxy/Mongrel, any Rack web server, and of course Passenger.

The change is that if you do not already have an investment in an alternative solution, or if you’re feeling pain with that solution, you should definitely consider Passenger to be the default choice for Rails.

Update: Phusion has posted a guide to how you control the Rails and Ruby environment variables under Passenger. Useful for tweaking the GC settings etc.

Videos from Paris on Rails '08

I love how so many conferences are recording their sessions. Paris on Rails is another of those conferences and they’ve recently published their recordings as both video and audio. Most of these sessions are in French, but Michael Koziarski did a great session on Rails Performance that’s in English and so was my iChat Q&A session.

This Week in Edge Rails

December 6-December 12, 2008 Edition

Here’s your weekly roundup of the action out on the edge, where the code is exciting and sometimes even stable. There were only 32 commits this week; perhaps we’re going into a bit of year-end hibernation.

Memory Improvements

A couple of memory leaks got squashed this week. There was one that occurs in Rails 2.2 when you set config.log_level = :warn (this fix has been applied to the 2-2-stable branch as well as to edge) and one that turns up when you call reset_session in an around filter (this fix is on edge, and will be ported to 2.2 soon). commit commit

Application Templates

There was a bit of cleanup work on the application template feature that was introduced last week. This included support for installing plugins as git submodules, a couple of small cleanup items, and a rake task to apply a template to an existing application . With this task, you can now run

rake rails:template LOCATION=~/template.rb

And the changes indicated in the template will be layered on top of whatever code the project already contains.

I18n Cleanup

A few more bits of the Rails infrastructure are now exposed to internationalization, including connectors in Array#to_sentence and separators in Active Record validation messages . Also of interest to the international Rails community is a patch to add an ActiveSupport::Multibyte::Chars#ord method to properly return the Unicode value of the first character in a string.

Odds and Ends

There’s a new version of the memcache-client ( bundled with edge Rails. This fixes some bugs in the original 1.5 and adds better support for failover and timeouts. You can read more about it on the Lighthouse ticket commit

Active Record’s to_xml support gets just a little bit more flexible with the addition of a :camelize option, which converts all of the column names in the output to camel case. commit

If you test a lot of view code that uses partials, you’ll be happy to know that assert_template now takes a :partial option that can look for them in the output directly. commit

And finally, there’s a patch that has not been applied to Rails yet, but that you might want to go look at in Lighthouse – it adds support for HTTP status code 418 so that Rails can conform to RFC2324 . I’m sure the core team would welcome further discussion of this change.

RailsWay - German Rails Magazine

If you can read German and you develop with Rails you’ll be happy to know that it republik, the same company that also produces a Java, .NET, and PHP magazine, is about to release a Ruby on Rails Magazine called RailsWay.

The magazine is in print and will be out in Germany, Switzerland and Austria by December 17, 2008. For 2009 they plan to release six issues (in the German language), and I hear they’re planning on an English version, but no ETA on that. The content of the first issue includes:

  • Getting started with Ruby on Rails
  • AJAX and Rails
  • RAILS 2.2 – what’s new
  • IDE’s for RAILS
  • Behaviour Driven Development
  • Metaprogramming With RUBY
  • RESTFull Web Services
  • Featured Website:
  • Tips, Tricks, Performance & Security

A DVD comes with the magazine which includes tutorials, videos, free rails books, and a bunch of IDEs. The Editor in Chief is Michael Johann, who has done a great deal to promote Rails in Germany.

New REE with OS X, 64-bit, Solaris support and GC patches

Phusion is on a roll today. Not only did we just get a new Passenger, they’ve also just dropped a new REE (the Ruby patches for copy-on-write) that includes 64-bit support as well as compatibility with OS X and Solaris. They’ve also fitted the excellent RailsBench patches from Stefan Kaes that allows you to tweak the GC settings in Ruby if you need to.

This edition was sponsored by 37signals, Curve21, Dr Dispatch Transportation Software, InfoEther, Martian Interactive, New York Times, Shopify, Trevor Turk, and Utah Imaging.

This Week in Edge Rails

November 29-December 5, 2008 Edition

Rails 2.3 development is still continuing at a healthy clip. Here’s an overview of what’s happened this week.

Application Templates

Rails now incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a template once and use it over and over again when you run the rails command. Pratik Naik provided a thorough writeup of this feature over on his blog. commit

Find Improvements

Even the venerable code used by Active Record to talk to databases can be improved, and we had a couple of improvements checked in this week. First, the SQL generated for preloading records in has_and_belongs_to_many associations has had an extra “AS” excised, which makes it work better with some databases. Also, we now have a :having option on find (as well as on has_many and has_and_belongs_to_many associations) for filtering records in grouped finds. As those with heavy SQL backgrounds know, this allows filtering based on grouped results:

developers =  Developer.find(:all, :group => "salary", 
  :having => "sum(salary) >  10000", :select => "salary")

commit commit

Rack Integration

The tighter integration of Rails with Rack continues. This week saw the death of the venerable CGI processor within Rails, as well as the use of Rack to handle FCGI. There was also some refactoring down in the Rails tests to make them play nicer with Rack. commit commit

Configuration File Cleanup

Over the years, the default environment.rb file had gotten a bit cluttered with comments, random documentation, and settings that mot people don’t need. DHH took a few minutes to clean up the mess, and the current state of this file is much leaner. As part of the cleanup, session store configuration has been moved out to its own file. commit

No More Process Scripts

If you’re one of the (fairly rare) Rails developers who deploys in a fashion that depends on the inspector, reaper, and spawner scripts, you’ll need to know that those scripts are no longer included in core Rails. For Rails applications that are reployed with current alternatives like Passenger, these are just unnecessary baggage. But if you need them, you’ll be able to pick up copies via the irs_process_scripts plugin. commit

Odds and Ends

The fact that Rails checks for IP spoofing can be a nuisance for sites that do heavy traffic with cell phones, because their proxies don’t generally set things up right. If that’s you, you can now set ActionController::Base.ip_spoofing_check = false to disable the check entirely. commit

Rails.root now returns a Pathname object, which means you can use it directly with the join method to clean up existing code that uses File.join. commit

ActionController::Dispatcher now implements its own middleware stack, which you can see by running rake middleware. commit

Gem loading is smarter about missing dependencies. commit

auto_link now properly applies options (such as :target and :class) to generated e-mail links. commit

Phusion Passenger 2.0.5 now compatible with Edge Rails

The Phusion team keeps blazing ahead with Passenger and improving it rapidly. They’ve just released version 2.0.5, which includes a few fixes and introduces compatibility with the Rack-based Edge Rails.

At 37signals, we’ve already switched over Ta-da List and are busy working on getting the rest of our suite running on Passenger. It’s just so much easier to deal with and the memory savings you get through REE are a nice cherry on top.

I keep getting a steady stream of success reports from all over the world as well. I’ve even read of a few people getting back into Rails development because Passenger finally took out the inconvenience of deploying.

It’s hard to argue with the usability. I’ve personally been setting up a new server running Ubuntu 8.10 and using Apache 2 with Passenger. The time it took me to go from a fresh install to a complete production setup was ridiculously low. There’s just so much less to worry about.

If you haven’t given Passenger a chance yet, now is definitely the time.

This Week in Edge Rails

Yesterday was Thanksgiving holiday for US-based developers – but it certainly hasn’t looked like a holiday week in edge Rails. Things are moving fast, with some major changes afoot for version 2.3 of Rails.

Rack integration

The underpinnings of script/server have been simplified and rewritten somewhat. The explicit list of supported servers that used to be in script/server is gone. Instead, Rails now depends on the installation of Rack, and script/server goes through this – which means that Rails supports any server that Rack does.

Efficient routes

Routing sees a couple of big changes this week. The formatted_ route helpers are gone, in favor just passing in :format as an option. This cuts down the route generation process by 50% for any resource – and can save a substantial amount of memory (up to 100MB on large applications, according to the Lighthouse ticket ) If your code uses the formatted_ helpers, it will still work for the time being – but that behavior is deprecated and your application will be more efficient if you rewrite those routes using the new standard. Another big change is that Rails now supports multiple routing files, not just routes.rb. You can use RouteSet#add_configuration_file to bring in more routes at any time – without clearing the currently-loaded routes. commit commit

Better support for engine plugins

The second routing change enables better support for Rails Engines: routing files in engines are automatically loaded and reloaded now (as are those in other plugins). Engines are getting some love other than routing. If your plugin has an app folder, then app/[models|controllers|helpers] will automatically be added to the Rails load path. There’s active discussion of just how this should work, and how much to pick up from the current engines plugins, so it’s likely we haven’t seen the last commits in this area. Engines also support adding view paths now. commit commit commit commit

Sensible backtraces for your tests

If you’re a fan of the Thoughtbot Quiet Backtrace plugin, which allows you to selectively remove lines from Test::Unit backtraces, you’ll be happy to find ActiveSupport::BacktraceCleaner and Rails::BacktraceCleaner in core. This supports both filters (to perform regex-based substitutions on backtrace lines) and silencers (to remove backtrace lines entirely). Rails automatically adds silencers to get rid of the most common noise in a new application, and builds a config/backtrace_silencers.rb file to hold your own additions. commit

Ruby 1.9 integration

A variety of commits continue the drive towards Ruby 1.9 and minitest compatibility. This should ensure that Rails 2.3 is ready to handle the latest Ruby underpinnings when it’s released. Those on the bleeding edge at the moment need to beware, though: one of the changes in edge Rails depends on a ruby-core patch that hasn’t yet been applied there. You’ll also (temporarily) need to pick up Jeremy Kemper’s fork of Mocha for MiniTest compatability as required by this commit .

Faster boot time in development mode with lazy loading/autoload

Jeremy Kemper and Josh Peek have been doing a ton of work on making sure that bits of Rails (and its dependencies) are only brought into memory when they’re actually needed. Check out the commits from November 23 for a bunch of lazy-loading changes. The core frameworks – Active Support, Active Record, Action Controller, Action Mailer and Action View – are now using autoload to lazy-load their individual classes. This work should help keep the memory footprint down and improve overall Rails performance. commit commit commit commit commit


You can specify using the new preload_frameworks option whether the core libraries should be autoloaded at startup. This defaults to false so that Rails autoloads itself piece-by-piece, but there are some circumstances where you still need to bring in everything at once – Passenger and JRuby both want to see all of Rails loaded together. commit

Asset hosts get more flexible in edge Rails with the ability to declare an asset host as a specific object that responds to a call. DHH has supplied a sample project, asset-hosting-with-minimum-ssl , that demonstrates one good use for this functionality. commit

You can now configure the location of the helpers folder for a Rails application by setting ActionController::Base.helpers_dir. This will be a boon in some unusual circumstances – the original use case is for building a Rails application that encourages extension via plugin rather than by altering the application itself. commit

Token generation for CSRF protection has been simplified; now Rails uses a simple random string generated by ActiveSupport::SecureRandom rather than mucking around with session IDs. As a result, the :digest and :secret options to protect_from_forgery are deprecated and have no effect on edge. commit

While we’re on the subject of secrets, some people will find novel uses for ActiveSupport::MessageEncryptor, which provides a simple way to encrypt information for storage in an untrusted location (like cookies). commit

Active Support’s from_xml no longer depends on XmlSimple. Instead, Rails now includes its own XmlMini implementation, with just the functionality that it requires. This lets Rails dispense with the bundled copy of XmlSimple that it’s been carting around. commit commit

As you probably recall, last week’s improvements included the renaming of application.rb to application_controller.rb. This week there’s a new rake task, rake rails:update:application_controller to do this automatically for you – and it will be run as part of the normal rake rails:update process. commit

Good news if you’re using ActiveSupport::OrderedHash: it now implements each_key and each_value. commit

One more bit of core Rails is open to I18n: the units used by number_to_human_size. If you’re maintaining a translation file, you need to add the storage_units: [Bytes, KB, MB, GB, TB] to your translations. commit

Support for Rails components – which were famously called “a shining example of what happens when eagerness overtakes prudence” in Agile Web Development With Rails – is finally gone. If a couple of years of warning about this deprecation wasn’t enough for you, then it’s time to find an alternate solution at last. commit

Various files in /public that deal with CGI and FCGI dispatching are no longer generated in every Rails application by default (you can still get them if you need them by adding --with-dispatches when you run the rails command, or add them later with rake rails:generate_dispatchers). commit commit

Just a reminder: I’m not providing pointers to every single commit here, just trying to highlight things. This week’s edge changes actually included 136 commits from a wide variety of contributors.