Activist Status & Wiki Project

So last week we were announced as activists, but what have we been doing? I’m sure there are some wondering, so here is our first report. Read on for news of our first big project, what each of us is working on, and how you can help today.

Firstly, we’ve been very pleased by the number of people who are actively interested in helping us improve the Rails ecosystem. Your input through UserVoice, the Activism Mailing List, Twitter, e-mail, and instant messages has been inspiring (and a bit overwhelming!). We listened, and one of the most popular requests revolved around the Wiki.

The Rails Wiki Reform

The Rails Wiki is in poor shape, for many reasons and it’s not one person’s fault. The good news is that our first major project is going to be revitalizing the wiki. To get our engines started, we’re forming a dedicated team to shape it into something the Rails community can be proud of.

If you’re interested in helping, simply join the Ruby on Rails Wiki Google Group. After you join, you’ll find a note on the group from Matt Aimonetti with more details.

What we’re doing

All of the Activists are busy with projects that fall under the general heading of “helping Rails.” Here’s a sampling of what we’re up to:

Gregg Pollack

Matt Aimonetti

  • Putting together Case Studies.
  • Investigating the current state of the Rails wiki and possibilities.
  • Working on the Merb book (yes, Merb projects help Rails too).

Ryan Bates

  • Doing Railscasts.com.
  • Creating a Screencast application for aggregating Rails screencasts all into one feed on the RubyOnRails.org website.

Mike Gunderloy

  • Covering This Week in Edge Rails on the official Rails blog.
  • Working with Chad Woolley to improve the Rails CI server, including builds across multiple versions of Ruby and JRuby.
  • Providing user support via #rubyonrails and #rails-activism on IRC freenode.
  • Posting daily link roundups pointing to things of interest to Rails developers.

Other Stuff You Can Do

If you’re asking “What can I do today to help Ruby on Rails?”, aside from joining one of our mailing lists and collaborating with us, here are a few items that come to mind:

  1. Contact Matt Aimonetti if you think you have a good Case Study on Rails.
  2. Go to RubyForum or IRC and help new Rails developers.
  3. Go to your local Ruby Users Group, or start one.
  4. Go to a Rails related conference.
  5. The next time you tackle a hard problem in your rails app, write a blog entry.
  6. Take one of your Rails Libraries/Modules, and turn it into a Gem or Plugin.
  7. If you’re surfing Rails blog articles and you find one that is outdated leave a comment or let the author know the content should be updated or marked as obsolete.

Flickr Credits: Big Red Button Okinawa-Churaumi Aquarium marbles

This Week in Edge Rails

January 3, 2009 -January 9, 2009

It was a pretty light week for the edge Rails tree: about 20 commits. We’re starting to see things coalesce for a 2.3 release, though there’s no official release date yet. Here’s some of the highlights of what’s been going on.

AssetTag Timestamp Caching

You’re likely familiar with Rails’ practice of adding timestamps to static asset paths as a “cache buster.” This helps ensure that stale copies of things like images and stylesheets don’t get served out of the user’s browser cache when you change them on the server. You can now modify this behavior with the cache_asset_timestamps configuration option for Action View. If you enable the cache, then Rails will calculate the timestamp once when it first serves an asset, and save that value. This means fewer (expensive) file system calls to serve static assets – but it also means that you can’t modify any of the assets while the server is running and expect the changes to get picked up by clients. commit

Object#tap Backport

Object#tap is an addition to Ruby 1.9 and 1.8.7 that is similar to the returning method that Rails has had for a while: it yields to a block, and then returns the object that was yielded. Rails now includes code to make this available to older versions of Ruby as well. commit

Rack Version Bump

During the week, the Rack project released version 0.9 and Rails was quick to respond – the required version of Rack for Rails is now 0.9. So, if you’re running on edge, it’s time to update your gems. commit

Continuous Integration Setup

If you’re interested in setting up your own continuous integration server to build the Rails source, the embedded instructions have been updated with the latest requirements and clarifications. Even if you want a CI server for something else, they’re worth looking at, as they’ll get you from zero to a running cc.rb instance very quickly. commit

Announcing the Rails activists

Railway station crowd

Bringing Rails and Merb together is about more than just merging the respective code. We’re also picking up the best ideas from both communities beyond the code. Following on Merb’s success in offering a strong evangelism effort, we’re pleased to announce the creation of the Rails activists:

The mission of the Rails activists is to empower and support the worldwide network of Ruby on Rails users. We do this by publicizing Rails, making adoption easier, and enhancing developer support.

At launch, we’ve identified seven areas where the Rails activists can contribute to the Rails ecosystem:

  • Public Relations with media of all sizes
  • Ombudsman work to ensure good user-to-user support
  • Community Leadership at events and conferences
  • Media Organization to help create good promotional opportunities
  • Website maintenance
  • Documentation efforts
  • Developer support

The initial members of the Rails activists are Gregg Pollack, Matt Aimonetti, Ryan Bates, and Mike Gunderloy. But we can’t do all this alone, nor do we want to! Our vision includes a large and vibrant Rails network composed of other activists, bloggers, event hosts, authors, and developers. Our intent is to provide connections, resources, and support to help the entire ecosystem to grow. To start things off, we’re bringing in a lot of our own projects, including videos, screencasts, case studies, Rails documentation, and more – we’re a working group, and we hope you’ll work with us.

If you have ideas about improving the Rails community, projects you want to participate in, or are just looking for ways to get involved, get in touch with us! There are a lot of ways to do that:

We look forward to hearing from you!

For additional perspectives from the activists, see the posts by Gregg Pollack, Matt Aimonetti, and Mike Gunderloy.

Photo by Flickr user caravinagre

This Week in Edge Rails

December 27, 2008-January 2, 2009

Happy New Year! Apparently the Rails core team was not doing too much partying to end the old year: we had 35 commits hit the edge tree, and some of them involved very substantial work. Here’s my weekly overview of some of the most visible and significant changes.

Optimization of respond_to

In some of the first fruits of the Rails-Merb team merger, Yehuda Katz took a look at the respond_to method, which is of course heavily used in many Rails applications to allow your controller to format results differently based on the MIME type of the incoming request. After eliminating a call to method_missing and some profiling and tweaking, he reports an 8% improvement in the number of requests per second served with a simple respond_to that switches between three formats. The best part? No change at all required to the code of your application to take advantage of this speedup. commit commit

If you want a preview of what else to expect from Rails 3, you might want to dip into Yehuda’s own fork of the Rails tree; I’ll be covering these changes as they make their way back into the master copy of edge Rails.

Dynamic Scopes for Active Record

You know about dynamic finders in Rails (which allow you to concoct methods like find_by_color_and_flavor on the fly) and named scopes (which allow you to encapsulate reusable query conditions into friendly names like currently_active). Well, now you can have dynamic scope methods. The idea is to put together syntax that allows filtering on the fly and method chaining. For example:


Order.scoped_by_customer_id(12)
Order.scoped_by_customer_id(12).find(:all, 
  :conditions => "status = 'open'")
Order.scoped_by_customer_id(12).scoped_by_status("open")

There’s some further discussion of this over on Ryan Daigle’s blog. commit

Other Active Record Updates

There were a few changes going on in Active Record this week. A trio of commits cleaned up some behavior of associations when the :primary_key option is specified. commit commit commit

On another front, ActiveRecord::Base#new_record? now returns false rather than nil when confronted with an existing record. While there was some discussion of the wisdom of this change, the consensus seems to be that it can’t hurt and might make things less surprising for some developers. commit

HTTP Digest Authentication

Rails now has built-in support for HTTP digest authentication. To use it, you call authenticate_or_request_with_http_digest with a block that returns the user’s password (which is then hashed and compared against the transmitted credentials):


class PostsController < ApplicationController
  Users = {"dhh" => "secret"}
  before_filter :authenticate
  
  def secret
    render :text => "Password Required!"
  end

  private
  def authenticate
    realm = "Application"
    authenticate_or_request_with_http_digest(realm) do |name|
      Users[name]
    end
  end
end

commit

Multiple Conditions for Callbacks

When using Active Record callbacks, you can now combine :if and :unless options on the same callback, and supply multiple conditions as an array:


before_save :update_credit_rating, :if => :active, 
  :unless => [:admin, :cash_only]

commit

Testing and continuous integration

A little flurry of activity cleaned up some loose ends in our testing strategy for Rails itself. This included not running symlink tests on Windows, adding test coverage to Rails::TemplateRunner, removing some assumptions in various tests, and getting the FCGI and sqlite2 tests working again. This is all part of a longer-term effort to make the Rails continuous integration server more useful moving forward. As you’ll see if you peek at the current build status, we’re not quite there yet, but we’re getting close.

By the way, if you want to set up your own CI server for Rails, there are instructions right in the source code.

Code Comments for Metaprogramming

One side effect of the changes to respond_to is that people really liked the inline comments that make the intent of the class_eval code clear. As a result, we now have similar comments throughout the Rails source code. For example:


def #{method_name}     # def year
  time.#{method_name}  #   time.year
end                    # end

If you’re just using Rails, you’ll never see these comments – but if you’re helping to maintain and improve the framework, you’ll appreciate them. commit

This Week in Edge Rails

December 20-December 26, 2008 Edition

The biggest change in Rails in recent memory isn’t reflected in edge Rails yet: I’m speaking, of course, about the merger of Merb into Rails 3 . There is a 3-0-unstable branch in the repository, but it hasn’t yet started to diverge from the main line of development. I’ll continue to focus on the master branch, which will be released as Rails 2.3, for the time being.

And Rails 2.3 is still cooking along. The team managed 39 commits this week, despite people taking holiday time off. Many of those were minor bug fixes, but here are a few things you might want to track in the new development.

Unified rendering

ActionController::Base#render is a lot smarter about deciding what to render. You can just throw things at it and expect to get the right results. If you’re using Rails 2.2, you often need to supply explicit information to render:


render :file => '/tmp/random_file.erb'
render :template => 'other_controller/action'
render :action => 'show'

Now in Rails 2.3, you can just supply what you want to render:


render '/tmp/random_file.erb'
render 'other_controller/action'
render 'show'
render :show

Rails chooses between file, template, and action depending on whether there is a leading slash, an embedded slash, or no slash at all in what’s to be rendered. Note that you can also use a symbol instead of a string when rendering an action. Other rendering styles (:inline, :text, :update, :nothing, :json, :xml, :js) still require an explicit option.

commit commit
commit
commit

ActiveRecord fixes

A couple of fixes to ActiveRecord get rid of failing cases for associations. One handles quoting table names in some has_many :through associations – if the table name contains a SQL keyword, then you can’t use it in such an association in Rails 2.2. commit

The other fix allows you to once again use a hash in conditions for a has_many relationship:


has_many :orders, :conditions => {:status => 'confirmed'}

That worked in Rails 2.1, fails in Rails 2.2, and will now work again in Rails 2.3 (if you’re dealing with this issue in Rails 2.2, you can use a string rather than a hash to specify conditions). commit

Some side effects of calling Model#last (it would change the order for other finders within the same scope) have been removed. commit

Prompts for Date Select Helpers

With this patch, you can supply custom prompts for the various date select helpers (date_select, time_select, and datetime_select), the same way you can with collection select helpers. You can supply a prompt string or a hash of individual prompt strings for the various components. You can also just set :prompt to true to use the custom generic prompt:


select_datetime(DateTime.now, :prompt => true)

select_datetime(DateTime.now, :prompt => "Choose date and time")

select_datetime(DateTime.now, :prompt => 
  {:day => 'Choose day', :month => 'Choose month', 
   :year => 'Choose year', :hour => 'Choose hour', 
   :minute => 'Choose minute'})

commit

Odds and Ends

The dbconsole script now lets you use an all-numeric password without crashing. commit

You can now use symbols for the :type option of send_file and send_data, like this: send_file("fabulous.png", :type => :png). commit

If you’re using Active Support delegates, the new :allow_nil option lets you return nil instead of raising an exception when the target object is nil. commit

You can now specify a particular timestamp for updated_at timestamps: cust = Customer.create(:name => "ABC Industries", :updated_at => 1.day.ago) commit

Merb gets merged into Rails 3!

It’s christmas, baby, and do we have a present for you. We’re ending the bickering between Merb and Rails with a this bombshell: Merb is being merged into Rails 3!

We all realized that working together for a common good would be much more productive than duplicating things on each side of the fence. Merb and Rails already share so much in terms of design and sensibility that joining forces seemed like the obvious way to go. All we needed was to sit down for a chat and hash it out, so we did just that.

What this will mean in practice is that the Merb team is putting their efforts into bringing all of the key Merb ideas into Rails 3. Yehuda Katz will outright join the Rails core team, Matt Aimonetti will work on a new evangelism team, and Carl Lerche and Daniel Neighman (hassox) will be co-starring the effort to bring all this over. We’ve immortalized the merge with plaque page at rubyonrails.org/merb.

What’s being brought over?
Some of the key ideas that they’ll be taking with them from Merb into Rails 3 are:

  • Rails core: Yes, Rails is a full-stack framework and will remain so, but there’s no reason we shouldn’t also make it possible to run with less than the full monty. Rails 3 will make it easy to run just a bare minimum and then allow you to opt in just the stuff you want, if that’s necessary for your particular situation. Think “rails myapp —core” (and “rails myapp —flat”).
  • Performance optimizations: Merb has a lot of Rails pieces rewritten to be faster. We’ll be bringing all that good stuff over. We’ll also bend the architecture in the places where that’s necessary for a big yield. In short, Rails 3 will get all the performance attention that the Merb guys are known for.
  • Framework agnosticism: Rails will always have a default answer to every question within the stack. If you don’t care about testing frameworks, you’ll get test/unit. If you don’t care about which ORM, you’ll get Active Record. But some people do care and want something else. Some people want RSpec for testing, others want to use Sequel or Data Mapper for ORM, others again prefer Haml for templating, and some might prefer jQuery for Ajax. All these people should feel like Rails is welcoming them with open arms. Yes, we’ll have a default, but we shouldn’t have any form of discrimination against alternatives.
  • Rigorous API: Too many plugins break when Rails is updated because it’s not clear where they can safely hook into the internals and when they’re monkeypatching and should expect things to break. The Merb guys committed to a public API with tests to ensure that it wouldn’t break. They’ll bring over that line of thinking and give Rails 3 a tested and documented API for extensions that won’t break willy-nilly with upgrades.

This is not a big bang rewrite
It’s important to understand, however, that this is not a “big bang” rewrite of Rails. We’re far beyond the time when we could just throw out everything and start over. This is going to be a progressive improvement of Rails that’ll carefully judge new initiatives on their impact on backwards compatibility as well as their general utility.

I’m sure there’ll be some parts of Rails 3 that are incompatible, but we’ll try to keep them to a minimum and make it really easy to convert a Rails 2.x application to Rails 3. The Merb guys will also be working hard on giving existing Merb users a manageable upgrade path to Rails 3. We’re working with lots of ideas including allowing existing Merb controllers to be mounted alongside new Rails 3 ones. We’ll see how it all plays out, but play out it will.

Also, the Merb guys aren’t just abandoning the existing Merb user base and their applications. They’ll still be doing bug fixes, security fixes, and work on easing the upgrade path to Rails 3. This will all progress in a nice, orderly fashion.

The timeline
Rails 2.3 is just around the corner. We hope to wrap up and release in January. It’s a blockbuster release packed with goodies to the tilt. But as soon as that’s done, all eyes will be on Rails 3.

The probably-overly-optimistic goal is to have at least a beta version ready for RailsConf 2009 in Las Vegas. Who knows if we’ll make it, but we’ll certainly have made tons of progress on it by then.

So all of these changes are pretty much effective immediately. We’ve already started the collaboration and we’ll be rolling out a bunch of public initiatives announcing the concrete elements of the work under the Rails 3 milestone very shortly.

No hard feelings, just kumbaja
This is quite a dramatic turn of events. We went from testy relations to coming together in not very long at all. But I’ve been incredibly positively surprised at how well everyone on both sides have been gelling behind the scenes. The more we talk, the more we realize that we want the same things. And in the few cases were we do care about something different, it’s usually complimentary.

I really hope that everyone within both communities will deal with this news as gracefully as the key contributors from both camps. Let’s just wipe the slate clean on anything that has gone before and cherish that we can now move forward in unity instead of as fractions of the same ideas.

Rails 3 is going to kick ass.

Also read what Yehuda wrote about this and Carl Lerche and Ezra and Matt.

Performance of Rails Metal

Josh recently added Rails Metal, which has been getting a fair bit of publicity. Metal is a great piece of functionality for those rare cases where the speed of your framework actually matters.

However, people have been reporting 25x speed increase over a regular Rails action, and that just doesn’t seem right. So I decided to do some benchmarking of ‘Hello World’ Rails action v/s Metal. Here are my results :

Rails action

Time per request : 1.244 [ms]
Throughput : 800 request/second

Metal

Time per request : 0.386 [ms]
Throughput : 3000 request/second

You can find more details about benchmark command/code at http://gist.github.com/38080

Of course, these are not very scientific benchmarks and your results may vary a little from what you see here. You should also make sure you run your benchmarks in production mode.

Now, if you compare these results, 3000 r/s against 800 r/s, you may think you’re seeing a 3x performance increase. However, that’d a wrong perception and throughput isn’t the best metric here.

Difference in Time per request is what you should looking at. In my benchmarks, speed increase I get when using Metal is about 1 millisecond. And that’s a constant speed increase I’ll get over a regular Rails action. It’s very important to understand that it’s a constant speed increase. It’ll always be 1 ms for me.

For example, if my Rails action takes 12ms, when I reimplement it all in Metal, it will take about 11 ms and not 4 ms.

To conclude, I’ll just quote DHH :

But for those few, specialized cases where you just need as much raw speed as possible, Metal can be exactly what the doctor ordered. It allows you to have the best of both worlds in one package. The lowest possible overhead for a Rack application when that matters and the full-featured goodness of Action Controller when it doesn’t.

This Week in Edge Rails

December 13-December 19, 2008 Edition

The Rails team hasn’t been hibernating this week: 50 commits hit the edge branch, from a variety of developers. Here’s my pick of the most interesting and significant new core code for the week.

Rails Metal

If you’ve been keeping in touch with Rails at all, you’ve heard about Metal this week: superfast endpoints inside of your Rails applications that bypass routing and Action Controller to give you raw speed (at the cost of all the things in Action Controller, of course). This builds on all of the recent foundation work to make Rails a Rack application with an exposed middleware stack. Rather than explain Metal in more detail, here are places you can read more about it:

Rack-based Session Stores

A big change pushed the underpinnings of Action Controller session storage down to the Rack level. This involved a good deal of work in the code, though it should be completely transparent to your Rails applications (as a bonus, some icky patches around the old CGI session handler got removed). It’s still significant, though, for one simple reason: non-Rails Rack applications have access to the same session storage handlers (and therefore the same session) as your Rails applications. In addition, sessions are now lazy-loaded (in line with the loading improvements to the rest of the framework). commit

A related change provides persistent session identifiers for cookie sessions, with API compatibility with the server-side stores. commit

MIME Type Handling

There are a couple of changes to the code for handling MIME types in Rails. First, MIME::Type now implements the =~ operator, making things much cleaner when you need to check for the presence of a type that has synonyms:


if content_type && Mime::JS =~ content_type
  # do something cool
end

Mime::JS =~ "text/javascript"        => true
Mime::JS =~ "application/javascript" => true

The other change is that the framework now uses the Mime::JS when checking for javascript in various spots, making it handle those alternatives cleanly. commit commit

Active Record Cleanup

Even though Active Record has been a core part of Rails basically forever, people are still eking out performance and usability gains here and there. This week saw commits to stop generating some useless queries when working with belongs_to associations (commit), give better error messages on failed find_by_foo! calls (commit), fix some association preloading issues (commit and commit), and improve performance with the MySQL adapter (commit).

Odds and Ends

Remember the cleanup for noise in the log files that edge got a couple of weeks ago? Building on that, we’ve now got prettier printing for output from any Gem in the backtrace. commit

If you’re on edge, Rails now enforces the requirement for Mocha 0.9.3 or higher, so that you can run the tests. commit

ETag handling has been cleaned up a bit: Rails will now skip sending an ETag header when there’s no body to the response or when sending files with send_file. commit

If you want to track down who worked on Rails in the past, it’s gotten easier thanks to a huge data collection and patching effort by Xavier Noria: he went through all of the changelogs and normalized author names, so we don’t have the confusing mash of names, handles, and email addresses in there any more. commit

Introducing Rails Metal

Rails Edge adopted Rack a while back and we’ve been exploring ways to expose that better. The first thing we did was to make it really easy to hook up any piece of Rack middleware in front of a Rails request. In your config/environment.rb file, you can do:

config.middlewares.use(Rack::Cache, :verbose => true)

This will make your application use Ryan Tomayko’s excellent Rack::Cache middleware for better HTTP caching. There are lots of other great generic, reusable Rack middleware pieces. Do go shopping.

But there’s another use case where Rack also comes in handy. If you have a Rails application that has service end points that need to be really, really fast. So fast that the few milliseconds that a trip through the Rails router and Action Controller path is too much.

For this scenario, we’ve built a thin wrapper around the generic Rack middleware and given it a place in the hierarchy along with the name “Metal”. Rails Metal is meant for things like the Campfire poller service, which is doing hundreds of requests per second and needs to be really fast and is at the same time really simple (2 database calls and returning a string). This service is currently written in C, but we’ll look into converting it to Rails Metal instead.

Rails Metal is a subset of Rack middleware
Think of Rails Metal as a subset of middleware intended for application-specific end points that need the extra speed (“write to the metal”, hence the name). Thanks to recent adoption of Rack sessions, these pieces of metal even have access to the same session as Rails. You can either run them as part of the Rails process or run them as individual services through rackup.

We’ve even included a generator for Metal, which by default will create a stub ala:

class Poller def self.call(env) if env[“PATH_INFO”] =~ /^\/poller/ [200, {"Content-Type" => “text/html”}, “Hello, World!”] else [404, {"Content-Type" => “text/html”}, “Not Found”] end end end

As you can see, all this does is capture requests to /poller and return “Hello, World!” (and the request is not for /poller, it’ll let the Rails router handle it — that’s what the 404 return does). It will do that just about as fast as is technically possible in Rack (cue useless metric of how Rails now has the fastest, most meaningless “hello world” stat!!).

Note that this is a pretty specialized tool. While you could write an entire Rails application using just metal, it would be a terrible idea in most cases. The vast majority of applications out there will never do what they do so fast that the overhead of the already heavily optimized Rails router and Action Controller execution path will matter one bit (pick up Michael Koziarski’s talk on Rails Performance through torrent for more on this).

Also, if you’re writing a generic filter in the vein of Rack::Cache, Metal is not the right choice either. You should just be creating regular middleware which can be shared across all Rack applications.

But for those few, specialized cases where you just need as much raw speed as possible, Metal can be exactly what the doctor ordered. It allows you to have the best of both worlds in one package. The lowest possible overhead for a Rack application when that matters and the full-featured goodness of Action Controller when it doesn’t.

We hope you like this. Josh Peek has putting tireless hours into getting Rails on Rack and this is just the latest product of his efforts. Thanks a bunch, Josh!

Update: Jesse Newland wrote up a great introduction to Metal as well. It includes benchmarks and even more examples!

Update 2: Changed code example to match latest style.