Edge Rails: PATCH is the new primary HTTP method for updates

What is PATCH?

The HTTP method PUT means resource creation or replacement at some given URL.

Think files, for example. If you upload a file to S3 at some URL, you want either to create the file at that URL or replace an existing file if there's one. That is PUT.

Now let's say a web application has an Invoice model with a paid flag that indicates whether the invoice has been paid. How do you set that flag in a RESTful way? Submitting paid=1 via PUT to /invoices/:id does not conform to HTTP semantics, because such request would not be sending a complete representation of the invoice for replacement.

With the constraints of the methods GET, POST, PUT, DELETE, the traditional answer is to define the paid flag of a given invoice to be a resource by itself. So, you define a route to be able to PUT paid=1 to /invoices/:id/paid. You have to do that because PUT does not allow partial updates to a resource.

Now let's think about ordinary edit forms in typical Ruby on Rails applications. How many times are we sending a complete representation for replacement? Not always, perhaps we could say that it is even rare in practice that you do so. For example, the conventional created_at and updated_at timestamps normally can't be set by end-users, though they are often considered to belong to the representation of resources that map to records.

PUT in addition is an idempotent method. You should be able to replay a request as many times as you want and get the same resource, something that sometimes is violated by conventional idioms for creating children resources using nested attributes while updating a parent resource.

There's nothing theoretical preventing PUT from doing partial updates, but when HTTP was being standarized the replacement semantics were already deployed.

Because of that, the PATCH method was defined in 1995 and standarized later. PATCH is a method that is not safe, nor idempotent, and allows full and partial updates and side-effects on other resources.

In practice, as you see, PATCH suits everyday web programming way better than PUT for updating resources. In Ruby on Rails it corresponds naturally to the way we use update_attributes for updating records.

Thus, PATCH is going to be the primary method for updates in Rails 4.0.

Routing

This is an important change, but we plan to do it in a way that is backwards compatible.

When a resource is declared in config/routes.rb, for example,

resources :users

the action in UsersController to update a user is still update in Rails 4.0.

PUT requests to /users/:id in Rails 4.0 get routed to update as they are today. So, if you have an API that gets real PUT requests it is going to work.

In Rails 4.0, though, the router also routes PATCH requests to /users/:id to the update action.

So, in Rails 4.0 both PUT and PATCH are routed to update.

Forms

Forms of persisted resources:

form_for @user

get "patch" in the hidden field "_method". The RFC is deliberately vague about the way to represent changes in a PATCH request. Submitting a form is perfectly valid, client and server must simply agree on the accepted ways to update a resource.

Let me emphasize that the "_method" hack is a workaround for the limitations in web browsers. As you probably know Rails routes real HTTP methods. That is, actual PUT, DELETE, and now, PATCH requests are routed to their respective actions.

General availability

PATCH requests are available in all places where the rest of the methods are available today. There is a patch macro for the routes DSL, :via understands the symbol :patch. Tests can issue PATCH requests, request objects respond to patch?, etc. Please see the original commit for details (with an important followup here).

Will my web server understand PATCH?

Yes, it should. I have personally tried Apache, nginx, Phusion Passenger, Unicorn, Thin, and WEBrick. They all understood PATCH requests out of the box.

Also, HTTP clients should be in general able to issue PATCH requests. For example in curl(1) you'd execute:

curl -d'user[name]=wadus' -X PATCH http://localhost:3000/users/1

Credits

We would like to thank David Lee for this contribution and endless patience to keep interested in this even after months of discussion.

Also I would like to highlight the quality of the patch itself. It is excellent: code, tests, all docs revised, comments in code revised. Everything carefully and thoroughly updated. An exemplar patch.

What's new in Edge Rails: EXPLAIN

There are some new features related to EXPLAIN in the forthcoming Ruby on Rails 3.2 we'd like to share:

  • Running EXPLAIN manually
  • Automatic EXPLAIN for slow queries
  • Silencing automatic EXPLAIN

As of this writing they are available for the adapters sqlite3, mysql2, and postgresql.

Running EXPLAIN Manually

You can now run EXPLAIN on the SQL generated by a relation this way:

User.where(:id => 1).joins(:posts).explain

The result of that method call is a string that carefully imitates the output of database shells. For example, under MySQL you get something similar to

EXPLAIN for: SELECT `users`.* FROM `users` INNER JOIN `posts` ON `posts`.`user_id` = `users`.`id` WHERE `users`.`id` = 1
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type  | possible_keys | key     | key_len | ref   | rows | Extra       |
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
|  1 | SIMPLE      | users | const | PRIMARY       | PRIMARY | 4       | const |    1 |             |
|  1 | SIMPLE      | posts | ALL   | NULL          | NULL    | NULL    | NULL  |    1 | Using where |
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
2 rows in set (0.00 sec)

and under PostgreSQL the same call yields something like

EXPLAIN for: SELECT "users".* FROM "users" INNER JOIN "posts" ON "posts"."user_id" = "users"."id" WHERE "users"."id" = 1
                                  QUERY PLAN
------------------------------------------------------------------------------
 Nested Loop Left Join  (cost=0.00..37.24 rows=8 width=0)
   Join Filter: (posts.user_id = users.id)
   ->  Index Scan using users_pkey on users  (cost=0.00..8.27 rows=1 width=4)
         Index Cond: (id = 1)
   ->  Seq Scan on posts  (cost=0.00..28.88 rows=8 width=4)
         Filter: (posts.user_id = 1)
(6 rows)

Please note that explain runs the query or queries and asks the database for their respective query plan afterwards. This is because due to eager loading a relation may trigger several queries to fetch the records and their associations, and in such cases some queries need the result of the previous ones.

If the relation triggers several queries the method still returns a single string with all the query plans. This is an output meant for human consumption so we preferred to present everything as a string in a format which is familiar right away rather than a structure.

Automatic EXPLAIN For Slow Queries

Rails 3.2 has the ability to help in detecting slow queries.

New applications get

config.active_record.auto_explain_threshold_in_seconds = 0.5

in config/environments/development.rb. Active Record monitors queries and if they take more than that threshold their query plan will be logged using warn.

That works for anything running find_by_sql (which is almost everything, since most of Active Record ends up calling that method). In the particular case of relations, the threshold is compared against the total time needed to fetch the records, not against the time taken by each individual query. Because conceptually we are concerned with the cost of the call

User.where(:id => 1).joins(:posts).explain

rather than the cost of the different queries that call may trigger due to the implementation.

By default the threshold is nil in the test and production environments, which means the feature is disabled.

The value of that parameter is nil also if the threshold is not set, so existing applications will need to add it by hand if they migrate to 3.2 to be able to enable automatic EXPLAIN.

Silencing Automatic EXPLAIN

With automatic EXPLAIN enabled, it could still be the case that some queries are just slow and you know they have to be. For example, a heavyweight report in the backoffice.

The macro silence_auto_explain allows you to avoid having EXPLAIN run repeatedly in those areas of code:

ActiveRecord::Base.silence_auto_explain do
  # no automatic EXPLAIN here
end

Interpreting Query Plans

The interpretation of the query plans is another topic, these are some pointers:

Happy debugging!

jQuery: New Default

In Rails 3.1 jQuery is going to be the default JavaScript library. Also, RJS has been extracted out. This post explains what that means for new applications, and what to look for while upgrading existing applications.

New Applications

Starting with Rails 3.1

rails new my_app

generates an application with jQuery.

The -j option of the generator lets you choose Prototype and Scriptaculous instead:

rails new my_app -j prototype

Such an application does not have RJS yet available though. From now on prototype-rails is needed for RJS, see below.

Upgrading Applications Using No RJS

Existing applications using no RJS should remove any references to ActionView::Base.debug_rjs in the project. Typically that means deleting

config.action_view.debug_rjs = true

from config/environments/development.rb. Other than that, upgrading should work out of the box as far as these changes is concerned.

Upgrading Applications Using RJS

Existing application using RJS should work out of the box with prototype-rails.

prototype-rails

RJS has been extracted to prototype-rails.

Applications using RJS have to add this line to their Gemfile while working against Rails master before the 3.1 release:

gem 'prototype-rails', :git => 'git://github.com/rails/prototype-rails.git'

prototype-rails is gonna be a gem when Rails 3.1 is out.

prototype-rails is the one who provides now the RJS template handler; the configuration flag ActionView::Base.debug_rjs; the ability to pass a block to link_to_function and button_to_function; the :update option of render, both in controllers and views; and the modules ActionView::Helpers::PrototypeHelper and ActionView::Helpers::ScriptaculousHelper.

Extraction has taken backwards compatibility into account as much as possible. With that goal in mind, everything is put back where it was before, either by reopening classes or modules, or via alias_method_chain when the refactor involved cutting some part of a method out, as happened with render :update.

There's an exception: ActionView::Helpers::PrototypeHelper and ActionView::Helpers::ScriptaculousHelper are no longer ancestors of ActionView::Helpers. They are now injected into ActionView::Base and ActionView::TestCase directly.

When you include a module M into a class C, the class stores a proxy to M in its ancestor chain. After inclusion, if you add methods to M instances of C respond to them. But if you include another module N into M, the ancestor chain of C is not updated. Instances of C won't respond to N's methods. Although a complete dynamic method dispatch would treat both ways of augmenting M equal, that's the way it works as of today.

By the time prototype-rails is loaded ActionView::Helpers has already been included into ActionView::Base and ActionView::TestCase, so we need to inject them directly where they are needed. Standard usage just works, but please take into account that change in case you reopened or used ActionView::Helpers assuming those ancestors.

Before release, prototype-rails is going to provide also a way to bring Prototype and Scriptaculous to any application.

One Last Detail: The xhr Test Helper

The xml_http_request/xhr test helper is a simple convenience method that sets the X\_REQUESTED\_WITH header to "XMLHttpRequest". If the test request has no Accept header, a fixed value of

[Mime::JS, Mime::HTML, Mime::XML, 'text/xml', Mime::ALL].join(', ')

serves as default.

The Accept header in Prototype calls corresponds to that value, but in jQuery it depends on the dataType attribute. Also, in other JavaScript libraries the Accept header may behave differently. What to do with the default?

This method has been left as is. In particular, if your application uses jQuery the default Accept header in the test will be different from the Accept header sent by jQuery. That's some vinegar: Rails encourages you to base interfaces on explicit format parameters. So, for example, best practice is to provide a /users.json endpoint, rather than a /users endpoint with a JSON representation chosen depending on the Accept header.

If you desperately need a matching Accept header in tests you are still able to pass it. And you can also write your own convenience test helpers easily, since a XHR request is just one whose X\_REQUESTED\_WITH header matches /XMLHttpRequest/i.

Try It Out!

If your application uses RJS it would be really helpful that you test it against master with prototype-rails and report any issues you may find. Thanks!

Why HTTP Streaming?

Rails 3.1 is going to support HTTP streaming, aka chunked responses, this post explains what's all about.

What Is HTTP Streaming?

Ordinary dynamic HTTP responses need a Content-Length header. Their timeline look like this:

HTTP request -> dynamic content generation -> HTTP response

Those are three serial steps because normally you need to generate the content in order to be able to know its size, and thus fill the Content-Length header of the response.

HTTP provides an alternative to this schema to be able to flush data as it is produced, known as chunked transfer encoding. That's what we are referring to as streaming in recent commits.

Streamed responses have no Content-Length header. Rather, they have a Transfer-Encoding header with a value of "chunked", and a body consisting of a series of chunks you write to the socket preceded by their individual sizes. Modulus details.

This is an example taken from Wikipedia:

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked

25
This is the data in the first chunk

1C
and this is the second one

3
con
8
sequence
0

Point is, you are able to flush chunks to the socket as soon as you have them, no need to wait for the whole thing to be generated.

When Do Web Browsers Fetch Assets?

Web broswers parse documents as their content is received. When they find an asset referenced, think an image, stylesheet, or script, a request to fetch them is fired. That happens in parallel while the document is being received and processed, no matter whether the content comes chunked or not.

Browsers have limits on the number of concurrent requests they are allowed to do, a global one (typically +30), and another per domain (nowadays typically 4 or 6), but within those limits, requests for getting assets happen as the content is parsed.

Modern clients do not even block on JavaScript files as old ones did, they implement scanners that look ahead for asset nodes and request them. For example, this is the preload scanner of WebKit.

Trivia: While investigating this I discovered by accident that if the MIME type is unclear, for example "text/html" without an explicit charset, then web browsers buffer 1 KB of data firing no asset requests to be able to peek at the content and do an educated guess.

So What's The Benefit Of Streaming?

Streaming doesn't cut latency, neither it cuts the time a dynamic response needs to be generated. But since the application sends content right away instead of waiting for the whole response to be rendered, the client is able to request assets sooner. In particular, if you flush the head of an HTML document CSS and JavaScript files are going to be fetched in parallel, while the server works on generating content. The consequence is that pages load faster.

Followup

Streaming is still being polished for Rails 3.1, expect another post in the future covering its practical aspects in Ruby on Rails applications.

Thanks

Tony Gentilcore provided his insider's guidance into this, thank you very much Tony! Also, thanks a lot to the Browserscope project for their really useful tables.

What's New in Edge Rails

So, Edge Rails is still chugging right along. There are new and interesting fixes, changes, and refactors going on all of the time. So, lets take a look at just a few that've gone in since the last post (it's been a while, I know, I'm sorry!).

ActionView and Helpers

XSS escaping is now enabled by default. This means that if you want to explicitly output HTML to your views, you'll probably have to mark it as html_safe! before sending it through.

<%= 'my <a href="http://www.rubyonrails.org">safe</a> string'.html_safe! %>

Many of the built-in helpers have been updated for this change and if you see an issues with the Rails helpers being incorrectly sanitized, you should create a new ticket.

distance_of_time_in_words has gained 'over', 'about', and 'almost' keywords, thanks to Jay Pignata and John Trupiano. This provides you with an improved level of granularity when approximating the amount time passed. So, instead of just "2 years ago", it can now also report "almost 2 years ago," "about 2 years ago," and "over 2 years ago," depending on the proximity to being exactly 2 years old.

assert_equal "almost 2 years",  distance_of_time_in_words(from, to + 2.years - 3.months + 1.day)
assert_equal "about 2 years",   distance_of_time_in_words(from, to + 2.years + 3.months - 1.day)
assert_equal "over 2 years",    distance_of_time_in_words(from, to + 2.years + 3.months + 1.day)
assert_equal "over 2 years",    distance_of_time_in_words(from, to + 2.years + 9.months - 1.day)
assert_equal "almost 3 years",  distance_of_time_in_words(from, to + 2.years + 9.months + 1.day)

The HTML form helper, fields_for - generally used for nesting additional model forms - now allows for explicit collections to be used, thanks to Andrew France. So, instead of just including all of your blog.posts, you should have it only display your published blog.posts, for example. Or:

<% form_for @person, :url => { :action => "update" } do |person_form| %>
  ...
  <% person_form.fields_for :projects, @active_projects do |project_fields| %>
    Name: <%= project_fields.text_field :name %>
  <% end %>
<% end %>

API Change for content_tag_for: The third argument - being the optional CSS prefix - will now also affect the generated CSS class. This prefix will now be appended to the generated element's CLASS attribute.

<%= content_tag_for(:li, @post, :published) %>
# => <li id="published_post_123" class="published_post">...</li>

ActiveResource and ActiveRecord

Taryn East has added update_attribute(s) methods to ActiveResource. These methods act very similarly to the ActiveRecord methods we already know and love.

Building or creating an object through a has_one association that contains conditionals will now automatically append those conditions to the newly created object, thanks to Luciano Panaro.

class Blog
  has_author :commit_author, :class_name => 'Author', :conditions => {:name => "Luciano Panaro"}
end

@blog.build_commit_author
# => #<Author name: "Luciano Panaro" ... >

Pratik Naik added a new option to ActiveRecord's accepts_nested_attributes_for to :limit the number of records that are allowed to be processed. Also, while we're covering accepts_nested_attributes_for, José Valim as renamed the _delete option to _destroy to better follow what is actually occurring. A deprecation warning has been added to _delete, for the time being.

Jacob Burkhart updated the new autosave option in Rails 2.3 to allow for an :autosave => false, which will disallow saving of associated objects, even when they are new_record?s.

Some Internals

Previously, CDATA elements could be ignored when converting from XML to a Hash, so now, thanks to John Pignata, Hash#from_xml will now properly parse and include CDATA elements values.

Josh Peek has relocated global exception handling into ActionDispatch::Rescue. So, this is now being handled at the Rack middleware level.

And finally, Yehuda Katz and Carl Lerche began work on a Rails::Application object to better encapsulate some of the application start up and configuration details. Also, a good bit of initialization has now gone on to move into this new object.

Remember, if you prefer to have a shorter audio summary of some of this content and more, you should check out the Ruby5 podcast over at Envy Labs; it's released every Tuesday and Friday with the latest news in the Ruby and Rails community.

Photo: Clock Tower by Brian Taylor

What's New in Edge Rails: The Security Edition

It's been a bit over two weeks since the last WNiER ("winner"?) post and in the time since our last visit, Ruby on Rails 2.3.4 was released to fix some reported security issues. It is important that you try to upgrade your applications as soon as possible, or even just apply the provided patches if a full upgrade isn't easily accomplished in your situation.

Along with this release, you're also going to see several bug fixes and enhancements to the Rails framework, coming from many contributors, that have been discussed here over the previous weeks and even a few that are mentioned just below.

Security updates

Michael Koziarski posted fixes (here and here) for cleaning and verifying multibyte (unicode) strings. The problem was reported by Brian Mastenbrook and Manfred Stienstra provided input for the fix. These changes should disallow malformed unicode strings from getting past the HTML escaping logic provided by the form helpers.

Coda Hale reported and also added a patch to Rails, fixing a timing attack vulnerability in ActiveSupport::MessageVerifier. Although not likely to be exploited in the wild, the vulnerability may allow an attacker to forge the signatures which encode your application's cookie store. If successfully broken, an attacker could modify their session objects without altering your application to the change.

There have been some issues reported around the Rails 2.3.4 release, specifically with regard to Ruby 1.9 support. While they have not all yet been fully substantiated, this certainly underscores the importance of having proper test coverage and both a staging and production environment for your applications.

Down to the metal

Yehuda Katz and Carl Lerche put in quite a bit of work around ActionController::Metal and Rack's Middleware, recently. ActionController::Metal now acts as a Rack middleware and at the same time, there is a new ActionController::Middleware class that operates as normal Rack middleware.

And, if that wasn't enough, Yehuda went on to add ActiveModel::Lint. ActiveModel::Lint allows you to determine whether or not an object is compliant with the ActiveModel API, via:

    ActiveModel::Compliance.test(object)

The output is similar to a Test::Unit output and will indicate with which portions of the ActiveModel API the given object is - or more importantly is not - compliant.

If Metal is your thing, you may want to take a look at Yehuda Katz's recent blog post, How to Build Sinatra on Rails 3.

Pour some sugar on me

Quite a few changes, small and large, occurred around ActiveRecord and friends. Most of these cleaned up some existing functionality, either making it easier to use, perform more closely to what would be expected, or even adding some new features that will soon feel like old friends.

Taryn East added a little ActiveRecord-like love to ActiveResource. In this patch, ActiveResource received the familiar first, last, and all shortcut methods for wrapping the basic find method.

Proc and symbol support was added to the validates_numericality_of ActiveRecord validation, by Kane.

For those of you who use the :anchor option when generating URLs, you may notice that after this patch by Jeffrey Hardy, Rails will now execute the to_param method on the object provided as an :anchor.

    @post       = Post.first
    @comment    = Comment.first
    post_url(@post, :anchor => @comment) # => http://www.example.com/posts/1#comment-1

Well, something similar to that, anyway. :) This updates the :anchor options to follow a similar functionality as the other options provided when generating URLs.

José Valim cleaned up some bits in the Rails scaffold. The generated new and edit views will now reference a new _form partial. This is a much DRYer way to go about it, and more closely follows what would likely happen if you were to code it yourself. Also, while he was there, he removed a bit of inline CSS (specifically, a green flash message), in favor of a CSS class and updating the default scaffold stylesheet.

And, probably the most interesting change in this group is the addition of a new ActivRecord#previous_changes method, by Scott Barr. previous_changes allows you to see what changed before the last save in your local ActiveRecord object instance. This is particularly useful when calling after_save methods which might need to know what exactly had changed. I'll let him give you a code sample:

    person          = Person.find_by_name('bob')
  person.name = 'robert'
  person.changes                        # => {'name' => ['bob, 'robert']}
  person.save
  person.changes                        # => {}
  person.previous_changes   # => {'name' => ['bob, 'robert']}
  person.reload
  person.previous_changes   # => {}

Okay, let's do it your way

While a lot of us prefer US English, we (begrudgingly) recognize that we aren't always the center of the universe. As such, there are some more localization updates to report in Edge Rails:

Sven Fuchs added localization support to the ActiveRecord::RecordInvalid exception's error message. Then, Akira Matsuda followed Sven with support for localizing the SELECT tag helper's prompt text (the default being, "Please select").

Finally, this is certainly a welcome addition and potentially a major player in localization support within Rails: Antonio Tapiador del Dujo added a patch which allows Rails plugins to define and maintain their own locale files. All that is necessary for the plugin developer to do is to provide a config/locales/ directory within their plugin and then create their own .rb or .yml files (i.e. en.yml). That means that plugins can now be much more responsible for their own localization support and do not have to modify the application's locale files after installation.

Food for thought

Finally, just a small note that the default, preferred table collation for MySQL has been changed. Previously, Rails defaulted to utf8_general_ci when either the database or the table creation script did not dictate otherwise. Now, that has been changed to utf8_unicode_ci. Certainly worth a note with so many Rails applications using MySQL in their back-end.

Update: Set the attribution of previous_changes to Scott Barr. Sorry, Scott!

Photo: Security at the Hoover Dam by Alex E. Proimos

Three reasons to love ActionController::Responder

A couple weeks ago, I wrote about the newly added ActionController::Responder which summarizes your application behavior for a specified format in just one place. For example, the default html behavior is written as:


class ActionController::Responder
  def to_html
    if get?
      render
    elsif has_errors?
      render :action => (post? ? :new : :edit)
    else
      redirect_to resource
    end
  end
end

Here are three examples of the flexibility this new layer can provide.

1) HTTP Cache

A simple, but quite powerful use for responder is apply HTTP cache to all your resources easily. For simplicity, let's cache just get requests that are not handling a collection:


module CachedResponder
  def to_html
    if get? && resource.respond_to?(:updated_at)
      return if controller.fresh_when(:last_modified => resource.updated_at.utc)
    end
    super
  end
end

2) I18n flash messages

Is a common practice that all controllers, when the create, update and destroy actions are handled with success, a flash message is shown to the user. We could easily remove the flash messages from our controllers and let them be handled by the responder with the help of the I18n framework. And it’s quite straightforward to accomplish:


module FlashResponder
  # If it's not a get request and the object has no errors, set the flash message
  # according to the current action. If the controller is users/pictures, the
  # flash message lookup for create is:
  #
  #   flash.users.pictures.create
  #   flash.actions.create
  #
  def to_html
    unless get? || has_errors?
      namespace = controller.controller_path.split('/')
      namespace << controller.action_name
      flash[:success] = I18n.t(namespace.join("."), :scope => :flash,
       :default => "actions.#{controller.action_name}", :resource => resource.class.human_name)
    end
    super
  end
end

The first question then arises: what if I don’t want to add a flash message in an specific situation? This can be solved using options, since all options sent to respond_with are sent to the responder, we could use it in our favor as well:


class MyResponder < ActionController::Responder
  def to_html
    unless get? || has_errors? || options.delete(:flash) == false
      namespace = controller.controller_path.split('/')
      namespace << controller.action_name
      flash[:success] = I18n.t(namespace.join("."), :scope => :flash,
       :default => "actions.#{controller.action_name}", :resource => resource.class.human_name)
    end
    super
  end
end

And we can invoke it as:


class PostsController < ApplicationController
  def create
    @post = Post.create(params[:post])
    respond_with(@post, :flash => false)
  end
end

3) Instant pagination

Some people already start a project with pagination from scratch, others add at some point. Nonetheless, pagination is more like a rule than a exception. Can that be handled by Rails 3? First, let’s check an index action with respond_with:


class PostsController < ApplicationController
  def index
    @posts = Post.all
    respond_with(@posts)
  end
end

Right now, when we call Post.all, it returns a collection of posts in an array, so the pagination should be done before the collection is retrieved. Thanks to Emilio and his work integrating ActiveRelation with ActiveRecord, Post.all will return an ActiveRecord::Relation that will be sent to the responder:


module PaginatedResponder
  # Receives a relation and sets the pagination scope in the collection
  # instance variable. For example, in PostsController it would
  # set the @posts variable with Post.all.paginate(params[:page]).
  def to_html
    if get? && resource.is_a?(ActiveRecord::Relation)
      paginated = resource.paginate(controller.params[:page])
      controller.instance_variable_set("@#{controller.controller_name}", paginated)
    end
    super
  end
end

However, the code above is definitely smelling. Set the paginated scope seems more to be a controller responsability. So we can rewrite as:


module PaginatedResponder
  def to_html
    if get? && resource.is_a?(ActiveRecord::Relation)
      controller.paginated_scope(resource)
    end
    super
  end
end

class ApplicationController < ActionController::Base
  def paginated_scope(relation)
    instance_variable_set "@#{controller_name}", relation.paginate(params[:page])
  end
  hide_action :paginated_scope
end

As previously, you could make use of some options to customize the default pagination behavior.

Wrapping up

All the examples above were contained in modules, that means that our actual responder has yet to be created:


class MyResponder < ActionController::Responder
  include CachedResponder
  include FlashResponder
  include PaginatedResponder
end

To activate it, we just need to overwrite the responder method in our application controller:


class ApplicationController < ActionController::Base
  def paginated_scope(relation)
    instance_variable_set "@#{controller_name}", relation.paginate(params[:page])
  end
  hide_action :paginated_scope

  protected
  def responder
    MyResponder
  end
end

While those examples are simple, they show how you can dry up your code easily using Responder. And you? Already thought in an interesting use case for it?

What's New in Edge Rails: No REST for the weary

This week's post will be rather short and sweet. The notable commits of the week seemed to revolve mainly around refactoring and even slightly altering the way some of the bits work. Lets get into it:

I'm Partially impressed

Yehuda Katz was able to simplify some of the partial rendering logic this week, although in doing so seems to very slightly alter the call methodology. So now, when calling the local object from within a partial, you will have the choice of using either the partial name (i.e. "_post.html.erb" would be "post") as the reference for the local object, or you may make it unique by passing in the :as option (i.e. render :partial => "post", :as => "poster_boy" would be "poster_boy"). You no longer have the option of using both interchangeably from within your partial. Also, the :object option for render :partial has been removed, in favor of the methods previously mentioned.

We won't Accept it

The way in which Rails handles incoming Accept headers has been updated. This was primarily due to the fact that web browsers do not always seem to know what they want ... let alone are able to consistently articulate it. So, Accept headers are now only used for XHR requests or single item headers - meaning they're not requesting everything. If that fails, we fall back to using the params[:format].

It's also worth noting that requests to an action in which you've only declared an XML template will no longer be automatically rendered for an HTML request (browser request). This had previously worked, not necessarily by design, but because most browsers send a catch-all Accept header ("*/*"). So, if you want to serve XML directly to a browser, be sure to provide the :xml format or explicitly specify the XML template (render "template.xml").

I'll tell you where you can go...

Josh Peek finally removed the deprecated "best fit" route generation support. Mostly for the sake of speed and maintainability, the new router will simply use the first matching route in your routes.rb file, rather than the "best" match. If you've been working at all on the Edge and haven't noticed the deprecation warnings, then this probably won't affect you.

Oh, so that's why!

Jay Pignata provided a patch this week to help out all of you RESTful API developers. Previously, when a client sent invalid XML or JSON to your server, Rails would 500 with an oh-so-descriptive /!\ FAILSAFE /!\ error in your logs. This wasn't much help if you were trying to debug it. So, with this patch, you'll now get to look at the raw data that was posted to your server in your logs.

Speaking of Resources

Finally, a few updates went into ActiveResource this week which will make it a lot more familiar to those of us who are more comfortable with ActiveRecord. Validation support has been added, which will allow you to both validate your resources locally - before transmission - and remotely. ActiveResource no longer throws a ResourceNotFound error when you attempt to find a set of undefined resources [Resource.find(:all)]. Instead, ActiveResource is reverting to the more ActiveRecord-like empty set/nil response. And last, but not least, resource.save! will now raise a ResourceInvalid unless the resource is, actually .. well .. valid?

That's it for this week's update on Edge Rails. I hope that you've enjoyed it.

Photo: Hood River Sunset by Bernt Rostad

What's New in Edge Rails: The BugMash Edition

Another week, another update on Edge Rails. And man, you aren't making this easy on me, are ya? This weekend, in case you hadn't already heard, was the first Rails and RailsBridge BugMash. If my count is correct, there were roughly 300 commits to the Rails master over the past six days, with most of them pushing in over the weekend. Talk about a trial by fire! There's a lot of really good stuff in here, so let's get started:

All I ever wanted was a little Validation

ActiveRecord (or probably more accurately, ActiveModel) received a lot of validation love this weekend.

  • Thanks to James Hill you can now have your validations read from a custom method. This will allow you to more easily validate non-column - instance variable - data, for example.
  • Adam Keys added support for exclusive ranges in validates_length_of. So, validates_length_of :name, :within => (5...10) will actually restrict the valid length to between 5 and 9.
  • Thanks to Zac Williams, validates_length_of :name, :maximum => 10 will now allow nil values to validate, by default.
  • What good is validates_format_of if you can't validates_format_of :without => /.../? Well, not much, let me tell you... Elliot Winkler provided a patch do to just that.
  • And last, but certainly not least Jeff Dean committed a great patch, which now brings to us: validates_with. This little beauty now allows you place validation code into an external class. And that, my friends, means better encapsulation of responsibility, I'll bet on better readability, and even shared validation logic across your application or possibly even gems. Nice job, Jeff.

We're so much more Resourceful

There were quite a number of updates to ActiveResource over the weekend. And, I'm sure many of you are going to be quite thankful for some of these:

  • ActiveResource now supports HTTP proxies thanks to Marshall Huss. This should be highly useful in large corporate and other firewalled or isolated environments.
  • ActiveResource gets SSL options from Roy Nicholson. This allows you to use X509 certificates, SSL timeouts, peer verification, and more.
  • ActiveResource.exists? got some polish from Jatinder Singh. And, by polish, I really mean that now it works. Instead of raising Net::HTTP errors.
  • Fabien Jakimowicz added JSON error reporting support. So, now regardless of whether you're using either XML or JSON, errors will be correctly reported back to you.

There's strength in numbers

Thanks to Dan Cheail, we finally get a grouped_collection_select helper. I mean come on, there's no way you can tell me that you can't love this:

class Continent < ActiveRecord::Base
class Country < ActiveRecord::Base
class City < ActiveRecord::Base

<%= grouped_collection_select(:city, :country_id, @continents, :countries, :name, :id, :name) %>

<select name="city[country_id]">
  <optgroup label="Africa">
    <option value="1">South Africa</option>
<option value="3">Somalia</option>
  </optgroup>
<optgroup label="Europe">
<option value="7" selected="selected">Denmark</option>
    <option value="2">Ireland</option>
</optgroup>
</select>

I have massive Routes

This weekend brought a small, but incredibly useful patch to rake routes by Mike Breen. Now you can filter the listed results by passing in CONTROLLER=foo. That's. just. awesome. Personally, it's gotten to the point that my fingers just assume that they are to append the "| grep foo" portion whenever I type rake routes.. apparently I have nerd muscle memory. *sigh*

Short and sweet

I'll end the BugMash portion here with just a few more commits worth mentioning. Certainly this wasn't the extent of the BugMash and I could probably fill up another post just as long with even more mashed bugs and features, but it's gotta end somewhere, right?

  • Rizwan Reza added support for you to define custom RedCloth options via the textilize helper. So, now, textilize("Testing <b>HTML</b>", :filter_html) will actually filter the HTML!
  • You can now redirect_to(User) - note that User is a class not an instance here - as a synonym for redirect_to(users_url). Thanks to Niklas Holmgren for that.
  • Delivered mail items now have the ability to save to disk because of Eric Davis. When using the new :file delivery method, you can even define your own custom :location for directory storage.
  • And finally, it's not necessarily BugMash-related, but José Valim - among dozens of other commits - added model.destroyed?. This nifty method will return true only if the instance you're currently looking at has been successfully destroyed.

Now that we're clear of those, here are a couple of non-BugMash-related topics which were addressed:

A visit to the Oracle

There's a difference between knowing the path and walking the path. And this week saw some updates to Rails, mostly around testing, adding improved Oracle database support. A large amount of this effort appears to be coming from one man, Raimonds Simanovskis. I know when we talk about databases with Rails, it's always the big four - SQLite, MySQL, PostgreSQL, and that other one which shall not be named - so it's nice to get a little more focus the 800lb. gorilla in the room.

This update fixed failing test cases due to offset and limits, empty strings storing as null, numeric results from ActiveRecord.sum, Oracle not utilizing an 'AS' keyword on joins, and many more.

AbstractController::Responder

José Valim and Yehuda Katz (and many others, I'm sure) have put in some excellent work in refactoring the Rails renderer. In fact, the majority of the rendering code has come to find a home in a common object, the AbstractController::Responder. Certainly, this may not be its final resting place, but it's interesting to note that once a lot of the render logic was brought together the core team was able to quickly identify and refactor the logic to see some dramatic speed increases. At one point, Yehuda even mentioned a tested benchmark showing, "10% faster partial rendering," than Rails 2.3.

I know that in the RailsEnvy podcast I jabbed a few ribs about the amount of time it's taken to see a Rails 3.0 release, but it's important to note that these guys are doing some seriously awesome work. I'm seeing extensive testing and benchmarking and very intelligent refactorings. We've really got some great things yet to come.

I'm sure I've left out several important and/or interesting commits this week. So, I apologize if one of those was yours. I, and the rest of the community, certainly appreciate the effort you all put in this weekend and Rails is certainly better for it. So, thank you, thank you, thank you to all of you BugMashers out there. And, if you missed out on getting your commit in this round, we're certainly ready to welcome you into the next.

If you prefer to have a shorter audio summary of this content, you should check out a new podcast just launched by Envy Labs, called Ruby5; a 5 minute, twice-weekly podcast covering Ruby and Ruby on Rails news.

Photo: Brooklyn Bridge Virtual Tour by Diego_3336

What's New in Edge Rails: The Hodgepodgery

It's been quite a while since we've had a new Edge Rails post. I've really missed them and there have been a lot of changes, both big and small, on the Edge in the time since. In this post I'll cover a little bit of everything that's been happening over the past week or so in Rails. Along with this, you should expect to see more frequent and regular updates on all that is currently going on.

Goodbye SQLite dbfile

This is a relatively minor change, unless you still refer to your SQLite database as a dbfile in your database.yml. The dbfile option key is now gone, having been replaced a long time ago by the more standard, database key. So, if you're suddenly seeing, "No database file specified" errors after you update, this may be your cause.

Кирилица\n祝您好運 ??? We got your Кирилица\n祝您好運 RIGHT HERE

Sava Chankov provided a patch fixing Content-length HTTP response headers being incorrectly calculated with Ruby 1.9 and multi-byte characters. The content length is now calculated based off of the String#bytesize rather than just the String#size. This takes advantage of the multi-byte character support built in to Ruby 1.9, just as long as you remember to always properly tag your multi-byte file with the correct encoding header (i.e. # encoding: utf-8).

That's not our job

Some spring cleaning, mostly made possible through Rack support, allowed Edge to hand-off some additional responsibilities to the stack. The most interesting of which is now off-loading some of the content-length calculation to your web server rather than being processed directly from within Rails. The largest benefit of this is not so much for the application developers as for you middleware developers. This means that you no longer have to re-calculate the content length when manipulating the body of an HTTP response.

This does not appear to affect the previously mentioned update by Sava, because his fix seems to most largely affect streaming file responses directly through Rails.

We <3 the Hash, and now so does the Cache

Cache control is now being handled by an internal Hash rather than independent string values. Old and busted: headers['Cache-Control'] = 'private', new hotness: response.cache_control[:public] = true. Additional cache_control options include: :public, :max_age, :must_revalidate, and the oh-so-descriptive :extras, which is used mostly for your custom header content.

What time is it!?

Geoff Buesing provided a useful fix for Time, specifically when used in conjunction with ActiveRecord. Now you can save and search ActiveRecord objects using whichever local time zone you like, regardless of what your default time zone is configured for and everything now will just work. You no longer need to be concerned about converting your user's local time into your default application time or vice versa.

It's STATE of the art

ActiveRecord now has easy access to ActiveModel's StateMachine implementation. I don't know about you, but somehow I always seem to find a way to bring state machines into my Rails applications, and now building in - even fairly complex - state machines just got a whole lot easier.

For an example of ActiveModel::StateMachine and to get an idea of how you might use it, check out my more detailed blog post over at Envy Labs.

Front. Back. Side to side.

Paul Gillard committed a patch which now allows for both custom suffixes (which really isn't new) and prefixes (omg THAT IS!) on your ActiveRecord attributes. This gives you access to attribute_method_prefix, attribute_method_suffix, and now attribute_method_affix. While it may sound a little silly, check this out:


class Person < ActiveRecord::Base
  attribute_method_affix :prefix => 'me_mateys_', :suffix => '_is_in_pirate?'
  
  private
  
  def me_mateys_attribute_is_in_pirate?(attr)
    send(attr).to_s =~ /\bYAR\b/i
  end
end

person = Person.find(1)
person.name                               #=> 'Paul Gillard'
person.profession                         #=> 'A Pirate, yar!'
person.me_mateys_name_is_in_pirate?       #=> false
person.me_mateys_profession_is_in_pirate? #=> true

Okay, well, that's still a little silly, but hopefully you get the idea and can think of a few usage scenarios of your own. So, basically, now you can add your own dynamic ActiveRecord methods that can potentially affect any or all of it's attributes.

Clean yourself up

Paul Gillard posted another useful patch, utilizing that one previously mentioned, to now provide us all with this syntactic sugar: reset_attribute! And he even provided us with a pirate of his own to show off this new bounty:


pirate = Pirate.create!(:catchphrase => 'Yar!')
pirate.catchphrase = 'Ahoy!'
pirate.reset_catchphrase!
assert_equal "Yar!", pirate.catchphrase
assert_equal Hash.new, pirate.changes
assert !pirate.catchphrase_changed?

Nobody likes a dirty pirate.

This Week in Edge Rails

April 18, 2009 – April 24, 2009

Edge Rails has been undergoing major surgery for the past week, as the core team gets ready for a Rails 3.0 alpha release at RailsConf. We saw about 50 commits to the master branch in GitHub this week. Here’s a quick overview of the recent changes.

Rails 2.3.x Changes

Before digging into the changes on the master branch, there were a few things committed to the 2-3-stable branch. If you’re running on 2.3 edge, these are ready for you; they’re also ported to the master branch already.

  • A couple of bugs involving associations with hash conditions have been fixed, notable to make sure that :dependent => :delete will work.
  • The PostgreSQL Active Record adapter now does the right thing if you use tables in non-default schemas. You need to set the table name in your model class, of course: set_table_name 'other_schema.customers' commit
  • Also in the PostgreSQL adapter, a couple of bugs related to wrong quoting of names with capital letters are fixed.

ActionView::Path Refactoring

One major chunk of change in Rails 3 this week comes from the continued work to refactor Action Pack. This time, ActionView::Path was the target. Changes in commit include decoupling ActionView::Path from Action Controller and Action Mailer, which gives us two major benefits. First, consolidating similar code in one place makes it easier to understand and maintain. Second, by abstracting this stuff and giving it an API, we’ll make it possible for other components to participate in the controller layer, beyond mailers and traditional controllers.

There’s also some work here to set up for the future. The plan is to decouple templates from the file system, and to decouple localization from ActionView. Stay tuned!

Pluggable JSON Backends

You may recall that recently Rails went to pluggable XML support. This week, thanks to work from Rick Olson, we have pluggable JSON. This means that you can replace the default YAML-based JSON support with the JSON gem:


ActiveSupport::JSON.backend = "JSONGem"

As part of this change, Rails now uses ActiveSupport::JSON.encode() wherever it needs JSON. This replaces using #to_json, and Rails team is recommending that you do the same. If you do choose to use #to_json, you need to be aware that some libraries might override it. You can use this pattern to make sure that you’re getting Rails’ own definition of #to_json:


gem 'json'
# JSON gem loaded, which overwrites to_json
ActiveSupport::JSON.backend = "JSONGem"

class ActiveRecord::Base
  # replace the gem's to_json with Rails' own to_json
  alias to_json rails_to_json
end

Active Support à la carte

In Rails 2.x, require "active_support" pulls in all core extensions at once. In Rails 3, require "active_support" only makes the extensions available; you have to explicitly require those you wish to use. Many of this week’s changes were concerned with breaking up Active Support so you can take just what you need and nothing more.

The end result is that it’s easy to cherry-pick features from Active Support without feeling like you’ve invited a portly gentleman for a piggyback ride. As an added bonus, all core extension documentation is now consolidated in one place: the core class. You don’t have to go poking around every extension module to find a method.

Internally, the core extensions have been reorganized so they can be directly required without assuming all of Active Support is available. Want inflections like "car".pluralize? require "active_support/core_ext/string/inflections" and you’ve got them. Note: the implementation and organization will change as we settle on the best way to provide core extensions.

Other Edge Changes

  • The Rails extension to Pathname was dropped, since it was only used in one spot in the code. commit
  • Support for the DRb cache store has been removed. commit
  • The sqlite Adapter joins the PostgreSQL adapter in supporting microsecond resolution. commit

This Week in Edge Rails

April 4 – April 17, 2009

Just in case you missed the news: edge Rails is waking up, with Rails 3 changes starting to show up in the master branch at Github. So there’s a lot to discuss this week. If you want some more of the technical story on the Rails 3 rearchitecting to date, check out the detailed blog postings from Yehuda Katz.

Which Branch Should I Use?

The Rails 3.0 work is being done on the master branch in the Rails repository on Github. If you want to keep up with all the latest changes, this is the branch to track – with the caveat that this is currently alpha code. It’s entirely possible to build a working Rails application using the master branch. The Rails core team is committed to keeping the master branch passing all tests, and in fact improving our test coverage and continuous integration policies are both on tap as Rails 3 improvements. But it’s important to realize that the changes to Rails will result in many plugins needing a rewrite before they will work with Rails 3 (of course, some plugins will need more updating than others). This will make Rails 3 difficult to use for full-scale production deployments for some time to come.

For existing production applications, you should be tracking the changes on the 2-3-stable branch at Github. This branch contains bug fixes and improvements to the 2.3 release code, and should continue to remain compatible with the plugins and code that you’re already using.

Should I be Submitting Patches to Rails?

Absolutely! Rails has always been a community effort and that’s not changing for Rails 3. We welcome patches for either Rails 2.3 or Rails 3.0 – please use the appropriate tag (2-3-stable or 3.0) in your Lighthouse tickets to make it easy to tell which branch you’ve tested and based your patch on. The Contributing to Rails Guide will help you through the mechanics of submitting a patch to Rails.

David outlined the overall vision for Rails 3 back in December, and that’s still the basic plan. If there’s some piece that you’d especially like to get involved with, the Ruby on Rails: Core mailing list will help you get in touch with the developers to coordinate plans – and possibly save some headaches. Most of the major pieces of work are already underway, but we know that community ideas and contributions are essential to the success of Rails 3.

Rails 2.3.x Changes

Speaking of Rails 2.3, the last couple of weeks have seen some bug fixes deployed, and one interesting piece of new functionality: ActiveRecord::Base#touch. The idea is that you get a shortcut to push the current time into the updated_at or updated_on timestamp field in a record:


@customer.touch

What makes this more than a savings of a few keystrokes is that it’s also implemented as an option for belongs_to associations:


class Order < ActiveRecord::Base
	belongs_to :customer, :touch => :last_order_update
end

With this declaration in place, saving or destroying an order object will touch the corresponding parent customer object. This can come in very handy when you’re trying to invalidate a cached copy of the parent in that situation.

commit commit

Rails 3.0 Changes

ActiveRecord::Base#touch was also added to Rails 3.0 at the same time that it went into the 2.3 branch. In general, this should be the case: new functionality from Rails 2.3 will carry over into Rails 3. The core team is working hard to ensure easy upgrades and compatibility when the time comes to make the version change. But there were a lot of other changes this week that only apply to Rails 3: that’s what the rest of this posting is about.

Module Organization

There are spots in the Rails code where there are multiple levels of inclusion and inheritance and setup. As part of making it easier to follow what’s going on, Rails 3 is introducing new depends_on, use, and setup abstractions. These can make the intent of code more obvious on reading:


module AbstractController
  module Helpers
    depends_on Renderer
 
    setup do
      ...
    end
    ...
  end
end

module ActionController
  class Base2 < AbstractBase
    use AbstractController::Callbacks
    use AbstractController::Helpers
    ...
  end
end

commit More information

AbstractController

Probably the biggest architectural change so far in Rails 3 is the introduction of the AbstractController base class. This is a new implementation of the commonalities between ActionController::Base and ActionMailer::Base, and will eventually serve as a base class for the Rails analog of Merb’s parts. The intent is not to provide code that plugins or applications will use directly, but the build a solid low-level API that other things can depend on. For example, a consumer of AbstractController will be able to implement its own path-finding logic for templates, or add additional options to the basic render method.

commit commit More information

ActionDispatch

Another internal architectural change is the introduction of ActionDispatch. A new member of the Action Pack family, ActionDispatch exists as a home for all of the various HTTP libraries and middleware that were previously stuffed into ActionController, including such things as request handling, parameter parsing, status codes, and our bundled copy of Rack. commit

Defining the Surface Area

One of the goals of the Rails 3 work is to cleanly distinguish between public API methods that plugins and application code can rely on, and Rails internals that are subject to change. For an example of this, you can look at some of the refactoring that’s been done to ActionView::Template. Here, the change is to identify the methods that aren’t used by other classes, and to make them private. This immediately makes the intended API of the class more graspable, and in the long run will help give us a contract with plugins as to how they should interact with core code. After the current surface area has been formally identified, it’s likely that it will be reduced to result in a cleaner external API. commit

AJAX and authenticity tokens

If you’ve been writing much javascript by hand, you’ve discovered the nuisance of needing to include the Rails authenticity token in your requests. A closer analysis of the issues shows that there’s no need for these tokens in AJAX requests, because the same origin policy protects them. So, Rails 3 no longer checks for those tokens on AJAX requests, and that’s one less ugly bit of javascript for you to write. commit

Test Improvements

Another area of focus in Rails 3 is to bring the internal tests up to current best practices. There are a number of spots in the existing code where the tests are less systematic than they could be – changes have been checked in with test coverage, but no one has gone through and tried to ensure full test coverage of all normal “happy” code paths. For a glimpse of what’s happening in this area, check out some of the test files in actionpack/test/new_base (the new_base code is a rewrite-in-progress of ActionController::Base). There’s also active work going on to improve our continuous integration story, and the get together some basic high-level integration tests that can exercise Rails as a whole.

Loose Ends

There are some Rails 2.3 features that are currently not implemented in master. Some changes could not be merged cleanly with the new architecture, others are being reimplemented in different ways. You’ll find some specifics in this commit message, but the missing pieces include the performance boost for development mode rendering, template recompiling in production, and some template-picking logic. The developers working on Rails 3 have indicated that they intend to get back to feature parity as quickly as possible. In fact, a first pass over the rails-dev-boost changes has already been committed.

Removed: CGI Support

Rails 3 will not support direct CGI dispatching. This was deprecated in Rails 2.3, so it should be no surprise to anyone that it’s being removed entirely. If you need to use CGI for some reason, though, remember that it’s still supported through Rack. commit

One Giant Leap Forward

As of a few minutes ago, the first big bunch of Rails 3 changes were merged back to Rails master branch in github. This represents a good deal of work by Yehuda Katz, Carl Lerche, and others. I’ll have more to say on the changes later on this week, but for now, just a quick caution: if you’re in the habit of using Edge Rails, you should likely switch to the 2-3-stable branch for the time being. The master branch of Rails will likely take a little while to settle back into something that can be used for anything other than getting a preview of the Rails 3 excitement.

This Week in Edge Rails

March 28, 2009 – April 3, 2009

Things are still fairly quiet out on the edge: 5 commits this week. The work for Rails 3.0 hasn’t been merged back to the master branch yet, so if you need any of the post-2.3 patches you can still just update to edge without worrying about major upheaval.

Changes

  • When are 24 hours not 24 hours? When you add them across a DST boundary in Rails – at least, until one of this week’s commits. You can read more details of what was going on over at the Lighthouse ticket.
  • Looked at your development mode /rails/info/properties view lately? Look again after this commit and it will show all of the installed Rack middleware pieces in your application.

This Week in Edge Rails

March 21, 2009 – March 27, 2009

It’s been a quiet week for Rails on the surface: just half a dozen commits over the course of the week. Behind the scenes, though, the work is going on to merge initial Rails 3.0 work into the master branch of the git repository. We’ll have more news on that as it happens.

Fixes

Meanwhile, a few small issues in the 2.3 release have been cleared up in edge:

  • Tests with multiple POST requests in the same test block now properly handle parameters. commit
  • render :file with absolute paths now works on Windows systems. commit
  • Template extension parsing is now more robust (if you’ve been having problems with files like show.erb.orig, this one is for you). commit

Just a reminder, if you’re having any issues with Rails 2.3, we’d love to know about them – and we’d love it even more if you’d supply a failing test or a patch. Check out the details in the Contributing to Rails guide.

This Week in Edge Rails

March 14, 2009 – March 20, 2009

The big news in Rails this week, of course, was the release of Rails 2.3. But that certainly doesn’t mean the Rails edge story is over! To the contrary, we’re embarking on one of the more ambitious and exciting Rails projects of all: the creation of Rails 3.0. Read on to see where things stand.

Final 2.3 Changes

A few things went in to Rails 2.3 in the days leading up to release. These include:

  • DDL transactions for SQLite databases commit
  • Compatibility between render :file and Pathname commit
  • ActionController class naming conventions for Metal commit

Rails 2.3.2.1

Shortly after the release of Rails 2.3, which was version 2.3.2, it became necessary to make a Rails 2.3.2.1 tag. This is because the tagged 2.3.2 version in the Rails repository is actually missing an important fix (the installable gem version of Rails has the fix). The net result is that rake rails:freeze:edge RELEASE=2.3.2 would freeze a bad version of Rails into your application.

To fix this, the Rails team has re-tagged the master tree at a safer spot, after the critical fix. This new tag is for release 2.3.2.1. So if you’re freezing Rails 2.3 into your applications (as opposed to running it from gems) be sure to use rake rails:freeze:edge RELEASE=2.3.2.1. That .1 makes all the difference.

The Road to Rails 3.0

Now that 2.3 is out, what’s next? Rails 3.0, which has been a distant speck on the horizon for a while, is rapidly getting closer. The Rails core team is discussing exactly how to proceed, but the bottom line is that you are shortly going to see a lot of changes on edge Rails, as work that’s been going on in various forks gets merged back into the master branch. You’ll want to be cautious about using edge on existing applications. In particular, changes to the Rails internals may result in many plugins needing to be rewritten. Rails edge will continue to be the cutting-edge solution, but you’ll need to keep up with the changes and be prepared to work with them if you choose to run on edge.

But this doesn’t mean that Rails 2 is frozen in time either. There’s a new 2-3-stable branch in the Rails repository which will host any maintenance releases to the current release version. There will continue to be some work on making sure the 2.x releases of Rails work well, though the center of gravity of Rails framework development will shift quickly to Rails 3.0.

So stay tuned. We’ll continue to keep you posted with Rails 3.0 developments as they happen: the process will continue, as always, to be transparent and to welcome ideas and feedback.

This Week in Edge Rails

March 6, 2009 – March 13, 2009

Things have been pretty busy on the development side since the release of Rails 2.3 RC2. The core team has been making a serious effort to review all of the open bugs and patches with an eye towards getting us a solid release. At this point, the bar for new features is set fairly high, but even so, there have been an incredible 94 commits in the week since RC2 – mostly fixes to ensure expected behavior and stability. Here are some of the highlights.

Swappable Parsers for XMLmini

The support for XML parsing in ActiveSupport has been made more flexible by allowing you to swap in different parsers. By default, it uses the standard REXML implementation, but you can easily specify the faster LibXML or Nokogiri implementations for your own applications, provided you have the appropriate gems installed:


XmlMini.backend = 'LibXML'
XmlMini.backend = 'Nokogiri'

commit commit

rake gem Task Rewrite

The internals of the various rake gem tasks have been substantially revised, to make the system work better for a variety of cases. The gem system now knows the difference between development and runtime dependencies, has a more robust unpacking system, gives better information when querying for the status of gems, and is less prone to “chicken and egg” dependency issues when you’re bringing things up from scratch. There are also fixes for using gem commands under JRuby and for dependencies that try to bring in external copies of gems that are already vendored.

commit commit commit

Routing Fixes

A couple of small fixes to the routing engine. First, member routes with requirements now work (previously the requirements were ignored):


map.resources :orders, 
  :requirements => { :id => %r([^/;,?]+) }, 
  :member => { :accept => :get }

commit

Also, shallow routes now work properly with namespaces (commit) and you can now use the OPTIONS verb in route conditions (commit).

Client-side Caching Improvements

The expires_in, stale, and fresh_when methods now accept a :public option to make them work well with proxy caching.


expires_in 10.minutes, :public => true
fresh_when :last_modified => @user.updated_at.utc, 
  :public => true
fresh_when :etag => @user, :public => true

commit

Odds and Ends

The String#parameterize method now accepts an optional separator character.


"My big duck".parameterize =>      "my-big-duck"
"My big duck".parameterize('_') => "my_big_duck"

commit

The ActiveRecord::Base#invalid? method now works as the opposite of ActiveRecord::Base#valid?. (commit)

The ActiveSupport::Json.decode method now handles \u0000 style escape sequences. (commit)

You can now set content types such as multipart/mixed in Action Mailer. (commit)

Rails 2.3 will ship with a bundled version of Rack, but if you have Rack 1.0 installed as a gem it will use the gem version instead. (commit)

This Week in Edge Rails

February 14, 2009 – February 27, 2009

The sharp-eyed will notice that This Week covers two weeks this week. As Rails 2.3 approaches release, the core team is getting pickier about what goes into the source code, and is concentrating on fixing bugs. You can expect to see Rails 2.3 RC2 very soon, and that may be the last release candidate. So if you haven’t tested your own applications on Rails 2.3 yet, it’s time to do so. The edge code is generally stable right now, so it’s a great time to take a look.

Batch Processing

You can now process large numbers of records from an ActiveRecord model with less pressure on memory by using find_in_batches:


Customer.find_in_batches(:conditions => {:active => true}) do |customer_group|
  customer_group.each { |customer| customer.update_account_balance! }
end

You can pass most of the find options into find_in_batches. However, you cannot specify the order that records will be returned in (they will always be returned in ascending order of primary key, which must be an integer), or use the :limit option. Instead, use the :batch_size option, which defaults to 1000, to set the number of records that will be returned in each batch.

The new each method provides a wrapper around find_in_batches that returns individual records, with the find itself being done in batches (of 1000 by default):


Customer.each do |customer|
  customer.update_account_balance!
end

Note that you should only use this method when you have enough records to justify batch processing: for small numbers of records (less than 1000), you should just use the regular find methods with your own loop.

commit

A Note About Template Loading

After some extensive work by several authors (here’s the Lighthouse ticket), Rails 2.3 includes the ability to enable or disable cached templates for any particular environment. Cached templates give you a speed boost because they don’t check for a new template file when they’re rendered – but they also mean that you can’t replace a template “on the fly” without restarting the server.

In most cases, you’ll want template caching to be turned on in production, which you can do by making a setting in your production.rb file:


config.action_view.cache_template_loading = true

This line will be generated for you by default in a new Rails 2.3 application. But please note: if you’ve upgraded from an older version of Rails, you won’t have this setting in your production.rb and template caching will be off by default. Unless you really need the ability to update templates in production without restarting the server, you should be sure to add this setting when you upgrade.

This Week in Rails 3.0

Now that Rails 2.3 has hit the release candidate phase, some of the development effort is turning to Rails 3.0. With that activity heating up, it’s time to start keeping you all informed as to happenings on the 3.0 version of the Rails source. I’ll still be posting separate “This Week in Edge Rails” information focused on Rails 2.3, so you can keep straight which changes are ready now and which still lie in the relatively distant future.

The Vision

The Rails 3 vision is based on the announcement that was made in December: we’re bringing in the key ideas from Merb to Rails, including:

  • A more modular Rails core, so you can run applications with less than the full Rails stack
  • Performance optimizations
  • Framework agnosticism with sensible defaults
  • A tested and documented API for extensions

Rails 3 promises to substantially advance the state of the art in Ruby web frameworks, while still providing migration paths from Rails 2.x and Merb 1.×.

The Source Code

The Rails 3.0 branch in the main Rails project on GitHub is the place to be to see what’s going on:


git clone git://github.com/rails/rails.git
git checkout 3-0-unstable

As the branch name might tell you, this is still a fairly experimental place to be: you probably don’t want to roll this out for production applications just yet. But it is tested code (and it’s using continuous integration to stay that way), and it already includes substantial changes from Rails 2.x thanks to the efforts of Yehuda Katz, Joshua Peek, and others. The changes so far are focused on cleaning up and improving Rails internals, rather than on adding new features.

Action Dispatch

Action Dispatch is a new Rails component which lives in Action Pack (along with Action Controller and Action View). Action Dispatch is responsible for all the processing involved with dispatching requests: request and response handling, HTTP status codes, file uploads, URL and parameter parsing, session storage, and so on.

Action View Reorganization

There are substantial changes in the Action View internals. The overall goal was to clean up a bunch of special cases and extra hooks that had built up over the years, and to leave all callers into Action View using a single unified entry point. The code cleanup has been coupled with some rearrangement of the Action View source to make it easier to find bits of functionality. This was a substantial effort; if you’re interested in a detailed look at the refactoring, you can read up on it at Yehuda’s blog

Callback Optimizations

A new method of handling callbacks removes the need for iterating through the callback list at runtime, and provides a substantial speed improvement in this area of the code. Though this is a micro-optimization that may not have much effect by itself, the hope is that by carefully optimizing as many hot spots as possible we can get a visible overall speedup in page creation and delivery – which, after all, is the point of a web framework.

What’s Next?

Obviously, there’s a long distance between where we are today and the Rails 3.0 vision. We’re fortunate to have an excellent team of core programmers devoting substantial time to making that journey. The interim goal is still to have a beta version of Rails 3.0 out in time for RailsConf in May. You can help in the same ways as with earlier versions of Rails: download the source, start testing it with your applications, and submit your own ideas and patches to the Rails Lighthouse. Rails has been a joint effort of thousands of developers over the years, and Rails 3.0 will be no different in that regard.

This Week in Edge Rails

January 24, 2009 – January 30, 2009

This week we saw 35 commits in edge Rails – though many are bug fixes and minor things. Here’s one more preview of some of the recent and significant changes.

Localized Views

Rails can now provide localized views, depending on the locale that you have set. For example, suppose you have a Posts controller with a show action. By default, this will render app/views/posts/show.html.erb. But if you set I18n.locale = :da, it will render app/views/posts/show.da.html.erb. If the localized template isn’t present, the undecorated version will be used. Rails also includes I18n#available_locales and I18n::SimpleBackend#available_locales, which return an array of the translations that are available in the current Rails project.

commit
commit

Reconnecting MySQL Connections

MySQL supports a reconnect flag in its connections – if set to true, then the client will try reconnecting to the server before giving up in case of a lost connection. You can now set reconnect = true for your MySQL connections in database.yml to get this behavior from a Rails application. The default is false, so the behavior of existing applications doesn’t change.

commit

Easier Testing for JDBC

To make life easier for anyone using Rails on JRuby, Active Record now includes test tasks for a bunch of database accessible via JDBC: Derby, H2, hsqldb, MySQL, PostgreSQL, and sqlite3 (the latter three are also available through non-JDBC connections, as you know). You need to have the database, the activerecord-jdbc-adapter gem installed, and the specific activerecord-jdbcdatabase-gem for the database you’re testing. Then you can run tests like this: jruby -S rake test_jdbcmysql (with similar tests for the other adapters, of course).

commit

HTTP Digest Authentication Support

This one first appeared a couple of weeks ago, but was reverted due to some problems with the initial implementation. Fortunately, the problems were resolved, and Rails 2.3 will ship with built-in support for HTTP digest authentication. Ryan Daigle published some sample code.

commit

grouped_options_for_select Helper Method

Action View already has a bunch of helpers to aid in generating select controls, but now there’s one more: grouped_options_for_select. This one accepts an array or hash of strings, and converts them into a string of option tags wrapped with optgroup tags. For example:


 grouped_options_for_select([["Hats", 
   ["Baseball Cap","Cowboy Hat"]]], 
   "Cowboy Hat", "Choose a product...")

returns


 <option value="">Choose a product...</option>
 <optgroup label="Hats">
   <option value="Baseball Cap">Baseball Cap</option>
   <option selected="selected" value="Cowboy Hat">
     Cowboy Hat
   </option>
 </optgroup>

commit

This Week in Edge Rails

January 17, 2009 -January 23, 2009

Edge Rails saw 28 commits this week. Here’s a look at some of them. As always, you’ll want to go back to the GitHub commit list if you want to look at every single change. As we near 2.3, many of the commits we’re seeing are bug fixes rather than new features, and I’m generally not covering those here.

More Rack Middleware

The Rack-ification of Rails continues, with more and more Rails code being refactored into Rack middleware. This week, parsers for XML, JSON, and YAML got moved into ActionController::ParamsParser middleware. In the long run, this sort of refactoring will make many of Rails services open to other Rack clients, without every framework needing to reinvent all of the same wheels. commit

Deprecations

If you were one of the people who got used to running script/performance/request to look at performance based on integration tests, you need to learn a new trick: that script has been removed from core Rails now. But don’t worry if you depend on it. There’s a new request_profiler plugin that you can install to get the exact same functionality back.

Also on the deprecation list is ActionController::Base#session_enabled?, which now returns a deprecation warning when you try to use it. But given that sessions are lazy-loaded now, to disable them all you need to do to disable them is to not use them in the first place. commit

Local Caching for All!

Last week, we saw an improvement to caching performance when using MemCacheStore, keeping a local request cache to avoid redundant reads. This week, that work was refactored so that it can be used with any remote store. commit

This Week in Edge Rails

January 10, 2009 -January 16, 2009

24 commits for edge Rails this week (with one patch ported over to the 2.2 branch as well). If you want a pre-release look at Rails 2.3, this is a fine time to install a copy of edge, if you’re not already there. It’s plenty stable enough for test sites, though there are a few rough patches yet.

Nested Transactions in Active Record

Several people contributed to a big patch that gives us nested transactions in Active Record, a much-requested feature. Now you can write code like this:


User.transaction do
    User.create(:username => 'Admin')
    User.transaction(:requires_new => true) do
      User.create(:username => 'Regular')
      raise ActiveRecord::Rollback
    end
  end
  
  User.find(:all)  # => Returns only Admin

Nested transactions let you rollback an inner transaction without affecting the state of the outer transaction. If you want a transaction to be nested, you must explicitly add the :requires_new option; otherwise, a nested transaction simply becomes part of the parent transaction (as it does currently on Rails 2.2). Under the covers, nested transactions are using savepoints, so they’re supported even on databases that don’t have true nested transactions. There is also a bit of magic going on to make these transactions play well with transactional fixtures during testing. commit

MemCacheStore Enhancements

Nahum Wild contributed some work that (inspired by his spandex_mem_cache_store plugin) that enhances the performance of Rails when using MemCacheStore. The basic idea is to keep a per-request local cache of requests sent to MemCacheStore, cutting down on unnecessary reads and leading to better site performance. commit

Making Active Record Callbacks behave

You may recall the spot in the Active Record documentation that says “If a before_* callback returns false, all the later callbacks and the associated action are cancelled.” What you may not know is that this is actually broken in the current version of Rails: if you cancel a before_update or before_create callback, the after_save callbacks still run. In Rails 2.3, this will behave the way that the documentation says it does. commit

Fractional seconds for TimeWithZone

The Time and TimeWithZone classes include an xmlschema method to return the time in an XML-friendly string. As of this week, TimeWithZone supports the same argument for specifying the number of digits in the fractional second part of the returned string that Time does:


>> Time.zone.now.xmlschema(6)
=> "2009-01-16T13:00:06.13653Z"

commit

JSON Key Quoting

If you look up the spec on the “json.org” site, you’ll discover that all keys in a JSON structure must be strings, and they must be quoted with double quotes. As of this week, Rails does the right thing here, even with numeric keys. commit

Test Refactoring

Josh Peek spent some time refactoring various tests inside of Action Pack, including those for query string parsing, JSON parameter parsing, XML parameter parsing, multipart parameter parsing, and URL-encoded parameter parsing. While you won’t see any new functionality as a result of this work, it’s worth shining a spotlight on the often-thankless cleanup that keeps the Rails code in good shape.

Changes to Object#try

After some discussion, the semantics of Object#try have changed slightly: it now raises NoMethodError on private methods and always returns nil if the object is nil. commit

This Week in Edge Rails

January 3, 2009 -January 9, 2009

It was a pretty light week for the edge Rails tree: about 20 commits. We’re starting to see things coalesce for a 2.3 release, though there’s no official release date yet. Here’s some of the highlights of what’s been going on.

AssetTag Timestamp Caching

You’re likely familiar with Rails’ practice of adding timestamps to static asset paths as a “cache buster.” This helps ensure that stale copies of things like images and stylesheets don’t get served out of the user’s browser cache when you change them on the server. You can now modify this behavior with the cache_asset_timestamps configuration option for Action View. If you enable the cache, then Rails will calculate the timestamp once when it first serves an asset, and save that value. This means fewer (expensive) file system calls to serve static assets – but it also means that you can’t modify any of the assets while the server is running and expect the changes to get picked up by clients. commit

Object#tap Backport

Object#tap is an addition to Ruby 1.9 and 1.8.7 that is similar to the returning method that Rails has had for a while: it yields to a block, and then returns the object that was yielded. Rails now includes code to make this available to older versions of Ruby as well. commit

Rack Version Bump

During the week, the Rack project released version 0.9 and Rails was quick to respond – the required version of Rack for Rails is now 0.9. So, if you’re running on edge, it’s time to update your gems. commit

Continuous Integration Setup

If you’re interested in setting up your own continuous integration server to build the Rails source, the embedded instructions have been updated with the latest requirements and clarifications. Even if you want a CI server for something else, they’re worth looking at, as they’ll get you from zero to a running cc.rb instance very quickly. commit

This Week in Edge Rails

December 27, 2008-January 2, 2009

Happy New Year! Apparently the Rails core team was not doing too much partying to end the old year: we had 35 commits hit the edge tree, and some of them involved very substantial work. Here’s my weekly overview of some of the most visible and significant changes.

Optimization of respond_to

In some of the first fruits of the Rails-Merb team merger, Yehuda Katz took a look at the respond_to method, which is of course heavily used in many Rails applications to allow your controller to format results differently based on the MIME type of the incoming request. After eliminating a call to method_missing and some profiling and tweaking, he reports an 8% improvement in the number of requests per second served with a simple respond_to that switches between three formats. The best part? No change at all required to the code of your application to take advantage of this speedup. commit commit

If you want a preview of what else to expect from Rails 3, you might want to dip into Yehuda’s own fork of the Rails tree; I’ll be covering these changes as they make their way back into the master copy of edge Rails.

Dynamic Scopes for Active Record

You know about dynamic finders in Rails (which allow you to concoct methods like find_by_color_and_flavor on the fly) and named scopes (which allow you to encapsulate reusable query conditions into friendly names like currently_active). Well, now you can have dynamic scope methods. The idea is to put together syntax that allows filtering on the fly and method chaining. For example:


Order.scoped_by_customer_id(12)
Order.scoped_by_customer_id(12).find(:all, 
  :conditions => "status = 'open'")
Order.scoped_by_customer_id(12).scoped_by_status("open")

There’s some further discussion of this over on Ryan Daigle’s blog. commit

Other Active Record Updates

There were a few changes going on in Active Record this week. A trio of commits cleaned up some behavior of associations when the :primary_key option is specified. commit commit commit

On another front, ActiveRecord::Base#new_record? now returns false rather than nil when confronted with an existing record. While there was some discussion of the wisdom of this change, the consensus seems to be that it can’t hurt and might make things less surprising for some developers. commit

HTTP Digest Authentication

Rails now has built-in support for HTTP digest authentication. To use it, you call authenticate_or_request_with_http_digest with a block that returns the user’s password (which is then hashed and compared against the transmitted credentials):


class PostsController < ApplicationController
  Users = {"dhh" => "secret"}
  before_filter :authenticate
  
  def secret
    render :text => "Password Required!"
  end

  private
  def authenticate
    realm = "Application"
    authenticate_or_request_with_http_digest(realm) do |name|
      Users[name]
    end
  end
end

commit

Multiple Conditions for Callbacks

When using Active Record callbacks, you can now combine :if and :unless options on the same callback, and supply multiple conditions as an array:


before_save :update_credit_rating, :if => :active, 
  :unless => [:admin, :cash_only]

commit

Testing and continuous integration

A little flurry of activity cleaned up some loose ends in our testing strategy for Rails itself. This included not running symlink tests on Windows, adding test coverage to Rails::TemplateRunner, removing some assumptions in various tests, and getting the FCGI and sqlite2 tests working again. This is all part of a longer-term effort to make the Rails continuous integration server more useful moving forward. As you’ll see if you peek at the current build status, we’re not quite there yet, but we’re getting close.

By the way, if you want to set up your own CI server for Rails, there are instructions right in the source code.

Code Comments for Metaprogramming

One side effect of the changes to respond_to is that people really liked the inline comments that make the intent of the class_eval code clear. As a result, we now have similar comments throughout the Rails source code. For example:


def #{method_name}     # def year
  time.#{method_name}  #   time.year
end                    # end

If you’re just using Rails, you’ll never see these comments – but if you’re helping to maintain and improve the framework, you’ll appreciate them. commit

This Week in Edge Rails

December 20-December 26, 2008 Edition

The biggest change in Rails in recent memory isn’t reflected in edge Rails yet: I’m speaking, of course, about the merger of Merb into Rails 3 . There is a 3-0-unstable branch in the repository, but it hasn’t yet started to diverge from the main line of development. I’ll continue to focus on the master branch, which will be released as Rails 2.3, for the time being.

And Rails 2.3 is still cooking along. The team managed 39 commits this week, despite people taking holiday time off. Many of those were minor bug fixes, but here are a few things you might want to track in the new development.

Unified rendering

ActionController::Base#render is a lot smarter about deciding what to render. You can just throw things at it and expect to get the right results. If you’re using Rails 2.2, you often need to supply explicit information to render:


render :file => '/tmp/random_file.erb'
render :template => 'other_controller/action'
render :action => 'show'

Now in Rails 2.3, you can just supply what you want to render:


render '/tmp/random_file.erb'
render 'other_controller/action'
render 'show'
render :show

Rails chooses between file, template, and action depending on whether there is a leading slash, an embedded slash, or no slash at all in what’s to be rendered. Note that you can also use a symbol instead of a string when rendering an action. Other rendering styles (:inline, :text, :update, :nothing, :json, :xml, :js) still require an explicit option.

commit commit
commit
commit

ActiveRecord fixes

A couple of fixes to ActiveRecord get rid of failing cases for associations. One handles quoting table names in some has_many :through associations – if the table name contains a SQL keyword, then you can’t use it in such an association in Rails 2.2. commit

The other fix allows you to once again use a hash in conditions for a has_many relationship:


has_many :orders, :conditions => {:status => 'confirmed'}

That worked in Rails 2.1, fails in Rails 2.2, and will now work again in Rails 2.3 (if you’re dealing with this issue in Rails 2.2, you can use a string rather than a hash to specify conditions). commit

Some side effects of calling Model#last (it would change the order for other finders within the same scope) have been removed. commit

Prompts for Date Select Helpers

With this patch, you can supply custom prompts for the various date select helpers (date_select, time_select, and datetime_select), the same way you can with collection select helpers. You can supply a prompt string or a hash of individual prompt strings for the various components. You can also just set :prompt to true to use the custom generic prompt:


select_datetime(DateTime.now, :prompt => true)

select_datetime(DateTime.now, :prompt => "Choose date and time")

select_datetime(DateTime.now, :prompt => 
  {:day => 'Choose day', :month => 'Choose month', 
   :year => 'Choose year', :hour => 'Choose hour', 
   :minute => 'Choose minute'})

commit

Odds and Ends

The dbconsole script now lets you use an all-numeric password without crashing. commit

You can now use symbols for the :type option of send_file and send_data, like this: send_file("fabulous.png", :type => :png). commit

If you’re using Active Support delegates, the new :allow_nil option lets you return nil instead of raising an exception when the target object is nil. commit

You can now specify a particular timestamp for updated_at timestamps: cust = Customer.create(:name => "ABC Industries", :updated_at => 1.day.ago) commit

This Week in Edge Rails

December 13-December 19, 2008 Edition

The Rails team hasn’t been hibernating this week: 50 commits hit the edge branch, from a variety of developers. Here’s my pick of the most interesting and significant new core code for the week.

Rails Metal

If you’ve been keeping in touch with Rails at all, you’ve heard about Metal this week: superfast endpoints inside of your Rails applications that bypass routing and Action Controller to give you raw speed (at the cost of all the things in Action Controller, of course). This builds on all of the recent foundation work to make Rails a Rack application with an exposed middleware stack. Rather than explain Metal in more detail, here are places you can read more about it:

Rack-based Session Stores

A big change pushed the underpinnings of Action Controller session storage down to the Rack level. This involved a good deal of work in the code, though it should be completely transparent to your Rails applications (as a bonus, some icky patches around the old CGI session handler got removed). It’s still significant, though, for one simple reason: non-Rails Rack applications have access to the same session storage handlers (and therefore the same session) as your Rails applications. In addition, sessions are now lazy-loaded (in line with the loading improvements to the rest of the framework). commit

A related change provides persistent session identifiers for cookie sessions, with API compatibility with the server-side stores. commit

MIME Type Handling

There are a couple of changes to the code for handling MIME types in Rails. First, MIME::Type now implements the =~ operator, making things much cleaner when you need to check for the presence of a type that has synonyms:


if content_type && Mime::JS =~ content_type
  # do something cool
end

Mime::JS =~ "text/javascript"        => true
Mime::JS =~ "application/javascript" => true

The other change is that the framework now uses the Mime::JS when checking for javascript in various spots, making it handle those alternatives cleanly. commit commit

Active Record Cleanup

Even though Active Record has been a core part of Rails basically forever, people are still eking out performance and usability gains here and there. This week saw commits to stop generating some useless queries when working with belongs_to associations (commit), give better error messages on failed find_by_foo! calls (commit), fix some association preloading issues (commit and commit), and improve performance with the MySQL adapter (commit).

Odds and Ends

Remember the cleanup for noise in the log files that edge got a couple of weeks ago? Building on that, we’ve now got prettier printing for output from any Gem in the backtrace. commit

If you’re on edge, Rails now enforces the requirement for Mocha 0.9.3 or higher, so that you can run the tests. commit

ETag handling has been cleaned up a bit: Rails will now skip sending an ETag header when there’s no body to the response or when sending files with send_file. commit

If you want to track down who worked on Rails in the past, it’s gotten easier thanks to a huge data collection and patching effort by Xavier Noria: he went through all of the changelogs and normalized author names, so we don’t have the confusing mash of names, handles, and email addresses in there any more. commit

This Week in Edge Rails

December 6-December 12, 2008 Edition

Here’s your weekly roundup of the action out on the edge, where the code is exciting and sometimes even stable. There were only 32 commits this week; perhaps we’re going into a bit of year-end hibernation.

Memory Improvements

A couple of memory leaks got squashed this week. There was one that occurs in Rails 2.2 when you set config.log_level = :warn (this fix has been applied to the 2-2-stable branch as well as to edge) and one that turns up when you call reset_session in an around filter (this fix is on edge, and will be ported to 2.2 soon). commit commit

Application Templates

There was a bit of cleanup work on the application template feature that was introduced last week. This included support for installing plugins as git submodules, a couple of small cleanup items, and a rake task to apply a template to an existing application . With this task, you can now run


rake rails:template LOCATION=~/template.rb

And the changes indicated in the template will be layered on top of whatever code the project already contains.

I18n Cleanup

A few more bits of the Rails infrastructure are now exposed to internationalization, including connectors in Array#to_sentence and separators in Active Record validation messages . Also of interest to the international Rails community is a patch to add an ActiveSupport::Multibyte::Chars#ord method to properly return the Unicode value of the first character in a string.

Odds and Ends

There’s a new version of the memcache-client (1.5.0.5) bundled with edge Rails. This fixes some bugs in the original 1.5 and adds better support for failover and timeouts. You can read more about it on the Lighthouse ticket commit

Active Record’s to_xml support gets just a little bit more flexible with the addition of a :camelize option, which converts all of the column names in the output to camel case. commit

If you test a lot of view code that uses partials, you’ll be happy to know that assert_template now takes a :partial option that can look for them in the output directly. commit

And finally, there’s a patch that has not been applied to Rails yet, but that you might want to go look at in Lighthouse – it adds support for HTTP status code 418 so that Rails can conform to RFC2324 . I’m sure the core team would welcome further discussion of this change.

This Week in Edge Rails

November 29-December 5, 2008 Edition

Rails 2.3 development is still continuing at a healthy clip. Here’s an overview of what’s happened this week.

Application Templates

Rails now incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a template once and use it over and over again when you run the rails command. Pratik Naik provided a thorough writeup of this feature over on his blog. commit

Find Improvements

Even the venerable code used by Active Record to talk to databases can be improved, and we had a couple of improvements checked in this week. First, the SQL generated for preloading records in has_and_belongs_to_many associations has had an extra “AS” excised, which makes it work better with some databases. Also, we now have a :having option on find (as well as on has_many and has_and_belongs_to_many associations) for filtering records in grouped finds. As those with heavy SQL backgrounds know, this allows filtering based on grouped results:


developers =  Developer.find(:all, :group => "salary", 
  :having => "sum(salary) >  10000", :select => "salary")

commit commit

Rack Integration

The tighter integration of Rails with Rack continues. This week saw the death of the venerable CGI processor within Rails, as well as the use of Rack to handle FCGI. There was also some refactoring down in the Rails tests to make them play nicer with Rack. commit commit

Configuration File Cleanup

Over the years, the default environment.rb file had gotten a bit cluttered with comments, random documentation, and settings that mot people don’t need. DHH took a few minutes to clean up the mess, and the current state of this file is much leaner. As part of the cleanup, session store configuration has been moved out to its own file. commit

No More Process Scripts

If you’re one of the (fairly rare) Rails developers who deploys in a fashion that depends on the inspector, reaper, and spawner scripts, you’ll need to know that those scripts are no longer included in core Rails. For Rails applications that are reployed with current alternatives like Passenger, these are just unnecessary baggage. But if you need them, you’ll be able to pick up copies via the irs_process_scripts plugin. commit

Odds and Ends

The fact that Rails checks for IP spoofing can be a nuisance for sites that do heavy traffic with cell phones, because their proxies don’t generally set things up right. If that’s you, you can now set ActionController::Base.ip_spoofing_check = false to disable the check entirely. commit

Rails.root now returns a Pathname object, which means you can use it directly with the join method to clean up existing code that uses File.join. commit

ActionController::Dispatcher now implements its own middleware stack, which you can see by running rake middleware. commit

Gem loading is smarter about missing dependencies. commit

auto_link now properly applies options (such as :target and :class) to generated e-mail links. commit

This Week in Edge Rails

Yesterday was Thanksgiving holiday for US-based developers – but it certainly hasn’t looked like a holiday week in edge Rails. Things are moving fast, with some major changes afoot for version 2.3 of Rails.

Rack integration

The underpinnings of script/server have been simplified and rewritten somewhat. The explicit list of supported servers that used to be in script/server is gone. Instead, Rails now depends on the installation of Rack, and script/server goes through this – which means that Rails supports any server that Rack does.

Efficient routes

Routing sees a couple of big changes this week. The formatted_ route helpers are gone, in favor just passing in :format as an option. This cuts down the route generation process by 50% for any resource – and can save a substantial amount of memory (up to 100MB on large applications, according to the Lighthouse ticket ) If your code uses the formatted_ helpers, it will still work for the time being – but that behavior is deprecated and your application will be more efficient if you rewrite those routes using the new standard. Another big change is that Rails now supports multiple routing files, not just routes.rb. You can use RouteSet#add_configuration_file to bring in more routes at any time – without clearing the currently-loaded routes. commit commit

Better support for engine plugins

The second routing change enables better support for Rails Engines: routing files in engines are automatically loaded and reloaded now (as are those in other plugins). Engines are getting some love other than routing. If your plugin has an app folder, then app/[models|controllers|helpers] will automatically be added to the Rails load path. There’s active discussion of just how this should work, and how much to pick up from the current engines plugins, so it’s likely we haven’t seen the last commits in this area. Engines also support adding view paths now. commit commit commit commit

Sensible backtraces for your tests

If you’re a fan of the Thoughtbot Quiet Backtrace plugin, which allows you to selectively remove lines from Test::Unit backtraces, you’ll be happy to find ActiveSupport::BacktraceCleaner and Rails::BacktraceCleaner in core. This supports both filters (to perform regex-based substitutions on backtrace lines) and silencers (to remove backtrace lines entirely). Rails automatically adds silencers to get rid of the most common noise in a new application, and builds a config/backtrace_silencers.rb file to hold your own additions. commit

Ruby 1.9 integration

A variety of commits continue the drive towards Ruby 1.9 and minitest compatibility. This should ensure that Rails 2.3 is ready to handle the latest Ruby underpinnings when it’s released. Those on the bleeding edge at the moment need to beware, though: one of the changes in edge Rails depends on a ruby-core patch that hasn’t yet been applied there. You’ll also (temporarily) need to pick up Jeremy Kemper’s fork of Mocha for MiniTest compatability as required by this commit .

Faster boot time in development mode with lazy loading/autoload

Jeremy Kemper and Josh Peek have been doing a ton of work on making sure that bits of Rails (and its dependencies) are only brought into memory when they’re actually needed. Check out the commits from November 23 for a bunch of lazy-loading changes. The core frameworks – Active Support, Active Record, Action Controller, Action Mailer and Action View – are now using autoload to lazy-load their individual classes. This work should help keep the memory footprint down and improve overall Rails performance. commit commit commit commit commit

Misc

You can specify using the new preload_frameworks option whether the core libraries should be autoloaded at startup. This defaults to false so that Rails autoloads itself piece-by-piece, but there are some circumstances where you still need to bring in everything at once – Passenger and JRuby both want to see all of Rails loaded together. commit

Asset hosts get more flexible in edge Rails with the ability to declare an asset host as a specific object that responds to a call. DHH has supplied a sample project, asset-hosting-with-minimum-ssl , that demonstrates one good use for this functionality. commit

You can now configure the location of the helpers folder for a Rails application by setting ActionController::Base.helpers_dir. This will be a boon in some unusual circumstances – the original use case is for building a Rails application that encourages extension via plugin rather than by altering the application itself. commit

Token generation for CSRF protection has been simplified; now Rails uses a simple random string generated by ActiveSupport::SecureRandom rather than mucking around with session IDs. As a result, the :digest and :secret options to protect_from_forgery are deprecated and have no effect on edge. commit

While we’re on the subject of secrets, some people will find novel uses for ActiveSupport::MessageEncryptor, which provides a simple way to encrypt information for storage in an untrusted location (like cookies). commit

Active Support’s from_xml no longer depends on XmlSimple. Instead, Rails now includes its own XmlMini implementation, with just the functionality that it requires. This lets Rails dispense with the bundled copy of XmlSimple that it’s been carting around. commit commit

As you probably recall, last week’s improvements included the renaming of application.rb to application_controller.rb. This week there’s a new rake task, rake rails:update:application_controller to do this automatically for you – and it will be run as part of the normal rake rails:update process. commit

Good news if you’re using ActiveSupport::OrderedHash: it now implements each_key and each_value. commit

One more bit of core Rails is open to I18n: the units used by number_to_human_size. If you’re maintaining a translation file, you need to add the storage_units: [Bytes, KB, MB, GB, TB] to your translations. commit

Support for Rails components – which were famously called “a shining example of what happens when eagerness overtakes prudence” in Agile Web Development With Rails – is finally gone. If a couple of years of warning about this deprecation wasn’t enough for you, then it’s time to find an alternate solution at last. commit

Various files in /public that deal with CGI and FCGI dispatching are no longer generated in every Rails application by default (you can still get them if you need them by adding --with-dispatches when you run the rails command, or add them later with rake rails:generate_dispatchers). commit commit

Just a reminder: I’m not providing pointers to every single commit here, just trying to highlight things. This week’s edge changes actually included 136 commits from a wide variety of contributors.

This Week in Edge Rails

First up this week, a warning for those who don’t closely follow the state of the Rails repository – “edge” really means edge now. The bits for 2.2 are getting locked down for release, and the repository has been forked; for the moment, edge Rails is being identified as 2.3 , though that projected version number might change later. If you’re trying to install almost-released 2.2 bits on your machine, make sure you’re using the 2-2-stable branch, and not edge, which is currently seeing some major changes.

The 2-2-stable code is still seeing changes, but they’re either bug fixes or very small things. This week, that includes fixing a bug in assignment to has_one :through associations , some further tuning of CSRF protection , a fix to handling of checkboxes for Boolean attributes , updating the bundled copies of TZInfo, Prototype, and script.aculo.us, and some Ruby 1.9 compatibility work (though currently full Ruby 1.9 compatibility is targeted for Rails 2.3).

The biggest feature change in the 2.2 branch is the addition of explicit I18n support to newly-generated Rails projects, including a sample locale file, auto-loading all locales in config/locales, and sample settings in config/environment.rb. commit

Also worth noting in 2.2 is a chunk of code removal: a whole mess of special case tests for the SQL Server adapter have been chopped out of the Active Record test cases. That’s because Ken Collins has done tremendous work in making the SQL Server adapter work the way that Rails expects data adapters to work, giving us a big step in the area of backend portability. commit

On the actual edge code (the master branch in the repository), there’s a lot more action. With that branch just opened, some pent-up code has been checked in, and some big changes are being made. It’s an exciting time, and edge is definitely worth checking out. Here are some of the most notable changes in the past week.

One big set of changes has come from Jeremy Kemper, who has been overhauling the internal Rails testing to switch from Test::Unit::TestCase to ActiveSupport::TestCase. This work also includes requiring Mocha to test Rails (in the 2.2 code, some tests are skipped if you don’t have Mocha installed) and generally making the Rails testing strategy (both within core and for generated applications) more consistent moving forward.

If you’re one of the people who has always been bothered by the special-case naming of application.rb, rejoice! It’s been reworked to be application_controller.rb in the edge code. More info here and here . commit

Rails 2.3 will introduce the notion of default scopes : similar to named scopes, but applying to all named scopes or find methods within the class. For example, you can write default_scope :order => 'name ASC' and any time you retrieve records from that class they’ll come out sorted by name (unless you override the option, of course). commit

A lot of folks have adopted the notion of using try() to attempt operations on objects – Here’s Chris Wanstrath’s blog post introducing it. It’s especially helpful in views where you can avoid nil-checking by writing code like <%= @person.try(:name) %>. Well, now it’s baked right into Rails. commit

Also new on the syntactic sugar front is Enumerable#none? to check that none of the elements match the supplied block. commit

The render method has been getting smarter over the years, and it’s going to be even smarter in 2.3. If you have an object or a collection and the naming matches up, you can now just do <% render @article %> or <% render @articles %> and things will just work. Ryan Daigle has some more examples on this. commit

On a somewhat similar note, render_component goes from “deprecated” to “nonexistent” in 2.3. If you still need it, you can install the plugin . commit

The autolink helper has been refactored to make it a bit less messy and more intuitive. commit commit

There’s a fix to a memory leak connected to thread safety and asset tags, that could bite sites that were referencing a lot of external images. Aaron Batalion contributed the fix, as well as a blog post explaining the issue. commit and commit

Finally, it’s worth mentioning that some controversy has erupted over a change made to the 2.2 code five months ago – the addition of Array#second through Array#tenth as aliases for Array#[1] through Array#[9]. Without taking a stand on the controversy (I’ve done that elsewhere), I’ll just note that the most recent edge checkin as I write this trims this down to only support Array#second through Array#fifth – and uses the savings in overhead to implement Array#forty-two. commit

This Week in Edge Rails

The important news in edge Rails this week is the imminent release of Rails 2.2.1 – otherwise known as Rails 2.2 RC2. Getting ready for this release did lead to some significant changes in the Rails codebase.

First, it’s very likely that you’ll need to upgrade rubygems to run RC2: the required version of rubygems is now 1.3.1, which was just released yesterday. This dependency is part of the continued work to make vendored gems useful and stable. You may find that updating rubygems is less than smooth, depending on your current version; check out this article if you have any trouble. commit

The Rails routing engine has seen some serious work over the past week as well. For starters, Jeremy Kemper committed several fixes to the core routing engine that cut down on object creation and RegExp creation, trimming memory use. commit commit There are also new :only and :except options for map.resources, which can help cut down memory use if you have a lot of resource routes – see these articles for details (though there have been some tweaks in the way nested limited routes work after those were written). commit commit commit

The new ActiveRecord connection pooling code has seen some tuning as well, making it more efficient in development model and avoiding some issues with the Oracle adapter. commit

Polymorphic URLs now behave more intuitively if one of their parameters is nil. For example, a call to polymorphic_path([project, filter, @issue]) with a nil filter now returns project_issue_url instead of a NoMethodError. commit

The request forgery protection feature in Rails has been tightened up so that it only applies to HTML-formatted content requests. There is substantial discussion on the Lighthouse ticket that led to this change, but the bottom line is that the old implementation had some bugs, notably making destroy actions inaccessible via XML. Other types of requests are protected by other means – for instance, the same origin policy on AJAX requests substitutes for request forgery protection there. commit

This Week in Edge Rails

Rails, as you probably know, is under active development. So, for those of you who don’t have time to read every commit to the source, we’ve decided to revive this section of the weblog. This time around, I’m covering 3 weeks of commits: the time since Rails 2.2 RC1 (otherwise known as Rails 2.2.0) was released. Though there aren’t any major new features being added as Rails drives towards the 2.2 release, that doesn’t mean the source has been completely quiet: there have been about 75 commits in that three-week period. Here’s a look at some of those changes.

In the run-up to 2.2, we’re seeing a batch of little bug fixes, as people try to ensure quality in the release. These include:

  • Squashing a binary data corruption bug that surfaced in the PostgreSQL adapter. commit
  • The regex behind redirect_to can now accept a wider variety of URL schemes, making it possible to redirect to some destinations that were previously inaccessible. commit
  • A regression in date_select and datetime_select that could raise a Null Pointer Exception under some circumstances has been fixed. commit
  • The sanitize helper has been fixed to avoid double escaping already properly escaped entities. commit
  • FormTagHelper has been stopped from generating illegal HTML if the name contains square brackets. commit
  • A memory leak was squashed in Active Record scoped methods. commit

Some of the major features for 2.2 have been getting fine-tuned as well. There’s been work to clean up some loose ends in the thread safety department, and changes to make the I18n backend reload its translations in development mode. The included Prototype bits were bumped to the latest 1.6.0.3 release. The code for configuring, loading, and vendoring gems has had some attention, and the code for maintaining database connection pools has come in for some fine-tuning as well.

Just because we’re in feature freeze doesn’t mean that a few new features can’t sneak in:

  • The current_page method is a bit more reliable now in that it ignores options you don’t explicitly supply (making it more friendly to URLs that use the query string for pagination and the like). commit
  • The default logging has been cleaned up to be less chatty: you’ll see fewer duplicate log messages as Rails goes about its business. commit
  • The render method now takes a :js option to allow you to directly render inline JavaScript without using RJS. commit
  • If you’ve got a current (Ruby 1.8.7 or greater) version of Ruby, Action Mailer turns on STARTTLS if the server supports it; this makes Action Mailer compatible with GMail without the need for plugins. commit

One final note: I’m deliberately not trying to cover every single commit here; just those ones that struck me as most interesting. But if I left out something that you think is highly significant, feel free to add a pointer in the comments!

Hackfest

And once again, hackfest is back. Only this time, it’s better than ever, thanks to Git

The idea is quite simple. You get 5000 points for each of your patches that get merged to the core. The person with the highest points at the end of the month wins.

August hackfest is already on and almost 3 more weeks to go, so hack on for the first prize – a free pass to RailsConf Europe ( special thanks goes to O’Reilly for the prize )

If you’ve never contributed to Rails before, now is a good time. This railscast is a nice head start.

Thanks to all those who have contributed to Rails, making it better and better.

Internationalization in edge Rails and more

There won’t be a Living on the Edge this week, but you won’t be starved for info because the Rails community is keeping up.

Ryan Daigle has been keeping up with some of the changes on edge Rails and has done a few awesome explanatory posts on them:

Perhaps even more noteworthy is the introduction of I18n (internationalization) support to Rails core. Sven Fuchs explains the technical details and API as well as the history of Rails and I18n. I18n support is scheduled to be fully stable in the Rails 2.2 release. Have an interest in internationalization in Rails? Lend a hand with your ideas, feedback, and patches at the Google Group.

Living on the Edge (or what's new in Edge Rails) #3

There hasn’t been much of note in terms of big changes or features in edge Rails lately, so this time I’ll leave you to pore over the Rails commit logs for any bug fixes or minor changes that I haven’t pointed out. There has been some work in progress with ActionPack refactoring and multithreading work as well as some activity in ActiveModel too, but nothing really concrete yet (still very much a work in progress).

As usual, be sure to leave any suggestions and criticisms in the comments.

Thin support with script/server

script/server now checks for the availability of Thin and uses it. Pretty convenient if you are using Thin as your production server (and want to run the same when developing). You’ll have to add config.gem 'thin' to your environment.rb first to get this to work.

This patch was contributed by one of the guys at fluxin.

Changeset

String#humanize can be customized via inflection rules

The String#humanize core extension method is used convert strings with underscore, usually table column names, in them to pretty readable text. For example,

"actor_salary".humanize
=> "Actor salary"
"anime_id".humanize
=> "Anime"

Sometimes this doesn’t work out so well though, when you have legacy tables or simply “inhumanely” named column names like “act_sal_money” (which is really “Actor salary”, but would be #humanize-d to “Act Sal Money”).

You can now specify custom inflection rules (just like you would for plural/singular/irregular/uncountable inflection rules):

Inflector.inflections do |inflect|
  inflect.human /_cnt$/, '\1_count'
  inflect.human 'act_sal_money', 'Actor Salary'
end

Notice how you can also use a regular expression above to convert columns like “click_cnt” to “Click count”.

Thumbs up to Dan Manges and Pascal Ehlert for this patch.

Changeset

Allow conditions on multiple tables to be specified using hash.

Pratik has committed a tiny (but really useful) change to ActiveRecord that allows you to specify conditions on a joined table in its own hash. An example would explain it better:

Anime.all(
  :joins => :character,
  :conditions => {
    :active => true,
    :characters => { :gender => 'female' }
  }
)

The ActiveRecord query above would find all “active” anime with “female” characters.

Changeset

Outro

That’s it for this week’s Living on the Edge – do let me know if you like to see more write-ups on even the minor bug fixes and changes that’d I’d left out this week.

Living on the Edge (or what's new in Edge Rails) #2 - Performance improvements

The first Living on the Edge covered some of the API changes since Rails 2.1, and this time round, I’m going to cover the performance improvements as promised.

Jumping right in…

Faster Erb templates

Jeremy Kemper has made the Erb processing more efficient, especially the concat and capture helper methods.

The “special” Erb _erbout has been replaced with an instance variable, which allows for:

  • better (memory) performance because bindings are no longer being passed around,
  • fewer evals which are usually expensive,
  • there’s no need to slice the _erbout variable when you can swap in a new (string) buffer, and
  • the buffer is actually available via a output_buffer reader and writer methods (so you can override it if you want).

Relevant changesets: 933697a
- 0bdb7d3
- 4d4c8e2

Faster partials and JavaScript helpers

Partial template initialization and JavaScript helpers have been refactored and optimized for speed and efficiency by Jeremy Kemper. These are but a few of the optimizations Jeremy has been committing recently. Be sure to check out some of the commits to Rails (or for that matter, any quality Open Source project) – you could learn something!

Relevant changesets: partialsJavaScript helpers

RecordIdentifier methods speedup

The RecordIdentifier has been sped up by some simple use of memo-ization, thus reducing the number of inflections performed, among other things. The RecordIdentifier is used widely in cache keys, partial template paths, and in most places where you identify an ActiveRecord model without caring about its actual id.

Relevant changesets by Jeremy Kemper: c1a9820566d717

Lazy load cache and session stores

Update: Oops my bad, this change was later reverted in 6573f6a.

The various cache stores in the ActiveSupport::Cache module are now lazy loaded – this means that they are only required when you actually start using them.

Changeset by Pratik Naik

Living on the Edge (or what's new in Edge Rails) #1 - API changes and PerformanceTests

As Gregg Pollack mentioned a week or so ago, I’ll be keeping a weekly-or-so column about noteworthy changes on edge Rails. This is the first time Living on the Edge (of Rails) is appearing on the official Ruby on Rails weblog, so you’ll have to bear with my short introduction.

Living on the Edge is a weekly column I used to put up on my own blog after some prodding by Gregg Pollack of Rails Envy way back in December of 2007. I used to be a rather active Rails contributor back then so it was a no-brainer. Gregg and Jason were awesome enough to feature it weekly in their podcast.

And now it’s here, so try your best to be a tough crowd so I can tune these blog posts so that they are actually useful to you – when I was blogging these on my tiny personal blog it wasn’t that vital but now the audience is significantly larger. Leave your suggestions and criticisms in the comments – they are greatly appreciated!

Anyway, there’s been a ton of new features, API changes and performance improvements in the past 2 weeks or so since Rails 2.1 was released, so rather than dumping all into one mega-post, I’ve decided to break it into 2 posts for new features/API changes and performance improvements. In this post, I’m gonna talk about some of the new features and API changes.

Minor API changes

Let’s start jump straight in with some minor API changes.

link_to now takes a block

The link_to helper now takes a block argument for those occasions when you have really long hyperlink text with variables in them:

<% link_to(@profile) do %>
  <strong><%= @profile.name %></strong> -
  <span>Status: <%= @profile.status %></span>
<% end %>

Some people would find it cleaner than:

<%= link_to "<strong>#{@profile.name}</strong> -- <span>Status: #{@profile.status}</span>", @profile %>

Credit goes to Sam Stephenson (of Prototype fame) and DHH for this change.

Changeset details

ActiveRecord::Base#merge_conditions is now part of the public API

Jeremy Kemper has made ActiveRecord::Base#merge_conditions a public method.

This is pretty useful if you have conditions from multiple sources or like to combine any conditions for any reason.

Post.merge_conditions(
  {:title => 'Lucky ☆ Star'},
  ['rating IN (?)', 1..5]
)
=> "(`posts`.`title` = 'Lucky ☆ Star') AND (rating IN (1,2,3,4,5))"

Do note though that this merges with a SQL boolean AND only (no ORs).

Changeset details

Associations now take a :validate option

Association macros now accept a :validate option like so:

class Anime > ActiveRecord::Base
  has_many :characters, :validate => true
end

This tells ActiveRecord to validate the characters association when saving your Anime model – just like how :validates_associated works. The default is false, which is the current behavior in Rails 2.1 and earlier, so no need to fret. This works for all the other association macros as well (has_one, belongs_to, has_and_belongs_to_many).

Thumbs up to Jan De Poorter and Pratik Naik for this, which also fixes a nasty bug.

Changeset detailsTicket

ActiveSupport::StringInquirer and convenience Rails.env.development? methods

David Heinemeier Hansson (henceforth abbreviated as DHH – sorry!) recently added an ActiveSupport::StringInquirer String subclass that allows you to do this:

s = ActiveSupport::StringInquirer.new('awesome')
=> "awesome"
s.awesome?
=> true
s.sucks?
=> false

An immediate use of this is when you are checking the environment your app is running in: Rails.env is wrapped in a StringInquirer so you can use query methods like Rails.env.development? and Rails.env.production?.

Changeset details

Core extensions: Object#present? and Enumerable#many?

DHH also added some core extensions that while trivial, could make your code more readable. First up is Object#present?, which is essentially !Object#blank?

[].present?
=> false
[1, 2].present?
=> true
"".present?
=> false
"i'm here".present?
=> true

An Enumerable#many? extension was also added that is simply a boolean test for enumerable.size > 1:

[].many?
=> false
[:just_me].many?
=> false
[:just_me, 'my_friend'].many?
=> true

Object#present? changesetEnumerable#many? changeset

Declarative block syntax for writing tests

DHH was inspired by Jay Fields when he committed this bit of syntatic sugar: you can now write your tests (Test::Unit) in declarative block style like so:

test "an anime should be invalid if any of its characters are invalid" do
  # Your usual test code here.
end

I seldom use Test::Unit (except when submitting Rails patches) and prefer RSpec – this declarative style of writing tests is definitely more readable.

All Rails-generated test stubs now use this new syntax.

Changeset details

Performance tests

Jeremy Kemper has been hard at work optimizing and improving the performance of Rails, so it’s no surprise that he has also introduced a new type of integration test: the performance test.

You can use the performance test generator (added by Pratik in 23232a) to generate a performance test stub.

script/generate performance_test LoginStories

Running the performance test requires ruby-prof >= 0.6.1, which is still unreleased but you can get at it the development version by checking out the source and installing the gem yourself (I suggest you get Jeremy’s fork of ruby-prof for now). It’s interesting to note that with the 0.6.1 release, ruby-prof supports profiling tests cases written using Test::Unit.

Moving on… Put in some test code (request a few controller actions – whatever user story you want to test performance of) and run the test. You’ll get output like this (together with the usual ruby-prof profiling output in the test/tmp/performance directory of your Rails app):

> ruby performance/login_stories_test.rb 
Loaded suite performance/login_stories_test
Started
LoginStoriesTest#test_homepage (32 ms warmup)
        process_time: 11 ms
              memory: unsupported
             objects: unsupported
.
Finished in 0.870842 seconds.

The memory and objects results are “unsupported” because I haven’t patched my Ruby interpreter for memory profiling support. You’d need certain Ruby interpreter patches to enable memory and GC profiling. I wish I could tell you more about how to do so, but I’m treading unfamiliar ground here. There are some details here on how to patch Ruby for memory profiling. I leave it for wiser folks to explain how to do this :)

Changeset details

Outro

That’s it so far for new feature/API changes in Rails since Rails 2.1 – performance improvements are coming up in the next post and I’ve also intentionally left out mention of the Rack support that has been partially merged into edge.

If there were any errors or you have any suggestions on how to make this column better, please point them out in the comments. Any info on patching your Ruby interpreter for memory profiling support is also greatly welcome. If I’ve left out anything that I’d considered not noteworthy enough but you disagree, let me know in the comments too.

Rails premieres on GitHub

GitHub has now officially launched and Rails is right there at the premiere. The Rails repository now lives at rails/rails and you can check it out with:

git clone git://github.com/rails/rails.git

If you don’t have git, or don’t enjoy running it on your platform, you need not fear. We’ve set up an automated task to produce a zip file of Rails Edge that’ll be kept up to date all the time: http://dev.rubyonrails.org/archives/rails_edge.zip. This is also what we’ve made the new rake rails:freeze:edge use.

This also means that development on the Subversion repository has stopped and will no longer be kept up to date. We’ll keep the Subversion repository around for some time for people to transition off svn:externals, though. But if you want the latest edge, you’ll have to use either git or the new zip files.

We’ll also soon go live with our new ticket management system, which will be running on a new version of Lighthouse. When that happens, the Trac installation will follow the Subversion repository into legacy. We’ll still keep it around so we can work through all the patches and tickets that are there, but everything new will happen on the Lighthouse setup.

We hope you’ll enjoy this upgrade to the Rails collaboration infrastructure. We’re really looking forward to the onslaught of marvelous patches that the Git lords have promised us will flow from the mountain now.

Rails is moving from SVN to Git

We’ve been preparing for Rails to move the official source repository from Subversion to Git for some time now and it seems that it’ll happen over the next week or so. The premiere will happen alongside the official launch of Github.

The move will also mean that we’re going to be switching ticket tracking to Lighthouse. So now both our repository and ticket tracking will be powered by Rails applications, which is a nice bonus treat.

When the move happens, we’ll freeze the existing Subversion repository and the Trac installation. Both will live on for a long time to come, but will be entirely deprecated. This means that your existing svn:externals will not break, but if you want the latest edge, you’ll have to get it from the new git repository.

So now is a great time to learn more about Git in anticipation of this move. I recommend starting with the Git for SVN’ers crash course.

Adapter gems available

The extracted adapter gems are now available for install from the gems.rubyonrails.org server. Say you want the Oracle adapter installed, you just do gem install activerecord-oracle-adapter. All the extracted gems are:

  • activerecord-firebird-adapter
  • activerecord-frontbase-adapter
  • activerecord-openbase-adapter
  • activerecord-oracle-adapter
  • activerecord-sqlserver-adapter
  • activerecord-sybase-adapter

The mysql, postgresql, and sqlite adapters are still included in Rails core.

These will be released to the standard Ruby repository alongside Rails 2.0 when that reaches final.

How to get a patch into Rails

For a while, the patch queue was getting badly out of hand. The flood of new entries was simply too great to be reasonably managed by a small group of people. Too much got stuff got stale and their creators got disillusioned, understandably so. But for a while now, we’ve been running with a new policy on patches, which seems to be working a lot better for those who’ve been following it.

But I’m sensing that a fair number of people are not aware of those changes in policy, so I thought best to bring them up again here.

Step 1: Raise the barriers of quality

Part of the reason that the original patch queue got out of hand was due to the large number of patches coming in that lacked essential qualities of a good patch. They were either missing a good rationale (why am I doing this? what’s the benefits?), good test cases, or didn’t update the relevant documentation. To apply such a patch meant that this work had to be shouldered by someone else, usually the guy who wanted to apply the patch.

Now the barriers of quality are more apparent. Your patch will simply not be considered for inclusion before it has all those elements. It’ll live with the “unverified” keyword until you or someone else that cares especially deeply about the patch (like someone else having the same problem) gets the quality up to par. Then the patch moves on to step 2.

Step 2: Get the community engaged to review your patch

The last step before your patch is ready to be put in the queue for inclusion is to get community support round up. We now require that three different people must review your patch, apply it, run the tests, read your documentation, and like what it does and how it’s implemented. When they do, they’ll make a comment on the ticket with a “+1”.

Get three such +1s and you can tag your patch with the “verified” keyword. That’ll make the patch appear in Report #12: Verified Patches, which is a bell telling the core team that you patch is baked and ready to be included (barring a final review).

The core team is trying to keep report #12 empty at all times. There shouldn’t be a big lag time between getting to “verified” and getting a final review of your patch, which will either send it back to unverified (because the implementation is deemed in need of work or because there’s some fundemental disagreement over whether or how this patch goes about its business) or it’ll be applied and available in edge.

So if you have a patch that you still care about sitting in the queue, dust it off, and put it through these two simple steps and you’ll be back on the road to glory. There are still no guarantees that your patch will receive immediate attention, but so far we’ve managed to keep report #12 moving very nicely. It’s all empty as of today!

The front page of http://dev.rubyonrails.org has been updated to reflect this policy.

Plugin loading internals have changed, for the better!

Until now, the task of locating and loading plugins into your app was handled by a handful of private methods on the
Rails::Initializer. These methods were fairly large, coarse grained, and as a result hard to hook into without resorting
to fragile cut and paste if you wanted to customize how your plugins are loaded.

New in Edge Rails, changeset 6277 replaces this smattering of methods with two new internal classes, Rails::Plugin::Locater and Rails::Plugin::Loader. If you need to hook into how plugins are loaded, you can define a subclass of Rails::Plugin::Loader, then register your custom class to be the class that handles plugin loading using the new plugin_loader configuration option in your config/environement.rb:


  Rails::Initializer.run do |config|
    # Config settings...
    config.plugin_loader = PluginLoaderWithDependencies
  end

This should make extensions on top of the plugin system, such as the Plugems approach developed by the team over at Revolution Health, far easier to implement and maintain.

To those monkey patching the plugin loading subsystem in Rails, this introduces substantial changes to the way that plugins are located and loaded. In the short term this might mean that your customizations to the internals will very likely break, but the good news is that in the long term the new implementation will be far easier to customize.

For those adventurous early adopters living on the Rails Edge, please give your apps a test run to ensure these changes
don’t break anything for you. As always, bug reports and patches are welcome: http://dev.rubyonrails.org/.

Shiny new Subversion and Trac cluster

You’ve all noticed the excruciating Rails svn updates and Trac molasses in the last couple of weeks. Following the release of Rails 1.2 we thoroughly overwhelmed our development server, no small feat for a hefty dual Xeon. Congratulations, all, for your hearty Rails appetite! Your sustained Mbps say more than words possibly could.

Our friends at TextDrive have stepped up once again to keep Rails development running smoothly and your production apps deploying predictably. Please give a warm welcome to our new development cluster, a load-balanced crew of SunFires and Thumpers hosting Trac at dev.rubyonrails.org and Subversion at svn.rubyonrails.org.

Subversion will remain available at the old dev URL so you needn’t touch your live apps. Feel free to migrate to the new URL at your own speed.

New Feature for Rails 2.0: Multiple Controller View Paths

Thanks to John Long, Rails now supports multiple view paths for each controller. For 90% of the apps, this probably won’t make a huge difference. But, for those of you wanting to productize various open source apps, you can now do this:

/beast/trunk$ mkdir vendor/plugins/test_view_paths
/beast/trunk$ cd vendor/plugins/test_view_paths
/beast/trunk/vendor/plugins/test_view_paths$ mkdir views
/beast/trunk/vendor/plugins/test_view_paths$ echo “ActionController::Base.view_paths.unshift File.join(directory, ‘views’)” > init.rb

What you’ve done is created a minimal rails plugin that inserts a ‘views’ directory in the plugin to the beginning of the #view_paths array. Now, copy a view from the rails app into the plugin’s ‘views’ directory, make a noticeable change, and start your app. You’ll notice that it picked your plugin’s view over the application’s view.

Using custom mime types

So, you’ve started using Rails’ new mime type support with responds_to, but you were wondering how to add your own custom types? Luckily, Geoffrey Grosenbach (aka topfunky) is there to show you the way. His example involves registering an extension for .png, and generating a custom icon for an order in his shopping cart with some RMagick-fu. His example looks something like this:

Mime::Type.register "image/png", :png

# then in your controller action
def show
  respond_to do |format|
    format.html { }
    format.png { }
  end
end

Mime::Type.register will add the image/png mime type to the collection of mime types, bind it to the .png extension. It also creates a special Mime::Type instance at Mime::PNG. Check out the great post and the comments for more tips on caching and RMagick.

Note: After some investigation, I’ve found that using Geoff’s :format hack is not required on Rails 1.2 if you make the request with the :format parameter. Using routes like formatted_post_path(@post, :xml) will give you a path like “/posts/1.xml”. Accessing that will write public/posts/1.xml, regardless of what the page_cache_extension is.

Reloading Revamped

A few days ago I checked in a significant improvement to Rails’ dependencies and reloading code. In the past, changes to dependencies.rb have shed the blood of those courageous enough to ride edge; We’ve worked hard to prevent accidental breakage this time, but there may be some changes that could break your app.

Before you freeze edge to the prior revision, I should explain that most breakage will be extremely simple to fix. Prior to this revision, Rails would happily load files from Ruby’s standard lib via const_missing; you will now need to explicitly require such files. (Rails’ autoloading was intended as a replacement for require_dependency; its replacement of Ruby’s require is unfortunate and undesired.)

This change is not the only one that has occurred. Rails’ Reloadable module has been deprecated, and the previously independent systems of automatic loading and unloading have been brought together in a happy union.

Dependencies’ new behavior should be more reliable and less annoying. Annoyances such as the lack of module reloading have been fixed. Accidentally loading stdlib packages will no longer occur.

The actual mechanics of Dependencies are now relatively simple. Instead of using Reloadable to decide which classes to unload, Rails records which constants are loaded via const_missing. When the request is completed, each autoloaded constant is removed, leaving the process in a clean state. The actual mechanics are slightly more complex, but not inordinately so. Feel free to open dependencies.rb and peruse the code.

Hopefully the newfound simplicity of this approach will improve the transparency of Dependencies — some software is best when not noticed. If you’re running on trunk this change does cause your application to error, please do open and assign Ulysses a ticket.

Before I depart, I’d like to mention another (independent) change to dependencies: When Rails fails to find a missing constant, you will now see a fully qualified constant name in the description. For example, if a method in your User class references Acount, (rather than Account,) Rails will state that User::Acount is missing rather than ::Acount. Rest assured that Rails has looked for Acount in Object as well as User, and is merely reporting the fully qualified constant name, as Ruby’s own const_missing does.