Reloading Revamped
Posted by nicholas August 11, 2006 @ 05:25 PM
A few days ago I checked in a significant improvement to Rails’ dependencies and reloading code. In the past, changes to dependencies.rb have shed the blood of those courageous enough to ride edge; We’ve worked hard to prevent accidental breakage this time, but there may be some changes that could break your app.
Before you freeze edge to the prior revision, I should explain that most breakage will be extremely simple to fix. Prior to this revision, Rails would happily load files from Ruby’s standard lib via const_missing; you will now need to explicitly require such files. (Rails’ autoloading was intended as a replacement for require_dependency; its replacement of Ruby’s require is unfortunate and undesired.)
This change is not the only one that has occurred. Rails’ Reloadable module has been deprecated, and the previously independent systems of automatic loading and unloading have been brought together in a happy union.
Dependencies’ new behavior should be more reliable and less annoying. Annoyances such as the lack of module reloading have been fixed. Accidentally loading stdlib packages will no longer occur.
The actual mechanics of Dependencies are now relatively simple. Instead of using Reloadable to decide which classes to unload, Rails records which constants are loaded via const_missing. When the request is completed, each autoloaded constant is removed, leaving the process in a clean state. The actual mechanics are slightly more complex, but not inordinately so. Feel free to open dependencies.rb and peruse the code.
Hopefully the newfound simplicity of this approach will improve the transparency of Dependencies—some software is best when not noticed. If you’re running on trunk this change does cause your application to error, please do open and assign Ulysses a ticket.
Before I depart, I’d like to mention another (independent) change to dependencies: When Rails fails to find a missing constant, you will now see a fully qualified constant name in the description. For example, if a method in your User class references Acount, (rather than Account,) Rails will state that User::Acount is missing rather than ::Acount. Rest assured that Rails has looked for Acount in Object as well as User, and is merely reporting the fully qualified constant name, as Ruby’s own const_missing does.
Update: One problem that has been rumoured to exist at the moment is that a file might be loaded twice. Behavior resulting from this can be repeated validations, (Model file loaded twice) or stack errors. (File which uses an alias method chain loaded twice.)
If you see this problem, make sure you’re not using require and require_dependency to load the same file. At the moment they’re totally independent and do not talk to each other. (But it’s on the list.)
Is there any logging/debugging we can enable to see this process at work? (e.g. “library not found … loading xyz”, blah, blah)
We’ll certainly review the dependencies.rb code, but having a trace or log of when this “magic” is happening would help us get our application life-cycle code correct, as opposed to just lucky.
Thanks.
This seems to have broken the script/about command.
Future versions of Dr Nic’s Magic Models will provide a way to “reload” an in-memory model class if a model file is subsequently created.
Finally! I’ve been complaining about this issue for ages. I’m glad I can now stop. Much appreciated.
Any chance you guys might finally deal with the Chainsaw Infanticide Logger Manuever while you’re at it?
Ah fantastic! Reloading of modules… I had my own little nasty hacks for these, thanks!
Brittain: I’ve just checked in some logging code for this. To enable it, set Dependencies.log_activity = true after initialization.
Nic: You might want to take a look at Dependencies.autoloaded_constants.
Bob, +1
Somebody should fix the problem with the Logger class, why not subclass it? why do you have to open it and change its behavior? It seems that Ruby is not suitable for large projects, since it encourage these kind of practices, it could easily become a nightmare to debug problems when anybody open classes and override the default behavior. I found the very same problem that Zed found a year ago and the problem is still there.
http://comments.gmane.org/gmane.comp.lang.ruby.general/110876
For what its worth, I’m seeing this behavior:
My models living in a subdir (app/models/subdir/modelxyz.rb) were loading fine in past revisions. Now I get some odd behavior. I am using the model during a request (page1) and everything is great, I’ll make another request to a different page (page2) in the same server session and then it breaks with:
./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:177:in `load_missing_constant’: Expected script/../config/../app/models/market/market.rb to define Market
The model class is being used in calculation in a layout shared by the 2 pages, so its being used in the exact same way during the 2 different requests. It even found the correct class file above according to the error message.
A reload of page2 returns fine however, and subsequent reloads are fine, until I load page1 again. Then its like it unloads the class and going to page2 it’s broken again.
I even tried to explicitly declare the subdir as a load path in my environment.rb, same behavior.
Anyone else seeing this kind of thing?
Nate, yeah I’m seeing this with my unobtrusive javascript plugin. It can find the file but it can’t see the defined module:
Expected /config/../vendor/plugins/unobtrusive_javascript/lib/unobtrusive_javascript.rb to define UnobtrusiveJavascript
DHH, what do you think about this related subject?
http://www.adam-bien.com/roller/page/abien?entry=ruby_on_rails_and_transactions
Could you leave a decent response on the page? My boss has seen this and is not happy, can you provide some reassurance?
The fix of the following error: dependencies.rb:291:in `const_missing’: uninitialized constant Foo::Bar after Reloadable deprecation can be fixed as below:
If you have added paths in config.load_paths in config/environment.rb, replace it with config.autoload_paths
Ex.
config.load_paths += %W( #{RAILS_ROOT}/app/domain #{RAILS_ROOT}/app/domain/sql_builder )
turns into
config.autoload_paths += %W( #{RAILS_ROOT}/app/domain #{RAILS_ROOT}/app/domain/sql_builder )
then remove all include Reloadable lines.
Hello, I tried to open a ticket in trac, but it is still down (or struggling).
To keep this short (I would post more info to trac), if you have a task with a require statement, and it has spaces within the quotes, you will see an error like this:
6bar8:~/dev/depot Adam$ rake migrate (in /Users/Adam/dev/depot) rake aborted! undefined method `autoload_paths=’ for Dependencies:Module
(See full trace by running task with—trace)
that is caused by this in one of my task files:
require ’ fileutils ‘
if I change that to ‘fileutils’, it works like a charm.
I’m not sure where that needs to be cleaned up, but somewhere it needs a file_name.trim.
Please email me if you want more specific info.
thanks, Adam
Adam: You’re not allowed to have spaces within the argument to require (or require_dependency):
~> ruby require ’ pp ‘
:in `require’: no such file to load -pp (LoadError) from -:1Hey Nicholas, Interesting. Just know that I chatted with a handful of people about this on #rubyonrails and the email list today. A few plugins are going to break either because of the space or they are missing the ‘require’ all together. Would it be worth documenting somewhere? Maybe this is good enough.
Thanks nicholas for cleaning the dependencies process. It sounds like a much needed improvement.
Adam: I just checked with rails 1.1.2 and require ’ foo ’ didn’t work there either. Is it possible some other library is adding some strange interaction?
Plugins shouldn’t need to require their files either; their lib/ directories should be added to the autoload path.
Hi Nicholas, I tried it with 1.1.4 -> .6, but edge breaks it. It was hard to track down. For me it was the ’ ’ in this task, but for other people it was the rails_rcov plugin, not spaces in the require statement.
Nicholas, ping me off-line if you think it is worth tracking this issue down. If so I would be more than happy to dig into it a bit more. I know how busy you core guys are!!
I’m getting problems declaring STI classes in one model since the upgrade to edge. “superclass mismatch” all over the place. Works fine in console though. Works fine if I put them in their separate model also. That’s what I’ve done for the moment..
I second that – I’ve had these same problems in my app. I’ve even found ActiveRecord association type mismatches with the message:
ClassName expected but was ClassName.
I turns out that if you breakpoint here, you’ll find that the object_ids of the two classes are different, even though they both inherit from active record and have the same attributes. The other thing that is strange is that one of them will have the methods and associations declared in my model files, while the other one raises NoMethod errors when you try to call anything other than the attributes on the DB table.
Hope it gets fixed soon. Thanks for cleaning up the code, most of this stuff just sounds like rough edges.