Freeze is cool, so freeze for goodness sake
Posted by David March 31, 2006 @ 04:50 AM
Many shared hosts did a quick dance up and down at the release of Rails 1.1 because a large number of their customers were floating off the gems, which meant that upgrading the gems automatically updated all the applications. Some applications couldn’t handle the upgrade (most notably Typo), so they broke. And the customers were none too happy and complained to the hosts: “Why oh why, dear host, would you upgrade and break my application?”
This is obviously bad. But nothing is so bad that you can’t learn from it. And this is a wonderful opportunity to learn that You Should Freeze Rails For Any Application In Production. Sure, we talked about that back in December, but talk is cheap (and often overheard).
So here follows a lifted finger and a promise. The lifted finger first:
If thou bless thee world with an application of open source, thou must ship it with the version of Rails that thou knowest it to work with in vendor/rails.
Here’s one counter argument that will not allow you to evade this finger: “But it’ll make my app X% larger to download”. In this day of age, nobody cares. Time is more valuable than disk space and saving hair-pulling aggravation over broken dependencies is infinitely more valuable than disk space.
Then the promise: The next version of Rails will by default extract the version of Rails it was created with into vendor/rails. This will get everyone into the Christmas spirit of being good on day one. It’ll be natural to desire less dependencies and you will soon froth at the notion of a shared host controlling the destiny of your application by choosing to update some gems. And you will be happy and content.
I’d rather be able to use the feature of gems that allow you to specify which version of a gem to use… thus a webhost should be able to have both 1.0 and 1.1 installed, and the app would just specify which version to use.
Why is this feature of gems broken in rails?
Check out one of these sites to fix Typo if yours isn’t working:
http://www.d-jones.com/articles/2006/03/31/typo-rails-1-1-upload-progress-error or http://nubyonrails.topfunky.com/articles/2006/03/29/surviving-rails-1-1-with-server-monitoring
I’m very depending on the freeze command for my app, cause I’ll use it to bundle Rails in a Cocoa app. It worked great so far. For those trying out the freeze command in 1.1 and having problems there is a already a patch available for a tiny glitch (http://dev.rubyonrails.org/ticket/4458)
Shame on you for not providing proper backwards compatibility!
Forcing developers to change their code when Rails changes is a sign of bad engineering, especially when the update documentation says nothing more than ‘gem install rails—include-dependencies’.
I would use freeze if it worked on Windows. Last time I tried, it wrecked the app beyond repair.
I agree with DGM. Why not use the gems versioning?
That said, I don’t understand why developers, or hosting companies, would change a key piece of their infrastructure with out testing and without communicating the changes in advance to their customers.
If you’re really concerned about disk space and the time it takes to transfer (and assuming you’re using svn and Capistrano), you could transfer or unpack Rails once, and the just make a new link to it each time you deploy. You’d do something like this:
svn:ignoreon therailsdirectory in yourvendordirectory.shareddirectory that Capistrano creates, unpack orsvn exportthe version of Rails you’d like.vendordirectory of new releases. (Use anafter_symlinktask?)Of course, you still have to keep the version of Rails you’re using for development the same as the one on your server. Freezing is the surer way.
er, “then just make a new link”
I agree with DGM. Why not use the gems versioning?
That said, I don’t understand why developers, or hosting companies, would change a key piece of their infrastructure with out testing and without communicating the changes in advance to their customers.
space is hardly an issue. It is about 6Mb of transfer that you incur once, and it occupies less than 20Mb (with the svn data). Unless you pay by the Kb, I wouldn’t worry about it. The bigger issue here was hosting companies, wanting to please everyone by being responsive, took a few too many shortcuts. More testing, and most of all, a fair warning would have avoided near chaos for some of us ;)
Let that be a lesson for everyone and let’s look forward to a smooth next upgrade!
I know backward compatibility is a tricky issue but say for example that you were running 100 apps and there was a glaring security hole in 1.0 fixed 1.1, I t would mean having to manually update all 100 apps to use the fix.
If there is anything that breaks between version it should be clearly stated when the new version is released.
I can understand maybe a few breakages in a major release i.e. 2.0 however for 1.x updates I thing backward compatibility should be taken into account.
Just my 2 squids worth.
Why don’t we use the version-freezing feature that gems already support? I for one like being able to keep my gems up to date centrally and to do mass upgrades. Having to do this for each and every Rails application will be a nightmare.
Is there a spot with good, thorough documentation on freezing? I’ve been reading for some time and am confused with freezing rails, freezing gems, what should go in vendor, freezing external, etc.
Also, on Windows I did a freeze and saw an “rm -rf vendor” in there that scared me. Not to mention the freeze didn’t work on Windows. What’s this really doing? (the freezing, not the rm -rf).
I know that disk space is cheap, but that doesn’t make it right to have a seperate copy of rails in every application. It still adds up, and it adds complexity.
It violates the DRY principal. Why do you make and use a framework that does everything possible to not repeat things, and yet you effectively reccomend a cut and paste to keep the right version of rails with your app. That’s all freeze edge does… a copy and paste of rails into your own app’s structure. Wasteful!
Why not include a complete copy of ruby while you’re at it!
gems have versioning capability. Please use it! I would even suggest issuing a 1.0.1 and 1.1.1 to fix this, so that hosting environments could host both versions.
You’re always free to use gem versioning to bind your app. You can see how at http://wiki.rubyonrails.com/rails/pages/HowtoLockToSpecificRailsVersions. But the new default is going to be that it binds the Rails, so you don’t depend on someone else to be managing gems properly.
Oh, and if you have 100 apps, you’re not the target of defaults anyway. You’ll be clever and experienced enough to figure something out that works better for your special situation.
Eric, don’t be a tool. There’s no vendor here. I’d recommend exchanging the fun of indignation for the fun of participation. It’s a much longer lasting joy.
I think that copying the entire Rails codebase into each app’s vendor/rails directory by default is an unfortunate knee-jerk response and a misguided attempt to solve this problem. The hosting companies screwed up—they should have known better. We need not try to save people from themselves at the expense of those that do know better.
People with a hundred Rails apps are not the only ones who will not need the new proposed default behavior. Even if I only have one application, do I really want the entire Rails core checked into Subversion? No, I don’t. And if I have ten apps? Now I really don’t.
At the very least, if the decision is made to copy the entire Rails codebase into each app’s vendor/rails directory by default, at least give people the option of not doing so at install time. Example:
$ rails MyNewApp --no-vendor-railsWith all due respect, David, I think this promise proclamation was probably premature. (It’s all about the alliteration. :)
“Oh, and if you have 100 apps, you’re not the target of defaults anyway. You’ll be clever and experienced enough to figure something out that works better for your special situation.”
Well, Textdrive and Dreamhost have hundreds of apps running on their servers, although each of these are run by a different user. It would be fucking crazy if all these users had to run a freeze to ensure theit apps run correctly.
Please provide a simple way to pick a specific Rails version inside environment.rb or as an environment variable.
Unfortunately, freezing gems broke in 1.1.
—Steve
Rather than point fingers, you might take this as a reminder that it is not always a good idea to include experimental features in a widely distributed application.
However, I think upgrading is the most painful part of the Rails world at this point. I can see a use for a Capistrano style tool that could upgrade plugins, upgrade rails, upgrade javascripts, do a simple controller test (if fail, rollback upgrade), and other simple scriptable actions on multiple servers/multiple sites.
Upgrading is never easy, because backwards compatibility isn’t easy. Rather than whining “You broke my application,” folks should have done the following steps instead:
1. Not installed 1.1 on a production server.
2. Install Rails 1.1 release code into app’s vendor directory on a test environment:$ cd MyRailsApp/vendor/rails$ svn co http://dev.rubyonrails.org/svn/rails/branches/stable/3. Test application.
These simple steps would most likely have avoided unpleasant production snafus.
DHH, Eric’s not the tool here, you are.
I followed your link, but locking the gems doesn’t work, I still get errors. If it did work, the typo folks would have been all over it as a solution, and all the webhosting folks wouldn’t have to revert the upgrade, because every application could be fixed easily.
Part of testing a release is testing the upgrade path… and proper gem locking is the obviously correct path.
At the same time, there’s plenty of blame to pass around… the webhosts shouldn’t have upgraded without testing… this bug should have been found before 1.1 was released…
I’d really prefer if rails actually pushed the gem, versioning since the beginning, say if it genrated boot.rb with exact require_gem statements. That would be the reasonable default, imo.
I think a default environment.rb with the currently installed versions required (a la the Wiki) makes the most sense.
I think a default environment.rb with the currently installed versions required (a la the Wiki) makes the most sense.
freeze by default is hasty. make it require the correct gem version by default.
Aaron, freeze by default is safest (but a bit more costly, true. You are protected from older gems being removed too. This worked for me: http://blog.nanorails.com/articles/2006/03/28/freeze-all-your-ruby-gems-on-a-shared-host
DGM, freezing the gems works, but you need to freeze all the gems, not just rails.
FWIW, typo’s trunk now works on both on rails 1.0 and 1.1. They merged the 1.1 branch back into the trunk.
This is why you dont use shared hosts.
gotta love free electrecity and a cheap 100mbit connection at home.
“Protecting from webhost removing gems” sounds dumb… Might as well protect yourself from the webhost removing ruby and install the whole tree yourself. What about mysql? they might screw that up so better host it yourself too. Now we’re talking about a dedicated server.
What? It’s spelled out in the service agreement that they will keep those around? Well, maybe the agreement should specify which gems will be always be available. Problem solved.
If you are hosting in an environment where you don’t have an agreement regarding the components that you require, you are asking to get burned. Installed gems is part of the system, and should be specified in the agreement.
The beauty of gems, is that you can have multiple revisions… and the webhost could advertise both/all versions.
If I owned a webhosting service, I’d be telling everyone how to set their require_gems properly… and I’d make it clear that both versions will always be available for use.
Oh, and that is a lot easier now that the wiki is actually right. When everyone pointed me (and others trying to do this too) at the wiki, it didn’t work. The typo folks found a patch that worked better, and I changed the wiki to match. Just a few lines in your environment.rb does the job, which rails could put there automatically instead of simply copying the entire rails source into the vendor.
Keep the code base DRY. It doesn’t make sense to make so many copies of the rails code base. As rails gets more and more popular, it’s going to come back to haunt us.
Pascal: I simply disagree. What about DRY?
There is already a system in place that should be taken advantage of.
personally, I prefer svn:externals to a particular revision, as this keeps all of that crap out of my repository and ensures i have it where i need it.
I agree with convention over configuration, but there is already a convention—gem depencies. Let’s use that as the default.
Pascal: I simply disagree. What about DRY?
There is already a system in place that should be taken advantage of.
personally, I prefer svn:externals to a particular revision, as this keeps all of that crap out of my repository and ensures i have it where i need it.
I agree with convention over configuration, but there is already a convention—gem depencies. Let’s use that as the default.
Well said DGM! Lets pray that rails wont be copied to vendor unless told to.
my suggestion is to have some param to the rails command
rails myapp—freeze-rails
Even more to the point…. as documented in http://docs.rubygems.org/read/chapter/16 it should lock to the current version using a “pessimistic operator” or ∼>
require_gem ‘rails’, ’ ∼> 1.0’
or now with 1.1:
require_gem ‘rails’, ’ ∼> 1.1’
This way, we can lock an app into a particular release, but if an update needs to be made (bugfix, securty fix), it can be released as 1.0.1 and the new gem will be included. (But 1.1.x won’t.)
This policy would make hosting very robust, allow for quick security fixes clear across the board, and yet keep a new incompatible version from disrupting your production application.
No system that requiries strict FHS compliance, Debian for example, would be able to implement a system that required gems for revision control.
It may be nice to have using gems revision control as an avilable option, for example through an included rake task, but it’s a very poor option to require it in anyway.
mgreenly,
I’m not familiar with FHS, but are you saying that debian can’t install a rails-1.0.0 gem and a rails-1.1.0 gem?
Seems like several systems have multiple libc versions, does debian not support that?
One more reason that Debian is broken. (IMHO)
If I’m reading this right, we’re on the right track…
The gems/fhs incompatability is actually an issue for almost every Linux/Unix distribution. Some distributions, Debian for example, stick to the guidelines tighter than others, but most claim to.
No distribution that actually adheres to the standard can use gems. In practice most people run gems anyway, it’s easier.
When/if gems become a standard part of Ruby it will be complicated but until then it’s a bad idea to force most distrubutions to break there packaging guidelines.
Has anyone figured out how to freeze_gems properly on a win platform, especially after you’ve upgraded rails to 1.1, it seems that freeze copies the current rails codebase into the vendor folder by default, is there a way to pass arguments with rake for it to freeze to a specific version of rails?
Please don’t force untold numbers of duplicate copies of Rails to be checked into SCM repositories the world over. This would be a Very Bad Idea.
Take a lesson from Maven, using versioned external dependencies from a central repository is a proven and reliable practice. It can also be extended to encompass your own custom dependencies from a local repository. Dependency Hell fiascos such as the one which prompted this post from DHH are a direct result of people not understanding these concepts. Automatic cloning of the an framework source tree into every app would be a very large step in the wrong direction. All Duplication Is Evil!
The Gems approach provides everything you need for controlled versioning and dependency management (except perhaps for resolution of dependency chains through enforced project dependency metadata, which is the next big step that Maven2 is tackling, but that’s an advanced topic that’s lightyears ahead of the current ruby/gems/rails status quo). Centralized hosting of dependencies is Good Thing. If you are really paranoid about the repository host screwing something up, then you can host your own local repository where you have complete control over everything – even if you are on a shared host.
—Chad Woolley
Please don’t force untold numbers of duplicate copies of Rails to be checked into SCM repositories the world over. This would be a Very Bad Idea.
Take a lesson from Maven, using versioned external dependencies from a central repository is a proven and reliable practice. It can also be extended to encompass your own custom dependencies from a local repository. Dependency Hell fiascos such as the one which prompted this post from DHH are a direct result of people not understanding these concepts. Automatic cloning of the an framework source tree into every app would be a very large step in the wrong direction. All Duplication Is Evil!
The Gems approach provides everything you need for controlled versioning and dependency management (except perhaps for resolution of dependency chains through enforced project dependency metadata, which is the next big step that Maven2 is tackling, but that’s an advanced topic that’s lightyears ahead of the current ruby/gems/rails status quo). Centralized hosting of dependencies is Good Thing. If you are really paranoid about the repository host screwing something up, then you can host your own local repository where you have complete control over everything – even if you are on a shared host.
—Chad Woolley
Doh! Sorry for the doublepost, hit submit twice. I kind of would have expected the message text to be cleared after the first submit. Maybe it’s a bug in the blog due to dependency management problems…