Using SQLite3 with Rails

Posted by jamis January 29, 2007 @ 09:09 PM

If you are using SQLite3 with a Rails application, make sure you are using SQLite3 3.3.7 or earlier. Versions after 3.3.7 incompatibly changed the way in which default values are stored, making it so that current versions of Rails get into quoting issues and problems with columns with NULL defaults.

The source code for version 3.3.7 may be downloaded here: sqlite-3.3.7.tar.gz. Likewise, a pre-compiled binary for Windows is available here: sqlite-3_3_7.zip.

I’ve got a ticket open on the SQLite3 trac about this (right here ); hopefully this will be recognized as an unacceptable change for a point release and will be reverted.

Posted in General | 14 comments

Comments

  1. Daniel Morrison on 29 Jan 22:22:

    Thanks Jamis. I’d been fighting this bug and thought it was something wrong with my setup.

    Glad to know I’m not crazy.

  2. nick on 29 Jan 22:48:

    I really like SQLite. But those SQLite freaks changed the file-format between 3.2 and 3.3, too. So keep in mind: Alwasy reassure which version of SQLite you use and which one your hoster supports.

  3. YoNoSoyTu on 29 Jan 23:01:

    I think I made a patch a while ago for this issue: http://dev.rubyonrails.org/ticket/6369

    I was having problems with a 3.3.8 version on Mac, but with the patch the adapter seemed to work fine in a 3.3.6 version on Windows.

    I still use the patch (to be fair, is monkey-patched) in Rails 1.1 on Mac with no apparent problems.

  4. Eliot on 30 Jan 00:11:

    Windows binary link above is wrong, I think. I think it’s is probably what you wanted: http://www.sqlite.org/sqlitedll-3_3_7.zip

    It would be nice if the Rails adapter could be patched to selectively work around this bug based on SQLite version.

  5. Bosko on 30 Jan 00:20:

    Thanks. I got bit by this last night as a whole bunch of my unit tests were suddenly failing. Took a while to figure out what was going on.

    Since when does a nil value for a string attribute become “NULL” anyway? :-)

  6. bobes on 30 Jan 06:52:

    I never used SQLite until few days ago. When I saw all those “NULL” strings in my models, I got back to PostgreSQL. I thought it was some crazy SQLite feature and didn’t want spend time investigating :) But now I’ll give it one more go…

  7. mk on 30 Jan 17:46:

    sqlite is really great, but mayb a nicer solution is dont promote a not so good database. Use the best of the best, postgresql.

  8. toupeira on 30 Jan 18:37:

    I’ve experienced this problem too, see the Rails ticket http://dev.rubyonrails.org/ticket/6523 and the Debian bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=397531

    A quick workaround is to set config.active_record.schema_format to :sql in environment.rb. This will make the tests work again, though new tables created with migrations will still have the same problems.

  9. Jens Alfke on 31 Jan 05:36:

    I agree with the comment on the Trac, that since the sqlite version with this unfortunate change has been in the wild for a while, Rails really needs to work around it.

    It seems it should be pretty simple, especially in Ruby, to strip any single-quotes from the ends of the default-value string if the column type is ‘string’.

    To ‘mk’: Make up your mind: is sqlite ‘really great’ or ‘not so good’? I happen to agree with ‘really great’. You can’t really compare it with postgres or MySQL because it’s not a server, so it doesn’t have the same industrial-strength. But industrial-strength is massive overkill for most web apps, and the setup and admin messes involved with db servers drive me nuts.

  10. Luke Redpath on 31 Jan 12:29:

    Mmm, this is a problem for me as I’ve never managed to get sqlite 3.7 (the version in Darwin ports) working with Rails migrations. I always get errors like the following:

    SQLite3::SQLException: near “ADD”: syntax error: ALTER TABLE pages ADD “version” integer

    I’ve tried various versions of the sqlite3-ruby bindings including the latest, 1.2.0. I have SWIG installed courtesy of darwin ports and it does seem to compile something during the gem install:

    Building native extensions. This could take a while… ruby extconf.rb install sqlite3-ruby checking for sqlite3.h… yes checking for sqlite3_open() in -lsqlite3… yes creating Makefile

    make gcc -I. -I. -I/usr/local/lib/ruby/1.8/i686-darwin8.8.1 -I. -DHAVE_SQLITE3_H -I/usr/local/include -fno-common -g -O2 -pipe -fno-common -c sqlite3_api_wrap.c cc -dynamic -bundle -undefined suppress -flat_namespace -L”/usr/local/lib” -L”/usr/local/lib” -o sqlite3_api.bundle sqlite3_api_wrap.o -lsqlite3 -lpthread -ldl -lobjc make install /usr/bin/install -c -m 0755 sqlite3_api.bundle /usr/local/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.0/lib make clean Successfully installed sqlite3-ruby-1.2.0

    Yet no luck. I compiled and installed the latest version of sqlite3 (.12 I think) and I no longer have this problem but of course I have the NULL problem. So I can’t use 3.3.7 or above?

  11. Don on 31 Jan 17:46:

    Wow, I tried for HOURS to solve the exact same problem as you Luke. I could never find anybody who was experiencing the issue. I’m using SQLite as installed by DarwinPorts as well.

    The newest version of the sqlite-ruby gem doesn’t require SWIG. What I ended up doing was installing the updated sqlite-ruby gem, and then reinstalling SQLite from DarwinPorts, and everything magically started playing nice.

  12. Nerdmaster on 01 Feb 05:31:

    Good info, thanks… while I do think SQLite is a great piece of software, having these kinds of changes in minor releases is pretty poor design. Glad I only use SQLite for development, and not production….

  13. Jason Frankovitz on 14 Feb 16:19:

    The link above for SQLite 3.3.7 isn’t valid (http://www.sqlite.org/sqlite-3.3.7.tar.gz) Anywhere else I can download this from? My client has 3.3.8 and it’s causing this exact bug. Thanks!

  14. Nathan Wright on 16 Feb 02:24:

    This link at sourceforge still seems to be valid. I installed it on my box and it seems to work fine. (Ruby URL b/c the link is crazy long)

    http://rubyurl.com/JVu