Intermittently we were seeing the failures in the application with the following messages in the passenger log:
NoMethodError: undefined method `each' for nil:NilClass
and
(NoMethodError) "undefined method `each' for nil:NilClass"
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/mysql2-0.2.4/lib/active_record/connection_adapters/mysql2_adapter.rb:628:in `select'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/connection_adapters/abstract/query_cache.rb:54:in `block in select_all'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `cache_sql'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/connection_adapters/abstract/query_cache.rb:54:in `select_all'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/base.rb:467:in `find_by_sql'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation.rb:64:in `to_a'
/usr/local/rvm/gems/ruby-1.9.2-p0/ruby/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/finder_methods.rb:143:in `all'We found similar people experiencing the same problem here though the issues page on github didnt state a resolution.
We discovered we were running the MySQL 5.5 client libraries - namely the libmysqlclient 18. mysql2.so was built against libmysqlclient 18:
[root@host]# ldd ./vendor/bundle/ruby/1.8/gems/mysql2-0.2.13/ext/mysql2/mysql2.so
linux-vdso.so.1 => (0x00007fff51788000)
libruby.so.1.8 => /usr/lib64/libruby.so.1.8 (0x00007f47f68e4000)
libmysqlclient.so.18 => /usr/lib64/libmysqlclient.so.18 (0x00007f47f6389000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f47f6165000)
libm.so.6 => /lib64/libm.so.6 (0x00007f47f5ee1000)
librt.so.1 => /lib64/librt.so.1 (0x00007f47f5cd9000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f47f5ad4000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f47f589d000)
libc.so.6 => /lib64/libc.so.6 (0x00007f47f551e000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f47f5307000)
/lib64/ld-linux-x86-64.so.2 (0x0000003eccc00000)
libfreebl3.so => /usr/lib64/libfreebl3.so (0x00007f47f50aa000)
We are running Centos6 and use the Percona MySQL client libraries, so we downgraded from Percona-Server-shared-55 to Percona-Server-shared-51 to provide the earlier client library.
We gem bundled against an earlier version of MySQL client libraries (libmysqlclient 16) and we no longer experienced that intermit failure:
[root@host]# ldd vendor/bundle/ruby/1.8/gems/mysql2-0.2.13/lib/mysql2/mysql2.so
linux-vdso.so.1 => (0x00007fff9e7d5000)
libruby.so.1.8 => /usr/lib64/libruby.so.1.8 (0x00007ff304784000)
libmysqlclient_r.so.16 => /usr/lib64/mysql/libmysqlclient_r.so.16 (0x00007ff30436e000)
libz.so.1 => /lib64/libz.so.1 (0x00007ff304159000)
librt.so.1 => /lib64/librt.so.1 (0x00007ff303f51000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff303d33000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007ff303afc000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007ff3038e3000)
libm.so.6 => /lib64/libm.so.6 (0x00007ff30365e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007ff30345a000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff3030db000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff302ec4000)
/lib64/ld-linux-x86-64.so.2 (0x0000003a52200000)
libfreebl3.so => /lib64/libfreebl3.so (0x00007ff302c67000)
I'm yet to discover the specific issue with what I think is asynchronous caching race conditions and libmysqlclient18 with the mysql2 gem, but wanted to share my experience with the wider community to avoid others from going down the same rabbit hole I went down today.
well - I'm not sure if what I see is relevant or even coherent with your observations, but I'm developing on Rails 3.1.0 with the mysql2-0.3.7 gem on ruby-1.9.2-p290 on Mac OSX Lion - and I experience very strange behaviour when seeding a database!
ReplyDeleteI do User.delete_all in my db/seeds.rb, and then I can eval #{User.all.size} to zero, but when I do User.create( { name: 'admin' ... }), Ruby barfs with a Mysql2::Error: Duplicate entry? Almost like it did not remove the users from the users table entirely or at least did not flush the indexes!?
Adding to the mystery; I'm relying quite extensively on a
class AbstractActionBase < ActiveRecord::Base
self.abstract_class=true
end
and my seeds.rb will not allow me past the first 2 model creations without complaining about the abstract_action_base missing - but rather, I reckon is on an entirely different 'thread' :(
Any update on this? We are having a similar problem with percona.
ReplyDelete