Why is my Rails app calling Solr so often?

I work on the back-end of a Rails app that uses Solr via Sunspot. Looking at the solr logs, I could see the same item being added/indexed repeatedly sometimes right before it was deleted from solr. I didn't write the code, but I was tasked with figuring it out.

Glancing at the main path of the code didn't show anything obvious. I figured the superfluous solr calls were  happening via callbacks somewhere in the graph of objects related to my object in solr, but which one(s).  Again, I didn't write the code, I just had to make it perform.

I hit on the idea of monkey-patching (for good, not evil) the Sunspot module.  Fortunately, most/all of the methods on the Sunspot module just forward the call onto the session object.  So, it's really easy to replace the original call with anything you want and still call the real Sunspot code, if that's what you want to do.

This is so easy to do that I even did it the first time in the rails console.  In that case, I was happy to abort the index operation when it first happened.  So, I whipped this up in a text file and pasted it into the console:

module Sunspot
  class <<self
    def index(*objects)
      raise "not gonna do it!"
    end
  end
end


Then, I invoked the destroy operation that was triggering the solr adds, got the stack trace, and could clearly see which dependent object was causing the index operation.

For another case, I needed to run a complex workflow in a script to trigger the offending solr operations. In that case, I wanted something automatically installed when the script started up, and I wanted something that didn't abort - all I wanted was a stack trace. So, I installed the monkey-patch in config/initializers/sunspot.rb and had a more involved index function:

    def index(*objects)
      puts "Indexing the following objects:"
      objects.each { |o| puts "#{o.class} - #{o.id}" }
      puts "From: =============="
      raise rescue puts $!.backtrace
      puts ===========\n"
      session.index(*objects)
    end

That last line is the body of the real version of the index method - like I said, trivial to re-implement; no alias chaining required.

Maybe there's some cooler way to figure this out, but this worked for me.

enjoy,
Charles.

Rails/Rspec does not clean up model instances on MySQL

I recently solved a thorn in my side relating to some Rspec tests in our code base when running on my development machine using MySQL.  For some reason, some instances that were created using Factory Girl weren't getting cleaned up, which in turn would cause subsequent test runs to fail because of duplicate data.  So, I'd DELETE the whole tables from the MySQL prompt.  I looked in the test.log file, and I could see the save points being issued before the objects were created, but they weren't getting removed at the end of the test. 

I didn't have a lot of time to look into it, and I didn't know where to look - Rspec, Factory Girl, Rails?  So, in the short-term, I just added after_each calls to destroy the objects.  And, I moved on.

Then, I was dumping schemas in MySQL using SHOW CREATE TABLE in order to analyze some tables and indexes, and I noticed the storage ENGINE flag on the tables.  I went back and looked at the tables in my test database that were giving me trouble, and, of course(?), they were MyISAM rather than InnoDB.  So, transaction rollback (used to clean up after tests) didn't work.

I changed the storage engine on those tables (ALTER TABLE t1 ENGINE = InnoDB), commented out the manual clean-up code, and voila!  It works right now.  Pretty obvious in retrospect, but I didn't even know where to start looking in our stack.

I hope this helps some other poor souls, too.


Charles.

Freeing up phone space on Android

For the last couple of months my Motorola Droid running Android 2.2.2 has been complaining about being "low on space" for the phone, not the SD card.  I pruned some apps, but that didn't help much. Things really came to a head this morning when my phone was so low on memory that it was no longer downloading email.

I found this article to be quite helpful -
http://www.androidcentral.com/monthly-maintenance-keeping-things-speedy

For me, the two big ones were Messaging and the Browser cache.  I had a couple of threads in Messaging containing a number of pictures.  Once I saved the pictures off to the SD card, I purged the threads, that freed up ~20MB.  Clearing the browser cache freed another ~20MB, but that will probably evaporate again as the browser caches things.

Here's a minor whine about Android:  the SD card and phone storage settings page tells you how big your SD card is and how much space is remaining, but the phone storage just says how much is left.  Without knowing how much I had to start with, it's hard to know if, say, 20MB is a lot or not.  As near as I can tell, Android seems to complain when the space is less than 25MB.

Update: I ran out of space again, and clearing the browser cache didn't help.  After bumping around some more, first I discovered that in "Manage Applications" the one and only menu option is to sort by size.  Doing that revealed that the new pig was the (post pay-wall version) New York Times application.  It was using over 60MB of data space in the Phone Storage area.  The app doesn't have a "clear cache" function, so I used the "Clear Data" button from within Manage Applications, and I was back in action.

enjoy,
Charles.