SCIM with Azure AD - no parameters coming through to Rails contoller

I've been working to integrate our application with Azure Active Directory via SCIM - i.e., to allow Azure AD to provision users in our application using SCIM.  The problem I was having was that when Azure went to create users, the parameters hash in Rails was empty - {}.  I opened a ticket with Microsoft and spent weeks (literally) going back and forth with them.  After they assured me that the parameters were being sent, I started dumping all the info I could about the incoming requests. 

Eventually, I found that request was coming in with a "Content-type: application/scim+json" header, even though Microsoft's documentation showed "application/json".  (I've opened an issue for this.)  Once I saw that, I could easily reproduce the bug locally with curl.  This fix was pretty straightforward - add a new MIME type.  I found a thread on GitHub hashing out how/when to do that with Rails API, but it applied just the same in vanilla Rails.

I hope this helps someone else out there.  Enjoy.

Open Classes and Lazy Loading in Rails Don't Always Mix

Once upon a time, I came across an odd bug in some Rails code that was kicking my butt for some time.  The reason it perplexed me was it was a classic Heizenbug that seemed to come and go.  (That, and this was a side project that I couldn't devote much focused time to.)

The bug sometimes manifested itself with this error when running tests -
NoMethodError: undefined method `all' for SocialQueue:Class
where SocialQueue is an ActiveRecord model.  If I subsetted the tests being run to just the ones that failed, the bug would go away.  And, if I ran other tests before, the error would go away.   In other words, the act of trying to observe the bug would change it.

Another variant of the error I found when running the server in development mode was:
undefined method arel_table for QueuedPost:Class.  Again, QueuedPost is a model class, and I assume it has that method somewhere in the voodo that is ActiveRecord.

The error showed up when I added an "innocuous" tracing statement.  If I replaced the tracing statement with a puts statement, it worked.  If I put a return statement as the first statement in the tracing method, the error persisted - i.e., nothing in the body of that tracing method was causing harm.

The tracing method was in a module, in a separate file, and just requiring that module would cause the errors.  How could that be?  The module in question wasn't methods to be included in a class - it's just a name space to put this tracing method.  What's so bad about requiring a module?

What else is in that file?  Oh yeah, I have some code in there that opens my model classes to add a method to each class.  I put the new methods in that file, away from the rest of the model's definitions, because this tracing facility was experimental, and I didn't want to commit to modifying the model classes just yet.

In the words of Merlin Mann, "turns out" in development and test modes, Rails loads class definitions lazily.  When my module that opened model classes was loaded, the model classes hadn't necessarily been loaded.  If the model was loaded, it worked.  Otherwise, it wasn't opening an existing model class, but rather it was opening a new class.  Then, when my code that tried to use the models was run, there was already a definition for the class, so Rails didn't load the model class, and the object I thought was a model, was basically a lump of uselessness with the one method I intended to inject into a model class but none of the model methods.  I have since heard of lazy loading causing problems in STI.

The long-term solution for my problem is to move those new, injected methods into the main definitions of the models, now that my experiment is over, and I know I want to keep those methods.

In the interim, I came up with a simple hack that can be used anywhere you want to open a model class from some other file (or, maybe I've convinced you not bother - just edit the model).  Right before you open the model class, just mention it.  For example:
SomeModel
class SomeModel
  def my_new_method()
  end
end
Mentioning the model causes Rails to load it.  Then, when you open it, you're actually opening the real model class.

Perhaps there are better ways to skin this cat, but this works, and in the process, I learned about the existence and dangers lazy loading in Rails.

enjoy,
Charles.

Active Record - joins + include Methods Causing an Unintended Join

I recently fixed a problem in my Rails 3.2 app where I was using both the joins and include methods in an Active Record query, and it was triggering a join that I didn't want.  WTF?  Why are you using include and joins, and you don't want a join?

I needed to run a query on table A and I needed to apply criteria against another table B.  Thus, I needed to (inner) join those two with the joins method.    For the rows of A that met the search criteria, I wanted to eagerly the corresponding rows from tables X, Y, and Z.  Of course, I wanted to avoid a 3N+1 query situation.  So, I also used the includes method.

Typically, the includes method generates a query by IDs for the related objects.  In my case, I was getting four INNER JOINs - one each for B, X, Y, and Z.  Under "normal" circumstance, maybe this would have been OK, but my problem is table Y is in a separate database, and you can't join across databases.  (You can't really do transactions across databases, either.)

My original code used an array of named associations in the joins method - joins(:bs).  On a lark, I decided to recode it to use a string - joins('INNER JOIN bs ON bs.a_id = as.id'), and it worked:  I got the inner join for B and three individual queries for X, Y, and Z.  Because Y is queried as a simple query with an array of IDs, the fact that Y is in another database isn't a problem - it just works.

Anyway, if you've stumbled across this post while trying to solve the same problem, I hope this helps.

Charles.

Why is my Rails app calling Solr so often?

I work on the back-end of a Rails app that uses Solr via Sunspot. Looking at the solr logs, I could see the same item being added/indexed repeatedly sometimes right before it was deleted from solr. I didn't write the code, but I was tasked with figuring it out.

Glancing at the main path of the code didn't show anything obvious. I figured the superfluous solr calls were  happening via callbacks somewhere in the graph of objects related to my object in solr, but which one(s).  Again, I didn't write the code, I just had to make it perform.

I hit on the idea of monkey-patching (for good, not evil) the Sunspot module.  Fortunately, most/all of the methods on the Sunspot module just forward the call onto the session object.  So, it's really easy to replace the original call with anything you want and still call the real Sunspot code, if that's what you want to do.

This is so easy to do that I even did it the first time in the rails console.  In that case, I was happy to abort the index operation when it first happened.  So, I whipped this up in a text file and pasted it into the console:

module Sunspot
  class <<self
    def index(*objects)
      raise "not gonna do it!"
    end
  end
end


Then, I invoked the destroy operation that was triggering the solr adds, got the stack trace, and could clearly see which dependent object was causing the index operation.

For another case, I needed to run a complex workflow in a script to trigger the offending solr operations. In that case, I wanted something automatically installed when the script started up, and I wanted something that didn't abort - all I wanted was a stack trace. So, I installed the monkey-patch in config/initializers/sunspot.rb and had a more involved index function:

    def index(*objects)
      puts "Indexing the following objects:"
      objects.each { |o| puts "#{o.class} - #{o.id}" }
      puts "From: =============="
      raise rescue puts $!.backtrace
      puts ===========\n"
      session.index(*objects)
    end

That last line is the body of the real version of the index method - like I said, trivial to re-implement; no alias chaining required.

Maybe there's some cooler way to figure this out, but this worked for me.

enjoy,
Charles.

Rails/Rspec does not clean up model instances on MySQL

I recently solved a thorn in my side relating to some Rspec tests in our code base when running on my development machine using MySQL.  For some reason, some instances that were created using Factory Girl weren't getting cleaned up, which in turn would cause subsequent test runs to fail because of duplicate data.  So, I'd DELETE the whole tables from the MySQL prompt.  I looked in the test.log file, and I could see the save points being issued before the objects were created, but they weren't getting removed at the end of the test. 

I didn't have a lot of time to look into it, and I didn't know where to look - Rspec, Factory Girl, Rails?  So, in the short-term, I just added after_each calls to destroy the objects.  And, I moved on.

Then, I was dumping schemas in MySQL using SHOW CREATE TABLE in order to analyze some tables and indexes, and I noticed the storage ENGINE flag on the tables.  I went back and looked at the tables in my test database that were giving me trouble, and, of course(?), they were MyISAM rather than InnoDB.  So, transaction rollback (used to clean up after tests) didn't work.

I changed the storage engine on those tables (ALTER TABLE t1 ENGINE = InnoDB), commented out the manual clean-up code, and voila!  It works right now.  Pretty obvious in retrospect, but I didn't even know where to start looking in our stack.

I hope this helps some other poor souls, too.


Charles.