SCIM with Azure AD - no parameters coming through to Rails contoller

I've been working to integrate our application with Azure Active Directory via SCIM - i.e., to allow Azure AD to provision users in our application using SCIM.  The problem I was having was that when Azure went to create users, the parameters hash in Rails was empty - {}.  I opened a ticket with Microsoft and spent weeks (literally) going back and forth with them.  After they assured me that the parameters were being sent, I started dumping all the info I could about the incoming requests. 

Eventually, I found that request was coming in with a "Content-type: application/scim+json" header, even though Microsoft's documentation showed "application/json".  (I've opened an issue for this.)  Once I saw that, I could easily reproduce the bug locally with curl.  This fix was pretty straightforward - add a new MIME type.  I found a thread on GitHub hashing out how/when to do that with Rails API, but it applied just the same in vanilla Rails.

I hope this helps someone else out there.  Enjoy.

Fixed: Problem with Safari To Go App for ACM Content

TL;DR - launch the Safari To Go app from the ACM Safari page.

One benefit of the ACM is access to a nice subset of O'Reilly's Safari library.  One can read the content on the website.  O'Reilly has an Android app for accessing the content called Safari To Go. (Note, they also have a newer app that doesn't work with the ACM library.)  However, there is a problem when trying to authenticate with ACM and Safari from the app.  The login page flashes a message about being signed in, and then it pops up a message saying the session has expired ("your session has been logged out").  I think this started when they added the ability to access books offline.  There are several one-star ratings in the app store bitching about this problem, and no responses from O'Reilly, both of which are frustrating.

Here's the simple solution I found:

1. Go to the ACM's learning site (https://learning.acm.org/) on your Android device
2. Login with your ACM web account credentials in the top right corner.
3. Click on "Safari" in the "Go to my..." menu in the top right corner.
4. Click on "Launch the mobile app..." at the top left.

This worked for me in February 2017. All interfaces subject to change.  Your mileage may vary.

enjoy,
Charles.

Updating Ruby versions in a Boxen - rbenv - ruby-build Environment

TL;DR: cd /opt/boxen/rbenv/plugins/ruby-build && git pull origin master

A while ago, I set up my development environment on my two machines using GitHub's Boxen.  To be honest, I got it going and moved on.  I have not been updating it.  Then, I recently started a new Ruby project, and I wanted the latest version of Ruby.  I went into rbenv, and it had no recent versions of Ruby - none since I set up Boxen.

How do I update the list of Ruby versions available in my Boxen environment?

I could update Boxen, but that's more than I wanted to bite off.  My first guess was to run brew update and brew upgrade rbenv ruby-build, but that failed because rbenv is part of the bootstrap of Boxen and not installed by Boxen's version of Homebrew.

So, I thought I'd need to update rbenv directly.  I figured it was installed as a git repo.  The question was: where?  which rbenv says it's a shell function.  Looking at that code, it un-aliases itself to run.

which -a  shows that rbenv is in /opt/boxen/rbenv/bin/rbenv.   From there I found the ruby-build plugin, which leads to the solution:

cd /opt/boxen/rbenv/plugins/ruby-build
git pull origin master

It's not like it's rocket-surgery, but it was a curious puzzle to sort out.  Soon I should update my Boxen environment, but that's a story for another day.

enjoy,
Charles.

Getting Started

How to begin the HashiQuest?

The world of AWS is huge these days.  The HashiCorp tools can be counted two hands, but since they interface with AWS, that limited count is deceiving.

I actually started by getting the lay of the land from AWS in Action, which Manning conveniently had on special just about the time I was interested in learning more about it.  The book isn't an exhaustive coverage of all of the AWS services, but it's an excellent overview.  I did their tutorial for building a WordPress site.  The authors provide their code examples online in GitHub, which is excellent.

After that, Terraform seemed like the logical place to start since it deals with building infrastructure in AWS.  Again, I followed the online tutorial, and was pleased by the lack of drama.

I don't for a minute think that doing either of these tutorials qualifies as any real expertise, but I found that just typing the command and checking the results in the AWS console starts to build both the physical and mental "muscle memory."  Both tutorials were done with the free tier in AWS, and free (as in beer) is always good.

Next, I checked out a new tool from HashiCorp - Otto.  Otto is a successor to Vagrant, but it's heading in very different directions.  If you start thinking that Otto is just Vagrant++, it's hard to understand the infrastructure and deployment functionality that Otto provides.  Otto provides a path from a development environment on a single machine, to a simple AWS deployment, to a more sophisticated AWS deployment.

Because Otto is based on some opinionated policies and best practices, it provides a great way to see how all the pieces of the HashiCorp ecosystem and AWS fit together.  It generates plain-text configurations and scripts in the .otto directory in your project's tree.  These are there to read and learn from.  Some AWS masters might chafe at the best practices, but everyone's gotta start somewhere, so it might as well be something sane.

I'm not sure if this is the best way to learn about AWS and the HashiCorp tools, but it's what I've done.  Your milage may vary.

Enjoy,
Charles.

HashiQuest

I'm starting a new project at work to build a new infrastructure for hosting our apps.  My objectives/requirements include:

• Elastic - something we can easily scale up and down.
• Redundant - something that can tolerate reasonable outages.  With the recent Xen security issues, I've had more than one ride in the reboot-rodeo, and I'm getting tired of that.
• Invented Here as opposed to Not Invented Here (NIH) - I inherited the existing infrastructure, and there's nothing wrong with it per-se, but because I didn't build it, it surprises me from time to time.
• Immutable - I want to get on the immutable infrastructure bandwagon because it's what the cool kids have been doing.  But, as I get into this, I realize that immutable infrastructure can lead to...
• Fearless - I want to be able to make changes quickly and easily without uttering "what could possibly go wrong?" before each change.

To achieve these objectives, I plan on using tools from HashiCorp to build out a pretty traditional infrastructure on Amazon Web Services.  I'm a big fan of HashiCorp and their tools.  Most of their tools are open source, which I like for cost and "religious" reasons.  Mitchell Hashimoto was my first guest on the SE Radio podcast when HashiCorp was just launching, and he's great.  Once my infrastructure is up and running, I look forward to using their Atlas tool to manage it all and pay Mitchell for all the great stuff he's done.

As mentioned, my initial plan is to build the first version of the infrastructure using AMIs running on EC2 instances as opposed to building Docker containers or running on Google Compute Engine. I made that decision in part to be more conservative (I hate explaining our current environment to prospective customers - no one ever got fired for picking IBM/Cicso/Amazon.)  However, by using HashiCorp tools, I am hoping that I can keep my options open in the future.

Hence, I have begun what I'm calling HashiQuest.  Stay tuned.

Charles.

A Pair of Interviews

I had two interviews released back-to-back on Software Engineering Radio around New Years:

• Episode 245: John Sonmez and I discussed his book Soft Skills - in particular the chapters on career management and marketing yourself.  He was a great guest.  I wish we could have gone over the whole book.
• Episode 246:  I interview John Wilkes from Google about the Borg cluster management software used at Google and Kubernetes. No one, except John, will ever know what a pain in the butt it was to record that episode - epic Skype fails the first time, but John was exceedingly helpful and understanding.

Check them out,

Charles.

A Pleasant Echo

My podcast interview with James Turnbull was transcribed and published by the nice folks at IEEE.  Check it out in May/June 2015 issue of IEEE Software.  That's something I wasn't expecting when I did the interview.

Charles.

Open Classes and Lazy Loading in Rails Don't Always Mix

Once upon a time, I came across an odd bug in some Rails code that was kicking my butt for some time.  The reason it perplexed me was it was a classic Heizenbug that seemed to come and go.  (That, and this was a side project that I couldn't devote much focused time to.)

The bug sometimes manifested itself with this error when running tests -
NoMethodError: undefined method `all' for SocialQueue:Class
where SocialQueue is an ActiveRecord model.  If I subsetted the tests being run to just the ones that failed, the bug would go away.  And, if I ran other tests before, the error would go away.   In other words, the act of trying to observe the bug would change it.

Another variant of the error I found when running the server in development mode was:
undefined method arel_table for QueuedPost:Class.  Again, QueuedPost is a model class, and I assume it has that method somewhere in the voodo that is ActiveRecord.

The error showed up when I added an "innocuous" tracing statement.  If I replaced the tracing statement with a puts statement, it worked.  If I put a return statement as the first statement in the tracing method, the error persisted - i.e., nothing in the body of that tracing method was causing harm.

The tracing method was in a module, in a separate file, and just requiring that module would cause the errors.  How could that be?  The module in question wasn't methods to be included in a class - it's just a name space to put this tracing method.  What's so bad about requiring a module?

What else is in that file?  Oh yeah, I have some code in there that opens my model classes to add a method to each class.  I put the new methods in that file, away from the rest of the model's definitions, because this tracing facility was experimental, and I didn't want to commit to modifying the model classes just yet.

In the words of Merlin Mann, "turns out" in development and test modes, Rails loads class definitions lazily.  When my module that opened model classes was loaded, the model classes hadn't necessarily been loaded.  If the model was loaded, it worked.  Otherwise, it wasn't opening an existing model class, but rather it was opening a new class.  Then, when my code that tried to use the models was run, there was already a definition for the class, so Rails didn't load the model class, and the object I thought was a model, was basically a lump of uselessness with the one method I intended to inject into a model class but none of the model methods.  I have since heard of lazy loading causing problems in STI.

The long-term solution for my problem is to move those new, injected methods into the main definitions of the models, now that my experiment is over, and I know I want to keep those methods.

In the interim, I came up with a simple hack that can be used anywhere you want to open a model class from some other file (or, maybe I've convinced you not bother - just edit the model).  Right before you open the model class, just mention it.  For example:
SomeModel
class SomeModel
  def my_new_method()
  end
end
Mentioning the model causes Rails to load it.  Then, when you open it, you're actually opening the real model class.

Perhaps there are better ways to skin this cat, but this works, and in the process, I learned about the existence and dangers lazy loading in Rails.

enjoy,
Charles.

James Turnbull on Docker

I interviewed James Turnbull on  the Software Engineering Radio podcast to discuss Docker.

Charles.

Mitchell Hashimoto on the Vagrant Project

I almost forgot to mention it here, but my first podcast episode with SE Radio went live the last week of July.  In it, I interviewed Mitchell Hashimoto about the Vagrant project.  Stay tuned for more...

Charles.