Author: Kerry Buckley

I was recently working with an ActiveRecord class that exposed some attributes retrieved from a remote API, rather than from the database. The rules for handling the remote attributes were as follows:

• If the record is unsaved, return the local value of the attribute, even if it’s nil.
• If the record is saved and we don’t have a local value, call the remote API and remember and return the value.
• If the record is saved and we already have a local value, return that.

Here’s the original code (names changed to protect the innocent):

class MyModel < ActiveRecord::Base
  attr_writer :foo, :bar

  def foo
    (new_record? || @foo) ? @foo : remote_object.foo
  end

  def bar
    (new_record? || @bar) ? @bar : remote_object.bar
  end

  def remote_object
    @remote_object ||= RemoteService.remote_object
  end
end

The remote_object method makes a call to the remote service, and memoises the returned object (which contains all the attributes we are interested in).

I didn't really like the duplication in all these accessor methods – we had more than the two I've shown here – so decided to factor it out into a common remote_attr_reader class method. Originally I had the method take a block which returned the remote value, but that made the tests more complicated, so I ended up using convention over configuration and having the accessor for foo call a remote_foo method.

Here's the new code in the model:

class MyModel < ActiveRecord::Base
  remote_attr_reader :foo, :bar

  def remote_foo
    remote_object.foo
  end

  def remote_bar
    remote_object.bar
  end

  def remote_object
    @remote_object ||= RemoteService.remote_object
  end
end

Here's the RemoteAttrReader module that makes it possible:

module RemoteAttrReader
  def remote_attr_reader *names
    names.each do |name|
      attr_writer name
      define_method name do
        if new_record? || instance_variable_get("@#{name}")
          instance_eval "@#{name}"
        else
          instance_eval "remote_#{name}"
        end
      end
    end
  end
end

To make the module available to all models, I added an initialiser containing this line:

ActiveRecord::Base.send :extend, RemoteAttrReader

Here's the spec for the module:

require File.dirname(__FILE__) + '/../spec_helper'

class RemoteAttrReaderTestClass
  extend RemoteAttrReader
  remote_attr_reader :foo

  def remote_foo
    "remote value"
  end
end

describe RemoteAttrReader do
  let(:model) { RemoteAttrReaderTestClass.new }

  describe "for an unsaved object" do
    before do
      model.stub(:new_record?).and_return true
    end

    describe "When the attribute is not set" do
      it "returns nil" do
        model.foo.should be_nil
      end
    end

    describe "When the attribute is set" do
      before do
        model.foo = "foo"
      end

      it "returns the attribute" do
        model.foo.should == "foo"
      end
    end
  end

  describe "for a saved object" do
    before do
      model.stub(:new_record?).and_return false
    end

    describe "When the attribute is set" do
      before do
        model.foo = "foo"
      end

      it "returns the attribute" do
        model.foo.should == "foo"
      end
    end

    describe "When the attribute is not set" do
      it "returns the result of calling remote_" do
        model.foo.should == "remote value"
      end
    end
  end
end

To simplify testing of the model, I created a matcher, which I put into a file in spec/support:

class ExposeRemoteAttribute
  def initialize attribute
    @attribute = attribute
  end

  def matches? model
    @model = model
    return false unless model.send(@attribute).nil?
    model.send "#{@attribute}=", "foo"
    return false unless model.send(@attribute) == "foo"
    model.stub(:new_record?).and_return false
    return false unless model.send(@attribute) == "foo"
    model.send "#{@attribute}=", nil
    model.stub("remote_#{@attribute}").and_return "bar"
    model.send(@attribute) == "bar"
  end

  def failure_message_for_should
    "expected #{@model.class} to expose remote attribute #{@attribute}"
  end

  def failure_message_for_should_not
    "expected #{@model.class} not to expose remote attribute #{@attribute}"
  end

  def description
    "expose remote attribute #{@attribute}"
  end
end

def expose_remote_attribute expected
  ExposeRemoteAttribute.new expected
end

Testing the model now becomes a simple case of testing the remote_ methods in isolation, and using the matcher to test the behaviour of the remote_attr_reader call(s).

require File.dirname(__FILE__) + '/../spec_helper'

describe MyModel do
  it { should expose_remote_attribute(:name) }
  it { should expose_remote_attribute(:origin_server) }
  it { should expose_remote_attribute(:delivery_domain) }

  describe "reading remote foo" do
    # test as a normal method
  end
end

[tags]ruby,rails,activerecord,metaprogramming,rspec,matcher,refactoring,dry[/tags]

Over the years I’ve tried a number of approaches for managing gem dependencies in a Rails project. Here’s a quick round-up of what I’ve tried, and the pros and cons of each.

Just use what’s on the system

This is probably most people’s default approach when first starting with Rails. Just sudo gem install whatever you need, require the appropriate gems (either in environment.rb or in the class that uses them), and you’re away.

This mostly works OK for small projects where you’re the only developer, but you still need to make sure the right gems are installed on the machine you’re deploying the application to.

Worse, though, is what happens when you come back to the project after a while, various gems have been updated, and things mysteriously don’t work any more. Not only do you have to mess around getting the code to work with the latest gem versions, but you probably don’t even know exactly which versions it used to work with.

Freeze (unpack) gems

I think I first came across this technique in Err the Blog’s Vendor Everything post. The idea is to install copies of all your gems into the project’s vendor/gems directory, meaning that wherever the code is running, you can guarantee that it has the correct versions of all its dependencies.

This got much easier in Rails 2.1, which allowed you to specify all your gems using config.gem lines in environment.rb (you can also put gems only needed in specific environments in the appropriate file, eg you might only want to list things like rspec and cucumber in config/enviroments/test.rb). You can then run sudo rake gems:install to install any gems that aren’t on your system, and rake gems:unpack to freeze them into vendor/rails, and be sure that wherever you check out or deploy the code, you’ll be running the same versions of the gems. There’s even a gems:build task to deal with gems that have native code (but more on that later).

Subsequent versions of Rails have improved on the original rake tasks – dependencies are now handled much better, for example – but there are still a few problems. The main one is the handling of gems that are required by rake tasks in your project, rather than just from your application code.

When you call a rake task in your Rails project, this is more-or-less what happens (I may have got some of the details slightly wrong):

1. The top-level Rakefile is loaded.
2. This in turn requires config/boot.rb, but not config/environment.rb.
3. It then requires some standard rake stuff, and finally tasks/rails (which is part of Rails – specifically railties). This finds and requires all the .rake files in your plugins and your project’s lib/rake directory.

The problems start when you have a task depends on the rails environment task, and also requires a gem which is listed in environment.rb. Because the gem-loading magic only happens when the environment is loaded, the rake task will be blissfully unaware of your frozen gems, and will load them from the system instead.

If the system gem is newer than the frozen one, you get errors like this:

can't activate foo (= 1.2.3, runtime) for [], already activated foo-1.2.4 for []

If you work on two projects that use different versions of a gem like this, you end up having to uninstall and reinstall them as you switch from one to the other, which gets tedious fairly quickly.

Specify gems, but don’t freeze

You can get round the wrong-version problem to some extent by specifying version numbers in environment.yml as ‘>=x.z.y’ (or by not specifying them at all). If you’re doing that, though, there’s not really much benefit in unpacking the gems, and you may as well just use rake gems:install to make sure they’re on the system. Of course the downside of this approach is that you can’t be sure that everyone’s running the exact same versions of the gems. Worse still, you can’t be sure that what’s on your production box matches your development and test environments.

GemInstaller

GemInstaller solves most of the problems with the built-in Rails gem management by running as a preinitializer, meaning it gets loaded before the other boot.rb gubbins.

GemInstaller uses the gems installed on the system rather than freezing them into the project, but because it gets to run first it ensures that the correct versions are used, even if there are newer versions installed. By default it checks your project’s gem list and installs anything that’s missing every time it runs (which is whenever you start a server, run the console, execute a rake task etc). You create a YAML file listing the gems you need (dependencies are handled automatically), and other options such as an HTTP proxy if necessary.

Of course on Unix-like systems, which is most of them (although I hear there are still people developing Rails projects on Windows), gems are generally installed as root. GemInstaller can get round this in two ways – either by setting the --sudo option and setting a rule in /etc/sudoers to allow the appropriate user(s) to run the gem commands as root without having to provide a password, or by using the built-in gem behaviour that falls back to installing in ~/.gem.

Personally I like to keep all my gems in one place, accessible to any user, so I went for the sudo approach. The only problem with this is that it uses sudo for all gem commands, rather than just install or update, which means it runs a sudo gem list every time your app starts up. Depending on the way you have Apache and Passenger set up this may mean granting sudo access to what should be a low-privileged user.

I ended up disabling the automatic updating of gems, and just warning when they’re missing instead. In fact later versions of GemInstaller don’t try to handle the update automatically anyway.

I created a separate script to do the update, which can be run manually, on a post-merge git hook, or as part of the Capistrano deployment task.

Because GemInstaller needs to go out to the network to fetch any new or updated gems, things get a bit more painful (as always) if you are unfortunate enough to be stuck behind a corporate HTTP proxy. Actually it’s easy enough to configure if you’re always behind a proxy, but it gets slightly trickier if your web access is sometimes proxied and sometimes direct. Nothing that can’t be solved of course.

Unfortunately you can still end up with version conflicts if a gem is required by one you have specified, then you explicitly require an older version, but these can usually be resolved by shuffling the order of the gems in geminstaller.yml.

Bundler

Bundler is the newest kid on the gem management block, and looks to have solved pretty much all the problems faced by the other approaches. It’s based on the gem management approach from Merb, and can be used in any Ruby project (not just Rails).

Bundler works by unpacking gems into the project (I recommend using a directory other than the default vendor/gems to avoid confusing Rails – this can be configured by setting bundle_path and bin_path in the Gemfile), but the intention is that you only commit the .gem files in the cache directory to source control. Gems are then installed locally within the project, including any platform-specific native code as well as the commands in bin.

Because Bundler resolves all dependencies up-front, you only need to specify the gems you’re using explicitly, and let it handle the rest, which hopefully means an end to version conflicts at last.

Here’s an example Gemfile:

source 'http://gemcutter.org'
source 'http://gems.github.com'
bundle_path 'vendor/bundled_gems'
bin_path 'vendor/bundled_gems/bin'

gem 'rails', '2.3.4'
gem 'bundler', '0.6.0'

gem 'capistrano', '2.5.8'
gem 'capistrano-ext', '1.2.1'
gem 'cucumber', '0.4.3', :except => :production
# [more gems here]

disable_system_gems

Note the two additional sources (rubyforge.org is configured by default), the path overrides, and the last line, which removes the system gems from the paths, avoiding any potential confusion.

I’ve put this in config/preinitializer.rb to update from the cached gems on startup (this doesn’t hit the network):

$stderr.puts 'Updating bundled gems...'
system 'gem bundle --cached'
require "#{RAILS_ROOT}/vendor/bundled_gems/environment"

To avoid any startup delays after an upgrade, I also call system 'gem bundle --cached' from the after_update_code hook in the capfile.

Finally, to make sure only the .gem files are checked in, add these lines to .gitignore (you’ll still need to explicitly git add the bundled_gems/cache directory):

vendor/bundled_gems
!vendor/bundled_gems/cache

[Update 3 November] Yehuda Katz just posted an article all about Bundler, including features coming in the imminent 0.7 release.

[tags]ruby,rails,gems,rubygems,geminstaller,bundler[/tags]