Typically, cucumber step definitions look like this:

When /^I select the recipient named "([^"]*)"$/ do |recipientname|
 within_jqm_page do
   page.find('a', :text => recipientname).click
 end
end

Step definitions implemented like this end up becoming a pain to re-use. Essentially you are calling a function, except to do so you have to construct a string, and then execute that. We’ve all seen how that ends.

Given /^all the entered recipient details are valid$/ do
  When %{I fill in "name" with "Valid Name"}
  And %{I fill in "phone" with "4152234567"}
  And %{I fill in "email1" with "valid.email@example.com"}
end

The direct outcome is very hard to maintain step definitions — like the above — and thus very poor quality cucumber tests. Patterns like page objects and personas take work to extract when following this approach. This approach that is analogous to writing VB6 applications in the early 2000s where all logic was just a double-click on a form button away. And as with VB6, that cucumber directly encourages this approach is part of the problem.

I hear a lot of complaints about cucumber these days. These complaints tend to target the English language test scripts: while business readable acceptance tests may be nice, these are useless if the business is never actually reading the tests.

The unspoken heart of this complaint seems to be the English language tests are responsible for the poor maintainability of the average cucumber test suite. I agree. But that shouldn’t be the case.

After trying to build well-factored cucumber suites on a few occasions, I pretty much hate the pattern of just attaching anonymous blocks to regular expressions. Instead, I want step definitions that directly invoke methods. Something like:

Given /^I am on the (\w+) page$/ => :go_to_known_page,
  :on => { current_user }

class User
  def self.current_user
    …
  end

  def go_to_known_page(page_name)
    visit Page.page_named(page_name)
  end
end

World(User)

The significant part is in the first two lines. Instead of passing a block to the Given, a symbol for a method to invoke would be passed. This method would be invoked with the arguments matched by the regular expression, after any Transforms have been applied. The :on is optional. This is a block evaluated against the World. If :on is provided, the method would be sent to the result of this block instead.

Yes, this would force you to define classes and attach methods to those. Yes, this would require you to think about the interrelationships of your objects. Yes, this is quite a lot more limited than an arbitrary block of code. Yes, that is exactly the point.

After a quick glance through the cucumber source this does not look difficult to add. Should I go to the effort of adding it?

In the world of adults the solution to these problems is known as dependency management. There are now two tools to help Ruby: rvm and Bundler. These are both opinionated pieces of software that try to do a lot to help. Often far too much. After digging quite deeply into Bundler with Nick Drew on a recent project, here are some things that we learnt that I’ve tried to distill into a set of guides on good ways to use Bundler and rvm. This is just what I’ve discovered, if there are any tips or better ways to use these tools, or other tools that help, please share.

This is going to assume that you already know something about both.

• Firstly, use rvm and Bundler. If you have used them in the past and were unhappy, try again. The early versions had some truly horrible bugs. Development has moved quickly and things are a lot better now. If you’re using Rails 3 then you have little choice about Bundler. But for any projects you should use these.
• Use rvm to install your Ruby, don’t use the system Ruby. Create at least one gemset and use that.

• Create a .rvmrc for your project and check it into source control. Naming the gemset after your project, it should probably look something like:

  rvm_install_on_use_flag=1
  rvm --create use ruby-1.8.7-p330@project
  

• Use rvm on your build box. But not in production.

• Create a personal gem set and toss the gems you use on simple little scripts into that.
• Install Bundler into the global gemset for the Ruby you are using.
• Include development and test groups in your Gemfile. Be strict about the difference between a gem required to run the build system (development), used in tests (test — obviously) and a gem required to run the application.
• Don’t check the .bundle/config file into source control. Bundler often chooses to remember settings in this file. This will cause much irritation. This is an example of being more helpful than is helpful.
• Don’t use the --deployment switch to bundle install unless you really mean that. As well as producing a local bundle of gems, installing in deployment mode will freeze the Gemfile. This is remembered and will cause much frustration when you’re trying to add new gems. If this does happen you will get an opaque error complaining that you have not checked your Gemfile.lock into source control. To get around this, just delete your .bundle/config. You’re doing that to remove the remembered setting about freezing your gems.

• Your build script (the one you run before committing) should call bundle package. In fact, it should look quite a lot like:

  bundle package && bundle install && rake spec features
  

• Check the gem files in vendor/cache into source control.

• Definitely use bundle install --deployment --local --without="development test" either as part of your Capistrano tasks or as part of packaging for a release. Every one of those switches except the first should not be required. But they are. So there you go.
• Bundler attempts to take over your world. This is intensely irritating. Expect to be annoyed.

• If you ever need to start a Ruby program from your Ruby program but in a different Bundler world, you will need to do some cleaning of your environment. In particular you will need to clean the environment variables BUNDLE_GEMFILE, BUNDLE_PATH, BUNDLE_BIN_PATH and most surprisingly RUBYOPT. You may find the following function helpful.

  def without_bundler_env
    original_env = ENV.to_hash
    ENV.delete("BUNDLE_GEMFILE")
    ENV.delete("BUNDLE_PATH")
    ENV.delete("BUNDLE_BIN_PATH")
    ENV.delete("RUBYOPT")
    yield
  ensure
    ENV.replace(original_env.to_hash)
  end
  

• If you need to examine the Gemfile from another Ruby project in your Bundler controlled world, you will think that you can use Bundler::Definition.build("<i>path to Gemfile</i>", "<i>path to Gemfile.lock</i>", nil). That would be a good start. You will also need to set ENV["BUNDLE_GEMFILE"] to point to that file and call resolve_with_cache!, however. Try the following:

  ENV['BUNDLE_GEMFILE'] = 'Gemfile'
  bundle_def = Bundler::Definition.build('Gemfile',
                                         'Gemfile.lock',
                                         nil)
  bundle_def.resolve_with_cache!
  

Ultimately, the goal of Bundler and rvm are noble. Dependency management is a hard problem. And there are no alternatives if you are choosing to use Ruby. Are there things that could perhaps be done better? Absolutely. But there is probably nothing so bad that some patterns couldn’t fix. So please, let’s all work out how to get the most out of these tools.

Well, a script isn’t much use to me or anyone else: you can’t run a script on an iPhone. So like all software wants to be, this script is now a simple little web application: mutual. Please be gentle, and I know I’m not a designer.

This should be the start of more useful little applications like this appearing. I’ll blog about that. Also, for the programmers out there, this is written using the Sinatra Ruby web application framework. If you thought writing apps using Rails was fast, well, you ain’t seen nothing til you’ve seen Ol’ Blue Eyes take the stage.

I want to be able to ask the question ‘Who do we know in common?’, in short. A useful question, but one that can take quite a while to answer using the web site. I asked the lazy twitterverse if there was already an app for this, but my twitterverse is too small to get an answer. So, I wrote my own script. I don’t have any handy web space to run this from, so you’ll have to grab it and run it yourself. You will need to install the twitter4r gem first:

sudo gem install twitter4r

Then paste the following code into a Ruby file, and run. It takes two parameters, the names of the two users for who you want to find common ground.

require 'rubygems'
require 'open-uri'
require 'rexml/document'
require 'twitter'

class Twitter::User

  def all_friends
    users = friends.map { |f| f.screen_name }
    # If there's more than one page of users, we've already got the
    # first one
    page = 2
    found_users = friends.length
    while found_users >= 100
      found_users = 0
      open("http://twitter.com/statuses/friends/#{screen_name}.xml?page=#{page}") do |f|
        users_doc = REXML::Document.new(f.readlines.join(''))
        users_doc.elements.each('/users/user/screen_name') do |friend_name|
          users << friend_name.text
          found_users += 1
        end
      end
      page += 1
    end
    users
  end

end

def in_common(my_friends, other_friends)
  my_friends.select { |m_n| m_n if other_friends.member? m_n }
end

def main(me, other)
  c = Twitter::Client.new
  me_friends = c.user(me).all_friends
  other_friends = c.user(other).all_friends
  in_common(me_friends, other_friends).each do |f|
    puts "  #{f}"
  end
end

main(ARGV.shift, ARGV.shift)

Enjoy, and please let me know how it works out for you, or if you make any changes. And by the way, *this* is why RESTful APIs rock.

We’re pretty happy with this set up. It’s already found a couple of bugs in some plugins we’re using. More on that in an upcoming post. Here’s how we made our automatic deployment work. We’re using Capistrano for our deployment scripts, we’re deploying to Phusion Passenger running under Apache on FreeBSD and our continuous integration server runs an Ant script.

These instructions describe how to set up a Apache 2.2 web server with Phusion Passenger on FreeBSD; the Ant script to automatically deploy and how to configure a Rails app to be deployed like this.

This will give you two new environments for your apps: DEVTEST and UAT. UAT is a user acceptance testing environment, our system testers and analysts use and own this environment. We don’t automatically deploy to here, we release to here. DEVTEST is the environment we automatically deploy to.

Setting up Your Server

Installing Phusion Passenger

Installing Phusion Passenger on a FreeBSD server is no different to installing anywhere else:

$ sudo gem install passenger
$ sudo passenger-install-apache2-module

Configuring Apache

At the end of the second step, the installer tells you to add some config to the end of your Apache config. On FreeBSD, edit this with:

$ sudoedit /usr/local/etc/apache22/httpd.conf

And then add the following at the end:

LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.3
PassengerRuby /usr/local/bin/ruby18
NameVirtualHost *:80
<VirtualHost *:80>
    ServerName devtest.example.com
    ServerAlias devtest
    DocumentRoot /usr/local/www/rails/devtest
    <Directory "/usr/local/www/rails/devtest">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    </Directory>
    RailsEnv "devtest"
</VirtualHost>
<VirtualHost *:80>
    ServerName uat.example.com
    ServerAlias uat
    DocumentRoot /usr/local/www/rails/uat
    <Directory "/usr/local/www/rails/uat">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    </Directory>
    RailsEnv "uat"
</VirtualHost>

Unless you want to use two different servers for the two environments, you’ll need to use named virtual hosts, and ask your friendly administrator to add CNAME records to your DNS server pointing devtest and uat at the same physical server. They’ll know what you mean.

Create a Local User

You’ll need a local user on your server. This is the user that will run the automatic deployments.

$ sudo adduser
Username: deploy-robot
Full name: Deployment Robot
Uid (Leave empty for default):
Login group [deploy-robot]:
Login group is deploy-robot. Invite deploy-robot into other groups? []: www
Login class [default]:
Shell (sh csh tcsh zsh nologin) [sh]: 
Home directory [/home/deploy-robot]:
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]:
Username   : deploy-robot
Password   : ****
Full Name  : Deployment Robot
Uid        : 1001
Class      :
Groups     : 
Home       : /home/deploy-robot
Shell      : /usr/local/bin/sh
Locked     : no
OK? (yes/no): yes
adduser: INFO: Successfully added (deploy-robot) to the user database.
Add another user? (yes/no): no
Goodbye!

Deployment Directories

Set up the directories to hold your applications.

$ sudo mkdir -p /usr/local/www/rails/devtest
$ sudo mkdir -p /usr/local/www/rails/uat

These are the web roots for each of the environments, but applications will not be deployed here. Instead, symlinks will be created from here to where the applications are actually deployed.

$ sudo mkdir -p /usr/local/app/rails/devtest
$ sudo mkdir -p /usr/local/app/rails/uat

These last two directories, and everything under them should be owned by the deployment user you created above.

$ sudo chown -R deploy-robot:www devtest uat

Gems

Finally, there are some gems you’ll need installed on the target deployment server. Some of these depend on FreeBSD ports.

$ cd /usr/ports/comms/ruby-termios
$ sudo make install clean

And then just a couple of gems.

$ sudo gem install termios
$ sudo gem install capistrano

And that’s it for initial server configuration. There will be some more configuration when first deploying an application.

Preparing Your Application

Capistrano Config

Capify your application:

$ cd app
$ capify .

Edit your capistrano rules in deploy.rb. You’ll want them to look something like the following. These rules use no source control system to get the code. Our continuous integration server takes care of checking out the code, so it’s easier to deploy from the local code copy. And, this way we can be sure each deployment only contains one changelist.

# Overall config
set :use_sudo, false
# Application config
set :application, "app-name"
set :default_env, "production"
set :rails_env, ENV['RAILS_ENV'] || default_env
# Deployment source and strategy
set :deploy_to, "/usr/local/app/rails/#{rails_env}/#{application}"
set :deploy_via, :copy
set :scm, :none
set :repository,  "."
# Target servers
set :default_server, "localhost"
set :dest_server, ENV['SERVER'] || default_server
role :app, dest_server
role :web, dest_server
role :db,  dest_server, :primary => true
# Phusion Passenger specific restart task
namespace :deploy do
    desc "Restart Application"
    task :restart, :roles => :app do
        run "touch #{current_path}/tmp/restart.txt"
    end
end

Environment Configuration

Set up the two new environments for your application.

$ cp config/environments/production.rb config/environments/devtest.rb
$ cp config/environments/production.rb config/environments/uat.rb

Somewhere inside both those files you’ll need to set the RAILS_RELATIVE_URL_ROOT as the application will be running at a sub-URI on your server and Rails needs to know that. Something like:

ENV['RAILS_RELATIVE_URL_ROOT'] = "/app-name"

The two new environments will also need to be described in your database.yml file. This of course depends on your specific database server setup, so I’ll leave that bit to you.

Server-side Application Setup

Apache needs to know about the applications, and there needs to be symlinks from the web root to the application deployment folder. This setup only needs to be done once for each application.

To add the application to Apache, edit /usr/local/etc/apache22/httpd.conf again, and in the VirtualHost section for the devtest environment, add a line like the following:

RailsBaseURI /app-name

Now, set up the symlink:

$ ln -s /usr/local/app/rails/devtest/app-name/current/public /usr/local/www/rails/devtest/app-name

And you’re done with the application configuration.

Ant Deployment Scripts

Our company has an in-house continuous integration server. We’d be too embarrassed at cocktail parties if we didn’t have our own. Yes, yes, I know this is completely ridiculous. And to make it even worse, it only runs Ant scripts. Sigh. Anyway, here’s how you make Ant automatically deploy an application to devtest.

In a file called definitions.xml:

<project name="definitions_rake">
    <macrodef name="rake">
        <attribute name="app" />
        <attribute name="target" />
        <element name="variables" optional="true" />
        <sequential>
            <exec executable="rake" dir="@{app}" failonerror="true">
                <arg value="@{target}" />
                <variables />
            </exec>
        </sequential>
    </macrodef>
    <macrodef name="capistrano">
            <attribute name="app" />
            <attribute name="environment" />
            <attribute name="task" />
            <sequential>
                <exec executable="cap" dir="@{app}" failonerror="true">
                    <env key="RAILS_ENV" value="@{environment}" />
                    <env key="SERVER" value="${project.server}" />
                    <arg value="@{task}" />
                    <arg value="-s" />
                    <arg value="user=${project.user}" />
                    <arg value="-s" />
                    <arg value="password=${project.password}" />
                </exec>
            </sequential>
    </macrodef>
    <macrodef name="deploy">
        <attribute name="app" />
        <attribute name="environment" />
        <sequential>
            <capistrano app="@{app}" environment="@{environment}" task="deploy:setup" />
            <capistrano app="@{app}" environment="@{environment}" task="deploy:migrations" />
        </sequential>
    </macrodef>
    <macrodef name="test">
        <attribute name="app" />
        <sequential>
            <rake app="@{app}" target="db:migrate" />
            <rake app="@{app}" target="test" />
            <rake app="@{app}" target="spec" />
        </sequential>
    </macrodef>
</project>

Ant macros, while quite insane, are generally a better way to define new tasks than the complete insanity of trying to write a whole Ant plugin in Java. These macros define low-level tasks to run rake and capistrano tasks, and then use these to build up higher level tasks like test and deploy. All these tasks assume that Ant has been run from the directory immediately above your Rails app directory.

In a file called project.properties, set your server, user name and password. Having the password here is unfortunate, but it is a local account, with limited privileges on an internal server. Your call.

user=deploy-robot
password=deploy-robot-password
server=deployment-server

In a file called build.xml:

<project name="aegean" default="build">
    <import file="./definitions.xml" />
    <property file="project.properties" prefix="project" />
    <!-- Sample application.
         To add a new application:
         1. Copy the following targets.
         2. Replace 'depot' with your Rails app name.
         3. Add the 'app name' target as a dependency of the target 'build'.
    <target name="depot.deploy.devtest">
           <deploy app="depot" environment="devtest" />
    </target>
    <target name="depot.test">
           <test app="depot" />
    </target>
    <target name="depot" depends="depot.deploy.devtest, depot.test" />
    -->
    <target name="example.deploy.devtest">
           <deploy app="example" environment="devtest" />
    </target>
    <target name="example.test">
           <test app="example" />
    </target>
    <target name="example" depends="example.deploy.devtest, example.test" />
    <target name="build" depends="example" />
</project>

The large comment block is just helpful for other developers trying to add another application. From here, to try this out:

$ ant

It should run the deployment, and then run the test suites. If that works as you expect, then just configure your continuous integration server to run Ant over that file on every submit.

Hopefully this is of use to someone. Though this is how our environment is configured, I have written this all from memory, so I might have missed a critical step somewhere. Please let me know if there’s anything that needs to be changed.