I’ve had a long-running open source crush on Vagrant - kind of like a fanboy from a distance. I’ve long fantasized about conjuring up a pristine development server on my local workstation, provisioning some software to put it into a production-like state, and having it start serving requests for my application. And even though I’ve worked on several projects where it would have been a great fit, I’ve never found the time to sit down and really dig into it until now. The work I’m planning for BetterFBO will probably require me to set up infrastructure components that are a bit outside my usual development stack, so it seems like now is the time to dive a little deeper into Vagrant and Chef in order to better manage the configuration and avoid installing these pieces directly on my workstation.

Initial provisioning of virtual machines can take some time and experimentation, so my initial intermediate goal was to package up a box that would serve as a starting point for all future VMs. Fortunately, Vagrant makes this pretty simple.

Step 1: Install and Set Up Vagrant

I’ve had Vagrant installed for a long time, but it was in need of an upgrade. It used to be that this was a matter of installing the gem, but over time Mitchell has tried to steer the product toward a more general audience rather than just Ruby developers, and he’s been releasing platform-specific installation packages. Grab the right one for your OS of choice and install it on your system. When it’s done, you should have a vagrant executable installed somewhere on your $PATH which you can test out by typing vagrant -h on the command line. It should respond by printing out something like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

ck1@ramanujan:~/Projects/vagrant_berkshelf_example$ vagrant -h
Usage: vagrant [-v] [-h] command [<args>]

    -v, --version                    Print the version and exit.
    -h, --help                       Print this help.
      
Available subcommands:
     box          manages boxes: installation, removal, etc.
     destroy      stops and deletes all traces of the vagrant machine
     halt         stops the vagrant machine
     help         shows the help for a subcommand
     init         initializes a new Vagrant environment by creating a Vagrantfile
     package      packages a running vagrant environment into a box
     plugin       manages plugins: install, uninstall, update, etc.
     provision    provisions the vagrant machine
     reload       restarts vagrant machine, loads new Vagrantfile configuration
     resume       resume a suspended vagrant machine
     ssh          connects to machine via SSH
     ssh-config   outputs OpenSSH valid configuration to connect to the machine
     status       outputs status of the vagrant machine
     suspend      suspends the machine
     up           starts and provisions the vagrant environment
     vbguest

For help on any individual command run `vagrant COMMAND -h`

Step 2: Install a Base Box

Next, you’ll need to install a Vagrant box to serve as the starting point for your own. These can be found at the old Vagrantbox.es, which is a public directory of privately hosted boxes, or the new VagrantCloud, which is currently in beta and looks like it will be a box hosting service using a free-paid hybrid business model similar to GitHub. I generally use Ubuntu Server for production deployments, so I’m starting with a the most recent long-term support (LTS) version of that which happens to be 12.04 Precise Pangolin. In order to save myself some time, I created a base box with a baseline set of software packages installed and configured to save myself time during provisioning and called it chriskottom-precise64.box and added it to Vagrant with the command:

1

vagrant box add precise64 http://files.vagrantup.com/precise64.box

Step 3: Set Up Bundler

You’ll start by setting up your Gemfile with two gems. Chef is what you’ll use for provisioning software and managing the configuration on the virtual machines in your development environment, while Berkshelf provides a way of managing the cookbooks you’ll use for that purpose. In the end, your Gemfile should look like this:

1
2
3
4

source "https://rubygems.org"

gem "chef"
gem "berkshelf"

Step 4: Write a Chef Cookbook to Install the Required Software

Chef makes it pretty easy to define instructions for provisioning hardware in the form of cookbooks which define the instructions, configuration parameters, and templates needed to put a machine into some desired state. The syntax and libraries it provides for defining the instructions, known in Chef lingo as recipes, offers a platform-independent way of performing common tasks like installing packages, creating scheduled tasks, controlling users and groups, and so on. Complete specs of the DSL syntax and the available resources can be found on the Chef documentation site, but for now, we’ll define a simple cookbook in just a few steps.

Chef is packaged with a command-line tool known as knife that provides a bunch of utility functions for working with the software. Here we’ll use it to create a new cookbook called chriskottom_precise64 in the directory cookbooks with the following one-liner:

1

knife cookbook create chriskottom_precise64 -o cookbooks

The resulting directory structure in the cookbook directory you specified is comprehensive:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

ck1@ramanujan:~/Projects/vagrant_base_box$ tree cookbooks/chriskottom_precise64
cookbooks/chriskottom_precise64
├── attributes
├── CHANGELOG.md
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── README.md
├── recipes
│   └── default.rb
├── resources
└── templates
    └── default

For the purposes of this exercise, we only retain three files - metadata.rb, README.md, and recipes/default.rb - and the attributes/ directory, so you can delete everything else.

In attributes/ you’ll want to create a file called default.rb and use it to define the array of package names you want to install. This is what my list looks like:

1
2
3
4
5
6
7

default['chriskottom_precise64']['packages'] =
  %w( make automake autoconf gcc build-essential xinetd sharutils git-core
      wget htop tree chkconfig traceroute sysstat iptraf nmap ngrep ack iotop
      iftop ntp emacs23-nox sqlite3 openssl libsqlite3-dev libxml2-dev
      libxslt-dev libreadline-dev zlib1g zlib1g-dev libssl-dev libc6-dev
      libyaml-dev libcurl4-openssl-dev ncurses-dev libncurses5-dev libgdbm-dev
      libffi-dev libtool bison )

Then we only need to set up the default recipe to install all the packages on the list:

1
2
3
4
5
6

packages = node['chriskottom_precise64']['packages']
packages.each do |pkg|
  package pkg do
    action :install
  end
end

Step 5: Set Up Berkshelf

Berkshelf wasn’t a tool that I’d used before, but you can think of it as being like Bundler but for managing Chef cookbooks instead of Ruby gems with similar command line semantics and a manifest format similar to the one used for Gemfiles.

Set up your own Berksfile manifest by entering berks init on the command line and declare the cookbooks you’ll need to provision your environment by editing the Berksfile. It should end up looking like this:

1
2
3
4
5
6

site :opscode

cookbook 'apt'
cookbook 'chriskottom_precise64', path: 'cookbooks/chriskottom_precise64'
cookbook 'rbenv', github: 'fnichol/chef-rbenv', tag: 'v0.7.2'
cookbook 'ruby_build'

This is a simple example, but it showcases examples of Berkshelf’s ability to acquire cookbooks from a variety of resources: from the Opscode Community repository, from a GitHub repository, or from a local directory source.

Berkshelf implements a berks install command that can be used to download and install all the required cookbooks and their dependencies, but because the two are used together often, they’ve developed a Vagrant plugin that provides direct integration and handles the downloading of required cookbooks whenever you provision a VM. Install the plugin by typing vagrant plugin install vagrant-berkshelf.

It’s worth a brief comment here explaining why I’m installing the rbenv' andruby_build` cookbooks. The Ubuntu distribution I’m using as a baseline was originally released in 2012, and the new LTS won’t drop for a few weeks from this writing and won’t be installed on my production servers for a while after that. While these releases are still suitable and supported, they do contain some antiquated software including old versions of Ruby that won’t run some more recently developed Chef cookbooks. In order to get around this, I replace the bare-metal Rubies installed on most Vagrant base boxes with something more recent, and I use rbenv and ruby-build to manage the Ruby installations on my machines since it provides a little more ease of use in development and all other environments.

Step 6: Set Up Vagrant

Vagrant virtual machines are configured by way of a Vagrantfile in the project directory. You can create an initial template with the vagrant init command and edit it like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

VAGRANTFILE_API_VERSION = '2'

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = 'precise64'
  config.berkshelf.enabled = true

  config.vm.provision :chef_solo do |chef|
    chef.add_recipe 'apt'
    chef.add_recipe 'chriskottom_precise64'
    chef.add_recipe 'ruby_build'
    chef.add_recipe 'rbenv::system'
    chef.add_recipe 'rbenv::vagrant'
    chef.json = {
      rbenv: {
        global: '2.1.0',
        rubies: [ '2.1.0' ],
        upgrade: true,
        gems: {
          '2.1.0' => [{ name: 'bundler' },
                      { name: 'main' },
                      { name: 'map' },
                      { name: 'open4' },
                      { name: 'multi_json' },
                      { name: 'net-ssh', version: '~> 2.2.0' },
                      { name: 'aws-sdk' },
                      { name: 'chef' },
                      { name: 'ohai' }]
        }
      },
  }
  end
end

Let’s look at what I’ve done here: 1. I specified that I want to use the precise64 box as the basis for all created virtual machines. 2. I activate the Berkshelf plugin to manage Chef cookbook dependencies. 3. I indicate that I want to use the :chef-solo provisioner when setting up the VM. 4. I specify a list of Chef recipes that should be executed, in order, on the newly created VM. 5. I provide a Hash of parameters that the recipes will need to do their work.

(Note: Yes, I’m aware that Ruby 2.1.1 has been released, but some combination of Ubuntu Precise, the libraries that are installed, and Ruby 2.1.1 seems to make Nokogiri compilation choke. Downgrading Ruby to 2.1.0 seemed to solve the problem.)

Step 6: Vagrant UP!

It’s time to light this thing up with vagrant up.

Just to set expectations, the first run will take some time (dependent on your system specs and your connection throughput) due to the need to install loads of voluminous software, but when it finishes, you’ll have a running VM with all the required software.

Step 7: Package the Box

Vagrant lets you package up a running VM as a box from the command line with one line:

1

vagrant package --output chriskottom-precise64.box

The finished product should be a portable tar file called chriskottom-precise64.box containing the box configuration and disk image.

Step 8: Add the Box to Vagrant

Before the new box can be used as the basis for another project, you’ll need to import it into Vagrant’s library using the following command:

1

vagrant box add chriskottom-precise64 chriskottom-precise64.box

And that’s it. Future projects can now refer to the chriskottom-precise64 box and have a VM that is recently updated and running a current version of Ruby.

I put the code up in a GitHub repo for anyone who’s interested. Feel free to use it for your own explorations or as the basis for building something awesome.