Hosting RubyGems for local use

Last week at work I set up my own RubyGems repository to host some local builds of upstream gems. Following the recommendation on guides.rubygems.org, I tried out Gem in a Box. It works perfectly as advertised, but the setup is largely left as an exercise for the reader (who is assumed to know their way around a web app deployment process).

I’ll run through the process I used both as a step-by-step guide to setting up a reasonably locked-down and authenticated (for upload) gem server, and as an example of how to deploy an arbitrary Ruby web app using a sane, repeatable process.

Gem in a Box consists of a Sinatra app and a RubyGems plugin, packaged together in a single gem. We could thus just gem install geminabox and go from there, but we want to do this the Right Way(tm). That means:

• Being able to manage dependencies on deployment easily: Bundler
• Using a deployment tool: Vlad
• Keeping our setup in source control: Mercurial

Substitute Capistrano, Subversion, Git, or other tools as appropriate; the above make up my preferred toolkit. We’ll also be deploying with Unicorn, and we’ll want the appropriate Vlad plugins (namely vlad-hg and vlad-unicorn) to ease our deployment too.

1. Create a repository

$ hg init boxogems
$ cd boxogems

2. Create a Gemfile

source :rubygems

gem 'geminabox'
gem 'unicorn'

group :development do
  gem 'rake'
  gem 'vlad'
  gem 'vlad-hg'
  gem 'vlad-unicorn'
end

3. Set up Vlad

# Rakefile
begin
  require 'vlad'
  Vlad.load(:scm => :mercurial, :type => nil,
    :app => :unicorn, :config => 'vlad.conf.rb')
rescue LoadError
end

This is just your standard Vlad.load call, but note that we’re passing :type => nil. The :type option defaults to :rails — but we don’t want to load any of the Rails-specific setup tasks. This way we bypass creating unnecessary symlinks from our release code into the shared/ directory.

Also note that I’m using a non-standard filename for the Vlad config–if you want to create a config/ directory in your project and use the normal config/deploy.rb, that’s fine. Setting up Gem in a Box requires so little code that I decided to avoid creating unnecessary subdirectories.

Now we create the deployment configuration. We’ll put the whole gem repository, both the app deployment we’re creating here and the hosted gems, in a subdirectory of /srv, with the app in /srv/rubygems/app and the hosted gems in /srv/rubygems/data.

# vlad.conf.rb (or config/deploy.rb)
set :application, 'boxogems'
set :domain, 'backend.example.com'
set :deploy_to, '/srv/rubygems/app'
set :repository, 'ssh://backend.example.com//path/to/boxogems'
set :unicorn_command, "cd #{current_path} && bundle exec unicorn"

4. Create the rackup file

Now that we’ve got the deployment harness in place, we can describe our app instance:

require 'geminabox'
Geminabox.data = '/srv/rubygems/data'
map '/gems' do
  run Geminabox
end

I didn’t feel the need to create a whole new virtual host when I’ve already got a perfectly good back-end web server (with an SSL certificate), so I’m using the Rack::URLMap middleware to serve the app on a URI prefix (subdirectory, path prefix, etc.) of ‘/gems’.

5. Commit and push

$ hg ci -m 'initial Gem in a Box setup'
$ hg clone . ssh://backend.example.com//path/to/boxogems

6. Deploy the app

Now you should be able to tell Vlad to deploy your app:

$ rake vlad:setup vlad:update

If all goes well, you can now SSH into your server and…

7. Configure the server

Create the Unicorn configuration file at /srv/rubygems/app/shared/config/unicorn.rb. The specifics are up to you, but here’s what mine looks like:

# shared/config/unicorn.rb
listen '127.0.0.1:4242'
pid '/srv/rubygems/app/shared/pids/unicorn.pid'
stdout_path '/srv/rubygems/app/shared/log/unicorn.stdout.log'
stderr_path '/srv/rubygems/app/shared/log/unicorn.stderr.log'

If you’re following the usual practice, you’ll probably bundle the gems in your deployed app. I don’t do it that way–I use Bundler to resolve dependencies, not to duplicate the same gems all over my server’s hard drive. So after I run rake vlad:update, I always:

server$ cd /srv/rubygems/app/current
server$ sudo bundle install --system --without development

Now you should be able to start the app using Vlad:

$ rake vlad:start_app

Check for output from Unicorn in /srv/rubygems/app/shared/log/unicorn.stderr.log. You should see a line that says something like:

I, [2012-03-22T11:32:55.918690 #10411]  INFO -- : worker=0 ready

That means the app is ready to serve requests; now to make Apache proxy requests to it. This is a bit different from the usual Apache config for a Rails app, so we use a <Location> block inside the virtual host config to do the proxying:

# Proxy anything under /gems to Gem in a Box
RewriteEngine On
RewriteRule ^/gems http://127.0.0.1:4242%{REQUEST_URI} [P,QSA,L]

<Location /gems>
  # Limit access to local network
  Order deny,allow
  Deny from all
  Allow from 127.0.0.1 172.16.0.0/20

  # Set up HTTP Basic authentication
  # Substitute your own authentication info here
  AuthName "Box o' Gems"
  AuthType Basic
  AuthUserFile /srv/rubygems/app/shared/config/htpasswd

  # Only allow authenticated users to upload gems
  <LimitExcept GET>
    Require valid-user
  </LimitExcept>
</Location>

Of course, you can force authentication even to download gems just by moving the Require valid-user line outside of the <LimitExcept> block (and removing the empty block). Adjust to your own needs.

At this point you’re probably thinking, “Hey, shouldn’t we be letting Apache serve the static files?” I’ll leave that as an exercise for the reader, but I’ll mention that I tried it, and then went back to letting Gem in a Box handle it directly. To make Apache do it I would’ve had to teach it about the MIME type of .gem files.

Now restart Apache and you should have a fully-functional RubyGems server!

Extras

If you want to be fancy, you can add a little magic into your Rakefile to make rake vlad:setup_app write a default Unicorn config into the right place for you. Put the default config into your source repository as unicorn.example.rb, then add this inside the begin ... rescue LoadError ... end block in your Rakefile:

namespace :vlad do
  task :setup_app do
    commands = ["umask #{umask}"]
    commands << "mkdir -p #{shared_path}/config"
    commands << "chown #{perm_owner} #{shared_path}/config" if perm_owner
    commands << "chgrp #{perm_group} #{shared_path}/config" if perm_group
    run commands.join(' && ')

    put(unicorn_config) { File.read('unicorn.example.rb') }
    run %(chmod 640 #{unicorn_config})
  end
end

You can also add a chunk of code that automatically installs the gems like I showed above:

namespace :vlad do
  task :update do
    run %(cd #{current_path} && sudo bundle install --system --without development)
  end
end