A little over a year ago I wrote up some instructions for deploying drupal sites using capistrano. It’s proved a popular entry, still getting a good bit of traffic, but in the time since I wrote it Capistrano 2 has joined us and my techniques have moved on, so it seemed high time I updated the instructions with some new ones.

As before, I’m going to presume that anyone reading this already has capistrano installed and has shell access to their server. If you need help with the former, I’d recommend stopping by the Capistrano website, and for the latter you should probably talk to your hosting company.

My approach is to keep each site’s assets (modules, themes, etc) within that site’s folder, and then store each site in version control (git or subversion). In common with capistrano-based rails deployments, I then have the site’s files and images stored in a shared folder and symlinked into the site, so that they can be preserved between deployments.

To get started go into the folder for the site you want to be able to deploy (eg. /path/to/drupal/sites/mysite.com) and type at the command line:

mkdir config
capify .

That will create the basic files for deployment, which you will then need to edit with your configuration details. I start by opening the file config/deploy.rb and deleting everything in it, so as to start with a blank slate. I then put in some overrides to capistrano’s default deployment methods so that they work better with drupal:

set :asset_folders, %W(images files)
 
namespace :deploy do
  desc "Link the asset folders from the shared folder into our site"
  task :finalize_update, :except => { :no_release => true } do
    logger.info 'finalizing update with custom method'
    run "chmod -R g+w #{latest_release}" if fetch(:group_writable, true)
 
    asset_folders.each do |asset|
      run "rm -rf #{release_path}/#{asset}"
      run "ln -nfs #{shared_path}/#{asset} #{release_path}/#{asset}"
    end
  end
 
  desc "Set up the expected application directory structure on all boxes"
  task :setup, :except => { :no_release => true } do
    run <<-CMD
      umask 02 &&
      mkdir -p #{deploy_to} #{releases_path} #{shared_path}
    CMD
    asset_folders.each do |asset|
      run "mkdir #{shared_path}/#{asset}"
    end
  end
 
  task :set_permissions, :except => { :no_release => true } do
    # do nothing
  end
 
  task :restart do
   # do nothing 
  end
end

You then need to set some basic configuration, so also in deploy.rb place:

set :deploy_to, '/path/to/your/drupal/sites'
 
# Make sure this is the domain name of your app as it is 
# what your sites folder will be named
set :site_name, 'staging.scodigo.com'
role :web, "your.server.com"
set :current_path, "#{deploy_to}/#{site_name}"
set :shared_path, "#{deploy_to}/shared/#{site_name}"
set :repository, "svn://your.svn.com/path/to/repos/trunk"

With that in place you’re all ready to go. But there was an extra step I decided to add this time around. Since I am frequently setting up new staging/test sites and often need to upgrade them to new versions of drupal, I wanted to be able to install and update the drupal files using capistrano.

To do that, I wrote a recipe that will grab the tarball for a given release, unpack it and install it, without wiping out an existing sites folder. To add that you will need to put the following code into your deploy.rb file:

namespace :drupal do
  desc <<-DESC
  Grab the specified version of drupal and install it
 
  This presumes that the variables drupal_version and drupal_path have been set
  DESC
 
  task :install do
    run "cd #{shared_path} && curl -O http://ftp.drupal.org/files/projects/drupal-#{drupal_version}.tar.gz"
    run "cd #{shared_path} && tar xzvf drupal-#{drupal_version}.tar.gz"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}/sites"
    run "cd #{drupal_path} && rm -rf !(sites)"
    run "mv #{shared_path}/drupal-#{drupal_version}/* #{drupal_path}"
    run "mv #{shared_path}/drupal-#{drupal_version}/.htaccess #{drupal_path}/.htaccess"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}"
  end
end
 
after "deploy:setup", "drupal:install"
set :drupal_path, File.join(deploy_to, '../')
set :drupal_version, "5.8" # Or whatever version you want to install

(You can download a copy of the combined deploy.rb file here)

Now, to set up your drupal site you just need to change directory to your site and type and command line:

cap deploy:setup

and to deploy a new version:

cap deploy

In practice, I tend to use these techniques in combination with the Capistrano Multistage extension, so that I can deploy a site to a staging server first, and then a production server. I’d highly recommend that approach, but for now will leave it as an exercise for the reader.

Please note that this technique is also designed to support a multi-site drupal set up, but hasn’t been extensively tested in that context. It should work, but no guarantees!

Recommend this post:

Working with a number of non-profits I frequently find myself tasked with extending or upgrading drupal. Each new version of drupal has been a significant step forward and I’m usually keen to get up to date but there’s the small matter of the suite of modules most sites use that need to catch up with changing APIs. With the release of Drupal 6 a few weeks back I found myself wanting a tool that would help me check if my chosen modules were ready for the upgrade yet.

The quick solution I came up was a screen-scraper that will take a YAML file listing the relevant modules and their URLs and check to see if a Drupal 6 version has been released as yet. I wrote it in ruby, because after all that PHP work it’s nice to slip back into a language that feels so comfortable. (and hpricot is delightful for quick scraping solutions).

In case it’s helpful for anyone I’ve popped the ruby code and a sample yaml file up as pasties. Currently it’s invoked from the command line (providing you have ruby and hpricot set up) with:

./drupal-modules.rb my-file.yaml

A couple of nice enhancements would be to either grab the list of modules from an existing drupal database, or to set this up as a little web service where people input the modules they use and can check a web page or receive an email when there’s a status update. It could even lead into some discussion of where certain older modules should be considered redundant and/or replaced with newer options. Time’s not likely to let me build that, but I’d love to hear if someone else does.

Recommend this post:

In response to yesterday’s post about inlining CSS for HTML emails, I got a couple of comments suggesting alternatives to my CSS parser class. Not wanting to have to maintain code unless I have to, I decided to give them both a try and see how they worked out.

TamTam

First up is TamTam, suggested by batnight. I’d actually spotted TamTam and link blogged it a few weeks ago, which shows how transient attention can be. TamTam is a complete solution for inlining CSS, so I should be able to replace all my code with:

require 'rubygems'
require 'hpricot'
require 'tamtam'
 
inlined = TamTam.inline(
  :css => File.read('/path/to/my.css'),
  :body => File.read('/path/to/my.html')
)
 
puts inlined

That looks ideal, but unfortunately as soon as I tried passing my code into it an error emerged.

/Library/Ruby/Gems/gems/tamtam-0.0.2/lib/tamtam.rb:77:in `apply_to': Trouble on style td#email-header-message on element <td id="email-header-message"> (Exception): can't convert nil into String	from /Library/Ruby/Gems/gems/tamtam-0.0.2/lib/tamtam.rb:24:in `inline'
</td>

Digging into the library it seems that its parsing fails if there’s a CSS rule that doesn’t match any elements in the document. That’s not a problem if your CSS file is targetted to a specific block of HTML, but if the idea is to build a set of rules that can be applied across a selection of page/emails that may be a problem.

CSSPool

CSSPool, suggested by Dan Kubb, is a more generic CSS parser which is designed to work with hpricot to map between CSS and HTML. Initially I just used something very much like the examples they offer to get a sense of the format of the objects they give access to:

require 'rubygems'
require 'hpricot'
require 'csspool'
 
sac = CSS::SAC::Parser.new 
css_doc = sac.parse(File.read('/path/to/my.css')) 
html_doc = Hpricot.parse(File.read('/path/to/my.css')) 
css_doc.find_all_rules_matching(html_doc).each do |rule| 
  puts rule
end

Unfortunately this too failed on me with:

NoMethodError: undefined method `accept' for nil:NilClass

From a quick look at the source code it seemed that this error was occurring when a CSS selector didn’t match the supplied document, just as in TamTam. It turned out to be pretty easy to patch, though, and I’ve submitted a report in their tracker.

That done I was able to iterate over it and access the selectors easily enough, but what I’ve not yet been able to find is a way to get the parser to give me the actual CSS declarations appropriately formatted for including in the HTML. When I have a rule I can get the selector with:

rule.selector.to_css

But to find out what styles the selector applies isn’t so straightforward. If anyone has an easy way to do that, I’d love to hear about it in the comments!

Update (later that day): Version 0.2.3 of CSSPool is now out and includes my fix.

Recommend this post:

As anyone keeping an eye on my deli.cio.us feed may have noticed, quite a few links have appeared to information about the preparation of HTML email. It’s a nasty business, as a quick glance at the website of the email standards project will tell you. But sadly, nasty as it may be, sometimes it has to be done.

Even if the email I send out is going to have CSS scattered inline, for building the templates I’d much rather be able to focus on writing the structure of the document and leave worrying about my CSS for another time, and another file. That wouldn’t get me around the nastiness of having to use tables for anything but the simplest of layouts, but it still feels right to keep the separation for as long as possible.

I had a quick look for a tool that would take a stylesheet and an HTML document, and embed the rules online, but didn’t find one. So I turned to ruby. In theory it should be very easy to build something like this, because of hpricot’s support for CSS selectors. If we had the CSS stored in a hash all it would take would be something like:

require 'hpricot'
doc = Hpricot(open('my_page.html')
 
css_as_hash.each do |selector, rule|
  (doc/selector).set('style', rule)
end
 
puts doc

Obviously that wouldn’t play nicely if there were already any styles inline, but for the purposes of this project I assumed there wouldn’t be.

I had a quick look at the cssparser rubygem but found that the sample code threw ‘method not found’ errors so I decided to quickly roll my own class that would take a path to a CSS file, and convert it to a hash. All it took was a few minutes’ work and the result was:

# This class takes a CSS file and provides a method to
# parse it into a hash. Usage is:
# 
# parser = SimpleCSSParser.new('/path/to/myfile.css')
# hash_of_rules = parser.to_hash
#
# For more advanced CSS handling check out the cssparser gem
# http://code.dunae.ca/css_parser/
class SimpleCSSParser
 
  # Receive and open the CSS file, storing its contents
  def initialize(path_to_file)
    @css = open(path_to_file).read
  end
 
  # Convert the CSS into a hash, where the keys are the selectors
  # and the values are the rules
  def to_hash
    @to_hash ||= separate_rules.inject({}) do |collection, rule|
      identifiers, rule = prepare_selectors_and_rule(rule)
      identifiers.each do |identifier|
        collection[identifier] ||= ''
        collection[identifier] += rule
      end
      collection
    end
  end
 
  private
    def separate_rules
      @css.split('}')
    end
 
    # Strip comments and extraneous white space from our CSS rules
    def clean_up_rule(css_rule)
      css_rule = css_rule.gsub(/\/\*.+?\*\//, '')
      css_rule.gsub(/\n|\s{2,}/, '')
    end
 
    # Break apart our selector(s) and rule. We return an array
    # of selectors to allow for situations where multiple selectors
    # are specified (comma separated) for a single rule
    def prepare_selectors_and_rule(rule)
      parts = rule.split('{')
      selectors = parts[0].split(',').map(&:strip)
      return selectors, clean_up_rule(parts[1])
    end
end

With that in place, I can now call:

require 'hpricot'
 
doc = Hpricot(open('my_file.html'))
parser = SimpleCSSParser.new('my_file.css')
 
parser.to_hash.each do |selector, rule|
  (doc/selector).set('style', rule)
end
 
puts doc

and have the result I wanted all along. It’s rather brittle because of the way it splits the rules up, and it won’t pull in @include’d files, handle multiple CSS files, or do anything to honour the proper inheritance rules, but for my purposes that’s okay. I bundled it all up in a file that can be called from the command line. You can find that in this pastie.

A nice (and really quite simple) addition would be to take Campaign Monitor’s Guide to CSS Support in Email, parse it and spit out warnings about which email clients will have issues with which CSS rules. If I get round to implementing that I’ll blog about it here. If you get there before me, do post a comment and let me know.

UPDATE (5th Dec ‘07: I’ve posted a follow-up looking at some other Ruby CSS parsers.

Recommend this post:

One of the many tricky decisions to be made when building content management tools is how to allow users to control the basic formatting of their input without breaking your carefully crafted layouts or injecting nasty hacks into your pages. One approach has long been to provide your own markup language. Instead of allowing users to write HTML, let them use bbcode, or markdown, or textile, which have more controlled vocabularies and rules that mean it’s much less likely that problems will occur.

Textile in particular has a nice simple syntax and is increasingly popular thanks to its adoption in products like those of 37signals. In Ruby, there’s the RedCloth library which makes it fast and easy to convert textile to HTML. The one problem is if you already have a body of user generated HTML in your legacy system that needs converting. That’s the situation I found myself in this week and I quickly needed a tool to translate the content so that I could get on with the more interesting parts of the system.

Searching for options, the ClothRed library which offers some translation, but it doesn’t handle important elements like links. I considered patching it to handle the elements I need, but in the end I decided to take a different approach and used the SGML parsing library found here to port a python html2textile parser.

Porting code from python to ruby is a pretty straightforward process as the language’s are so similar on a number of levels, but there were several issues to work through, particularly relating to scoping, and quite a few methods to change to make them feel a little more ruby-ish. I’ve not converted all of the entity handling as I didn’t really need it, but there might be a bit of work to do in making sure character set issues are properly taken care of.

The end result is a piece of code that’s now served its purpose and that I’m unlikely to need again for quite a while. It’s not something that I’m particularly proud of, it could almost certainly be implemented more neatly, but I thought I’d throw it out there in case it could be useful to someone else. Should you be inspired to take it and twist it and turn it into a well-heeled, more robust and properly distributable solution, feel free, but please let me know so that at the very least I can update this entry.

Grab the code here or view it here.

Recommend this post: