Posts tagged capistrano

Week 118

Dec 21st

Posted by James Stewart in Notes

In retrospect I probably shouldn’t have expected to hit the ground running when travelling with a one year old. Dealing with one’s own jetlag can be bad enough, but dealing with another person’s increases it all exponentially. I’m quite pleased, then, that I got as much done as I did last week. I’m particularly grateful to the proprietors of Sparrows and Madcap for their tolerance of my hours of wifi usage.

The main phase of the big wordpress/government consultation project for Digital Public is wrapped for now. There’ll be a little more work on that after Christmas once it’s been fully put through its paces, but for now it’s good to have that off my plate. Work on the Greenbelt management system is settling down to just tidying up a few loose ends. Drapno launched with a party I sadly had to miss. And I can finally stop and think a bit and do some much needed admin.

That’s why this afternoon involved many console windows. Over the years I’ve become responsible for a lot of servers and I’ve been working hard to reduce the amount of time it takes to manage them. I’ve got a loose assembly of parts that (munin for activity monitoring, god for process monitoring, pingdom for a few key sites, capistrano shell for applying updates, apticron to keep on top of package upgrades) that are working pretty well, but a good chunk of today was spent bringing a few configurations in line and upgrading some packages to make the system hang together better.

The other big activity at the moment is business planning. I’ve never really done much of that, but after a hectic year it seemed sensible to look back through the past months’ projects and accounts and get a better sense of where the work’s coming from, what areas I want to be focussing on, and how best to grow things in 2010. I’ve got some vague ideas but looking at the data may reveal some things I’ve missed.

So between that and some much needed cleaning up of side projects (it’d be good to get on top of all the wordpress plugins I’ve released this year, really put my Palm Pre through its paces, etc) this won’t really be a week off. But it’ll be enough of a change of pace that there won’t be a weekly update next week. Maybe the week after.

, , , , , , , , ,

Selected (belated, extended) Saturday Links

Mar 28th

Posted by James Stewart in Notes

The past two weeks haven’t really left time to compile my selected links, though there have been many. A few days at SxSWi (on which more, later) followed by travelling with the family and the inevitable work backlog moved blogging way down the priority list. So here’s a mammoth selection to get me caught up. Particularly interesting has been the discussion around the future of newspapers (represented here by Clay Shirky, Steven Johnson and Russell Davies), which seem to have finally pushed beyond “how t ind a good business model for papers” to looking at where the real value for society lies and how we can preserve and extend that in a changing landscape.

  • Making a jQuery Plugin Truly Customizable » Learning jQuery – Tips, Techniques, Tutorials

    Some nice tips for managing options, and a reminder to find _useful_ customisations not just load with customisation options without much thought about/consultation with other potential users

  • iPhone Coding For Web Developers

    Presentation slides from the internet's Matt Biddulph

  • Rack::Test released: Simply test any Rack-compatible app — Bryan Helmkamp

    There's a _lot_ to like about increased adoption of rack. "With Rack::Test, we hope to make it easy for frameworks to encourage their users to write tests by making it trivial to provide a testing environment. We’d like to foster compatibility between Ruby web app testing environments (especially important as ideas like multi-framework apps become more prominent). The philosophy is the library should stay small and extendable so frameworks can layer on additional functionality they want to offer without modifying Rack::Test’s core behavior or resorting to monkeypatching."

  • Newspapers and Thinking the Unthinkable « Clay Shirky

    "That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen …. Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify." … and a a lot more

  • russell davies: newspapers and all that

    "If we are going to create a new news ecosystem involving advertisers (and a lot of people would be grateful for that money) then we're going to have to do something about that institutional bifurcation between content and commerce. We're going to have to design the relationship between the two with the care of a good experience designer." – a response to Ben Hammersley asking if anyone talking about the future of newspapers had talked to anyone in advertising

  • Streams, affordances, Facebook, and rounding errors – Laughing Meme

    "Simon Willison asked this week about best practice for architecting activity streams. And the answer is, “It depends.” Depends on the scope, scale, access patterns, and affordances you’re building — your contract with your users.

    Which is a long way of saying think hard about the promises you make to your users, implicitly or explicitly.

    And, Facebook, my friend, what the HELL are you thinking? You managed to negotiate the best deal in the business, talk about a racket, and you threw it away for a piece of Twitter’s pain? Are you stupid? Well, best of luck with that."

  • SXSW Interactive Videos and Podcasts | SXSW.com

    Most of the sessions were recorded and this is the place to get hold of them.

  • SXSWi: Location-based service is the trend at Austin, Texas |

    "Predictably, location-based services were a major feature this year, with launches that included Foursquare, a social, location-based game by the Dodgeball creator, Dennis Crowley, and a new Facebook application for the location management tool Fire Eagle. While early adopters such as the SXSWers have been exploring location-based services for some time, it is inevitable that more consumer and privacy-friendly versions will start to creep into the mainstream."

  • stevenberlinjohnson.com: Old Growth Media And The Future Of News

    "I think it’s much more instructive to anticipate the future of investigative journalism by looking at the past of technology journalism. When ecologists go into the field to research natural ecosystems, they seek out the old-growth forests, the places where nature has had the longest amount of time to evolve and diversify and interconnect. They don’t study the Brazilian rain forest by looking at a field that was clear cut two years ago." … and …" Measured by pure audience interest, newspapers have never been more relevant. If they embrace this role as an authoritative guide to the entire ecosystem of news, if they stop paying for content that the web is already generating on its own, I suspect in the long run they will be as sustainable and as vital as they have ever been. The implied motto of every paper in the country should be: all the news that’s fit to link."

  • On running a panel

    A mixup over bus times meant I didn't make it to Andrew's panel at SxSW, but I heard many good things. It's really great to see this kind of debriefing-in-public going on. Hopefully it'll make for a stronger set of talks and panels next year.

  • Guardian API Maps – Home

    "This is a site that lets you search the Guardian's new API and add location information to articles. All the place data we collect is being made available to anyone who wants it."

  • Foursquare, Hot New Phone App, Is Dodgeball on Steroids | The New York Observer

    Quite a few people seemed to be playing with Foursquare at SxSW but most of the Brits were excluded as we didn't want to use that much data and it wasn't available in the UK iTunes store. One to watch, though.

  • A few notes on the Guardian Open Platform

    I saw Simon present the Guardian Platform at SxSW and it looks like a great achievement. Waiting to see what developers build on it, and how they roll some of the ideas back in

  • Taking remote imagery offline to Nigeria :: High Earth Orbit

    Andrew's notes on trying to source good map data for use in Nigeria. It's a useful overview of a variety of services and ways to use them, though highlighting the absence of really accessible, high-quality data.

  • Pulse Laser: The Utility of the Unfinished

    "One technique that S&W has been using recently to illustrate design work is placing sketches or wireframes in situ. Whilst wireframes themselves are incomplete artefacts, designed to be work in progress, they still suffer for being uniformly incomplete. Wireframes themselves can be almost too beautiful, and this means that it becomes all-too-easy to criticise them as only wireframes, rather than as part of a product that exists in the world. Contextualising the sketches into the photograph places the design into the world. This enables the design to be understood within the world, and also (importantly) to highlight the seams between the unfinished design and the finished world around it"

  • Spike: a log file viewing & (if we’re being generous) analysis tool for Rails developers.

    Looks like a handy addition to the toolkit

  • Generation Open | FactoryCity

    "Sharing and giving away all that you can are the best defenses against fear, obsolescence, growing old, and, even, wrinkles. It isn’t always easy, but it’s how we outlive the shackles of biology and transcend the physicality of gravity." – Perhaps an overly optimistic piece, but it connects together a number of current themes and we can hope…

  • Testing Facebook with Cucumber | opensoul.org

    For those faced with the unpleasant task of writing facebook apps, some people are working on making sure they can be thoroughly tested.

  • scraplab : instant sinatra deployment with heroku

    A lot of people seem to be excited about heroku lately, and it does look like a nice simple way to put up quick ruby apps. Must play soon.

  • How to speed up gem installs 10x « The Budding Rubyist

    Handy little tip, particularly for server environments: turn off ri and rdoc generation in your .gemrc file, and speed things up considerably

  • Facebook in 2010: no longer a walled garden – O'Reilly Radar

    A more positive spin on facebook's changes from David O'Recordon, who suspects they're going to pull down the walls around their garden and become a proper citizen of the open web.

  • Facebook blinks, copies Twitter, still gets it wrong. – broadstuff

    Critical commentary on facebook's recent changes. I'm not sure I entirely agree with statements like "By 2009 it was clear no one gives a sh*t about the Social Graph" but facebook really do seem to be finding that their approach is overly complex and quickly trying to shift to a more twitter-like "web of flow" (to steal Stowe Boyd's phrase)

  • Acquia Search goes public beta | Acquia

    Hosted solr for drupal: "Acquia Search can be installed as a module on any Drupal 6 site, and enhances a site's search experience with faceted search navigation, content recommendations, and configurable results weighting, all delivered through a redundant hosted service infrastructure.".

  • Oauth using pecl/OAuth

    Looks like a nice simple way to interact with oauth from a PHP app

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Selected Saturday Links

Mar 7th

Posted by James Stewart in Notes

Big themes this week have mostly revolved around twitter, facebook, and openness. Some have focussed on facebook redesigning to embrace a more twitter-like “web of flow” approach, and others on the fact that they’re jumping on various open web bandwagons. It’s been interesting to see some tie in with the government transparency thinking going around, as particularly noted by Chris Messina on FactoryCity. Meanwhile there are quite a few nice new tools emerging, and I really must try heroku one of these days.

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Deploying a Drupal Site with Capistrano 2

Jul 23rd

Posted by James Stewart in Notes

A little over a year ago I wrote up some instructions for deploying drupal sites using capistrano. It’s proved a popular entry, still getting a good bit of traffic, but in the time since I wrote it Capistrano 2 has joined us and my techniques have moved on, so it seemed high time I updated the instructions with some new ones.

As before, I’m going to presume that anyone reading this already has capistrano installed and has shell access to their server. If you need help with the former, I’d recommend stopping by the Capistrano website, and for the latter you should probably talk to your hosting company.

My approach is to keep each site’s assets (modules, themes, etc) within that site’s folder, and then store each site in version control (git or subversion). In common with capistrano-based rails deployments, I then have the site’s files and images stored in a shared folder and symlinked into the site, so that they can be preserved between deployments.

To get started go into the folder for the site you want to be able to deploy (eg. /path/to/drupal/sites/mysite.com) and type at the command line:

mkdir config
capify .

That will create the basic files for deployment, which you will then need to edit with your configuration details. I start by opening the file config/deploy.rb and deleting everything in it, so as to start with a blank slate. I then put in some overrides to capistrano’s default deployment methods so that they work better with drupal:

set :asset_folders, %W(images files)
 
namespace :deploy do
  desc "Link the asset folders from the shared folder into our site"
  task :finalize_update, :except => { :no_release => true } do
    logger.info 'finalizing update with custom method'
    run "chmod -R g+w #{latest_release}" if fetch(:group_writable, true)
 
    asset_folders.each do |asset|
      run "rm -rf #{release_path}/#{asset}"
      run "ln -nfs #{shared_path}/#{asset} #{release_path}/#{asset}"
    end
  end
 
  desc "Set up the expected application directory structure on all boxes"
  task :setup, :except => { :no_release => true } do
    run <<-CMD
      umask 02 &&
      mkdir -p #{deploy_to} #{releases_path} #{shared_path}
    CMD
    asset_folders.each do |asset|
      run "mkdir #{shared_path}/#{asset}"
    end
  end
 
  task :set_permissions, :except => { :no_release => true } do
    # do nothing
  end
 
  task :restart do
   # do nothing 
  end
end

You then need to set some basic configuration, so also in deploy.rb place:

set :deploy_to, '/path/to/your/drupal/sites'
 
# Make sure this is the domain name of your app as it is 
# what your sites folder will be named
set :site_name, 'staging.scodigo.com'
role :web, "your.server.com"
set :current_path, "#{deploy_to}/#{site_name}"
set :shared_path, "#{deploy_to}/shared/#{site_name}"
set :repository, "svn://your.svn.com/path/to/repos/trunk"

With that in place you’re all ready to go. But there was an extra step I decided to add this time around. Since I am frequently setting up new staging/test sites and often need to upgrade them to new versions of drupal, I wanted to be able to install and update the drupal files using capistrano.

To do that, I wrote a recipe that will grab the tarball for a given release, unpack it and install it, without wiping out an existing sites folder. To add that you will need to put the following code into your deploy.rb file:

namespace :drupal do
  desc <<-DESC
  Grab the specified version of drupal and install it
 
  This presumes that the variables drupal_version and drupal_path have been set
  DESC
 
  task :install do
    run "cd #{shared_path} && curl -O http://ftp.drupal.org/files/projects/drupal-#{drupal_version}.tar.gz"
    run "cd #{shared_path} && tar xzvf drupal-#{drupal_version}.tar.gz"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}/sites"
    run "cd #{drupal_path} && rm -rf !(sites)"
    run "mv #{shared_path}/drupal-#{drupal_version}/* #{drupal_path}"
    run "mv #{shared_path}/drupal-#{drupal_version}/.htaccess #{drupal_path}/.htaccess"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}"
  end
end
 
after "deploy:setup", "drupal:install"
set :drupal_path, File.join(deploy_to, '../')
set :drupal_version, "5.8" # Or whatever version you want to install

(You can download a copy of the combined deploy.rb file here or view it as a pastie here)

Now, to set up your drupal site you just need to change directory to your site and type and command line:

cap deploy:setup

and to deploy a new version:

cap deploy

In practice, I tend to use these techniques in combination with the Capistrano Multistage extension, so that I can deploy a site to a staging server first, and then a production server. I’d highly recommend that approach, but for now will leave it as an exercise for the reader.

Please note that this technique is also designed to support a multi-site drupal set up, but hasn’t been extensively tested in that context. It should work, but no guarantees!

, , , ,

Project Launch: New Greenbelt website

Apr 13th

Posted by James Stewart in Notes

One of the numerous projects I’ve been juggling over the past few months has been a redesign of the Greenbelt Festival website. That redesign went live late last night.

Working from Wilf‘s designs I initially built new HTML and CSS templates and began to establish some rules for how we’d handle the new image management requirements for a site that is now very photo-heavy. When it came time to apply the new designs to the CMS, however, it became apparent that there was a much bigger job ahead.

That CMS (a bundle of custom PHP that I had inherited) has grown over time and within some quite onerous server configuration constraints to a point where it was due a significant overhaul. Sticking with PHP was a fixed requirement as we’re relying on various APIs and a server architecture that wouldn’t be happy with me shifting to, say, rails, but already having the problem domain mapped out and the opportunity to radically simplify a few things meant I got to enjoy the feeling of stripping out a lot of code without impairing functionality.

One note that Derek Sivers made in his controversial blog entry last year about switching from Rails to PHP was that working with Rails had made him a better PHP developer. I’ve found a similar effect. I have no intention of leaving the world of Rails (I still prefer it by orders of magnitude), but tackling projects like this in PHP are a reminder that working for a while with really good tools is likely to encourage you to seek out best practice in whatever environment you end up in.

Ruby developers who occasionally work on PHP projects as I do may be interested to note that we are using capistrano for deployment, and I intend to use rspec for some testing. I’ll try to write that up once it’s in place.

With a refreshed CMS, new templates, and some standardisation of our many javascript files on top of a jquery foundation, Paul, Greenbelt’s was able to manage the photos and copy to turn that new look into a vibrant and content-rich new site. You can see a few notes he wrote about the redesign on the site.

There’s still a fair way to go. I’ve got a lot of tests to write in order to make it easier for us to make further changes, and various aspects of the site are more than ready for a more fundamental rethink that will let the festival open up its archives better, but all concerned are very pleased to present the fruit of our labours.

, , , , , , , , , ,
« Older Entries