Posts tagged deployment

Selected (belated, extended) Saturday Links

Mar 28th

Posted by James Stewart in Notes

The past two weeks haven’t really left time to compile my selected links, though there have been many. A few days at SxSWi (on which more, later) followed by travelling with the family and the inevitable work backlog moved blogging way down the priority list. So here’s a mammoth selection to get me caught up. Particularly interesting has been the discussion around the future of newspapers (represented here by Clay Shirky, Steven Johnson and Russell Davies), which seem to have finally pushed beyond “how t ind a good business model for papers” to looking at where the real value for society lies and how we can preserve and extend that in a changing landscape.

  • Making a jQuery Plugin Truly Customizable » Learning jQuery – Tips, Techniques, Tutorials

    Some nice tips for managing options, and a reminder to find _useful_ customisations not just load with customisation options without much thought about/consultation with other potential users

  • iPhone Coding For Web Developers

    Presentation slides from the internet's Matt Biddulph

  • Rack::Test released: Simply test any Rack-compatible app — Bryan Helmkamp

    There's a _lot_ to like about increased adoption of rack. "With Rack::Test, we hope to make it easy for frameworks to encourage their users to write tests by making it trivial to provide a testing environment. We’d like to foster compatibility between Ruby web app testing environments (especially important as ideas like multi-framework apps become more prominent). The philosophy is the library should stay small and extendable so frameworks can layer on additional functionality they want to offer without modifying Rack::Test’s core behavior or resorting to monkeypatching."

  • Newspapers and Thinking the Unthinkable « Clay Shirky

    "That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen …. Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify." … and a a lot more

  • russell davies: newspapers and all that

    "If we are going to create a new news ecosystem involving advertisers (and a lot of people would be grateful for that money) then we're going to have to do something about that institutional bifurcation between content and commerce. We're going to have to design the relationship between the two with the care of a good experience designer." – a response to Ben Hammersley asking if anyone talking about the future of newspapers had talked to anyone in advertising

  • Streams, affordances, Facebook, and rounding errors – Laughing Meme

    "Simon Willison asked this week about best practice for architecting activity streams. And the answer is, “It depends.” Depends on the scope, scale, access patterns, and affordances you’re building — your contract with your users.

    Which is a long way of saying think hard about the promises you make to your users, implicitly or explicitly.

    And, Facebook, my friend, what the HELL are you thinking? You managed to negotiate the best deal in the business, talk about a racket, and you threw it away for a piece of Twitter’s pain? Are you stupid? Well, best of luck with that."

  • SXSW Interactive Videos and Podcasts | SXSW.com

    Most of the sessions were recorded and this is the place to get hold of them.

  • SXSWi: Location-based service is the trend at Austin, Texas |

    "Predictably, location-based services were a major feature this year, with launches that included Foursquare, a social, location-based game by the Dodgeball creator, Dennis Crowley, and a new Facebook application for the location management tool Fire Eagle. While early adopters such as the SXSWers have been exploring location-based services for some time, it is inevitable that more consumer and privacy-friendly versions will start to creep into the mainstream."

  • stevenberlinjohnson.com: Old Growth Media And The Future Of News

    "I think it’s much more instructive to anticipate the future of investigative journalism by looking at the past of technology journalism. When ecologists go into the field to research natural ecosystems, they seek out the old-growth forests, the places where nature has had the longest amount of time to evolve and diversify and interconnect. They don’t study the Brazilian rain forest by looking at a field that was clear cut two years ago." … and …" Measured by pure audience interest, newspapers have never been more relevant. If they embrace this role as an authoritative guide to the entire ecosystem of news, if they stop paying for content that the web is already generating on its own, I suspect in the long run they will be as sustainable and as vital as they have ever been. The implied motto of every paper in the country should be: all the news that’s fit to link."

  • On running a panel

    A mixup over bus times meant I didn't make it to Andrew's panel at SxSW, but I heard many good things. It's really great to see this kind of debriefing-in-public going on. Hopefully it'll make for a stronger set of talks and panels next year.

  • Guardian API Maps – Home

    "This is a site that lets you search the Guardian's new API and add location information to articles. All the place data we collect is being made available to anyone who wants it."

  • Foursquare, Hot New Phone App, Is Dodgeball on Steroids | The New York Observer

    Quite a few people seemed to be playing with Foursquare at SxSW but most of the Brits were excluded as we didn't want to use that much data and it wasn't available in the UK iTunes store. One to watch, though.

  • A few notes on the Guardian Open Platform

    I saw Simon present the Guardian Platform at SxSW and it looks like a great achievement. Waiting to see what developers build on it, and how they roll some of the ideas back in

  • Taking remote imagery offline to Nigeria :: High Earth Orbit

    Andrew's notes on trying to source good map data for use in Nigeria. It's a useful overview of a variety of services and ways to use them, though highlighting the absence of really accessible, high-quality data.

  • Pulse Laser: The Utility of the Unfinished

    "One technique that S&W has been using recently to illustrate design work is placing sketches or wireframes in situ. Whilst wireframes themselves are incomplete artefacts, designed to be work in progress, they still suffer for being uniformly incomplete. Wireframes themselves can be almost too beautiful, and this means that it becomes all-too-easy to criticise them as only wireframes, rather than as part of a product that exists in the world. Contextualising the sketches into the photograph places the design into the world. This enables the design to be understood within the world, and also (importantly) to highlight the seams between the unfinished design and the finished world around it"

  • Spike: a log file viewing & (if we’re being generous) analysis tool for Rails developers.

    Looks like a handy addition to the toolkit

  • Generation Open | FactoryCity

    "Sharing and giving away all that you can are the best defenses against fear, obsolescence, growing old, and, even, wrinkles. It isn’t always easy, but it’s how we outlive the shackles of biology and transcend the physicality of gravity." – Perhaps an overly optimistic piece, but it connects together a number of current themes and we can hope…

  • Testing Facebook with Cucumber | opensoul.org

    For those faced with the unpleasant task of writing facebook apps, some people are working on making sure they can be thoroughly tested.

  • scraplab : instant sinatra deployment with heroku

    A lot of people seem to be excited about heroku lately, and it does look like a nice simple way to put up quick ruby apps. Must play soon.

  • How to speed up gem installs 10x « The Budding Rubyist

    Handy little tip, particularly for server environments: turn off ri and rdoc generation in your .gemrc file, and speed things up considerably

  • Facebook in 2010: no longer a walled garden – O'Reilly Radar

    A more positive spin on facebook's changes from David O'Recordon, who suspects they're going to pull down the walls around their garden and become a proper citizen of the open web.

  • Facebook blinks, copies Twitter, still gets it wrong. – broadstuff

    Critical commentary on facebook's recent changes. I'm not sure I entirely agree with statements like "By 2009 it was clear no one gives a sh*t about the Social Graph" but facebook really do seem to be finding that their approach is overly complex and quickly trying to shift to a more twitter-like "web of flow" (to steal Stowe Boyd's phrase)

  • Acquia Search goes public beta | Acquia

    Hosted solr for drupal: "Acquia Search can be installed as a module on any Drupal 6 site, and enhances a site's search experience with faceted search navigation, content recommendations, and configurable results weighting, all delivered through a redundant hosted service infrastructure.".

  • Oauth using pecl/OAuth

    Looks like a nice simple way to interact with oauth from a PHP app

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Selected Saturday Links

Mar 7th

Posted by James Stewart in Notes

Big themes this week have mostly revolved around twitter, facebook, and openness. Some have focussed on facebook redesigning to embrace a more twitter-like “web of flow” approach, and others on the fact that they’re jumping on various open web bandwagons. It’s been interesting to see some tie in with the government transparency thinking going around, as particularly noted by Chris Messina on FactoryCity. Meanwhile there are quite a few nice new tools emerging, and I really must try heroku one of these days.

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Deploying a Drupal Site with Capistrano 2

Jul 23rd

Posted by James Stewart in Notes

A little over a year ago I wrote up some instructions for deploying drupal sites using capistrano. It’s proved a popular entry, still getting a good bit of traffic, but in the time since I wrote it Capistrano 2 has joined us and my techniques have moved on, so it seemed high time I updated the instructions with some new ones.

As before, I’m going to presume that anyone reading this already has capistrano installed and has shell access to their server. If you need help with the former, I’d recommend stopping by the Capistrano website, and for the latter you should probably talk to your hosting company.

My approach is to keep each site’s assets (modules, themes, etc) within that site’s folder, and then store each site in version control (git or subversion). In common with capistrano-based rails deployments, I then have the site’s files and images stored in a shared folder and symlinked into the site, so that they can be preserved between deployments.

To get started go into the folder for the site you want to be able to deploy (eg. /path/to/drupal/sites/mysite.com) and type at the command line:

mkdir config
capify .

That will create the basic files for deployment, which you will then need to edit with your configuration details. I start by opening the file config/deploy.rb and deleting everything in it, so as to start with a blank slate. I then put in some overrides to capistrano’s default deployment methods so that they work better with drupal:

set :asset_folders, %W(images files)
 
namespace :deploy do
  desc "Link the asset folders from the shared folder into our site"
  task :finalize_update, :except => { :no_release => true } do
    logger.info 'finalizing update with custom method'
    run "chmod -R g+w #{latest_release}" if fetch(:group_writable, true)
 
    asset_folders.each do |asset|
      run "rm -rf #{release_path}/#{asset}"
      run "ln -nfs #{shared_path}/#{asset} #{release_path}/#{asset}"
    end
  end
 
  desc "Set up the expected application directory structure on all boxes"
  task :setup, :except => { :no_release => true } do
    run <<-CMD
      umask 02 &&
      mkdir -p #{deploy_to} #{releases_path} #{shared_path}
    CMD
    asset_folders.each do |asset|
      run "mkdir #{shared_path}/#{asset}"
    end
  end
 
  task :set_permissions, :except => { :no_release => true } do
    # do nothing
  end
 
  task :restart do
   # do nothing 
  end
end

You then need to set some basic configuration, so also in deploy.rb place:

set :deploy_to, '/path/to/your/drupal/sites'
 
# Make sure this is the domain name of your app as it is 
# what your sites folder will be named
set :site_name, 'staging.scodigo.com'
role :web, "your.server.com"
set :current_path, "#{deploy_to}/#{site_name}"
set :shared_path, "#{deploy_to}/shared/#{site_name}"
set :repository, "svn://your.svn.com/path/to/repos/trunk"

With that in place you’re all ready to go. But there was an extra step I decided to add this time around. Since I am frequently setting up new staging/test sites and often need to upgrade them to new versions of drupal, I wanted to be able to install and update the drupal files using capistrano.

To do that, I wrote a recipe that will grab the tarball for a given release, unpack it and install it, without wiping out an existing sites folder. To add that you will need to put the following code into your deploy.rb file:

namespace :drupal do
  desc <<-DESC
  Grab the specified version of drupal and install it
 
  This presumes that the variables drupal_version and drupal_path have been set
  DESC
 
  task :install do
    run "cd #{shared_path} && curl -O http://ftp.drupal.org/files/projects/drupal-#{drupal_version}.tar.gz"
    run "cd #{shared_path} && tar xzvf drupal-#{drupal_version}.tar.gz"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}/sites"
    run "cd #{drupal_path} && rm -rf !(sites)"
    run "mv #{shared_path}/drupal-#{drupal_version}/* #{drupal_path}"
    run "mv #{shared_path}/drupal-#{drupal_version}/.htaccess #{drupal_path}/.htaccess"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}"
  end
end
 
after "deploy:setup", "drupal:install"
set :drupal_path, File.join(deploy_to, '../')
set :drupal_version, "5.8" # Or whatever version you want to install

(You can download a copy of the combined deploy.rb file here or view it as a pastie here)

Now, to set up your drupal site you just need to change directory to your site and type and command line:

cap deploy:setup

and to deploy a new version:

cap deploy

In practice, I tend to use these techniques in combination with the Capistrano Multistage extension, so that I can deploy a site to a staging server first, and then a production server. I’d highly recommend that approach, but for now will leave it as an exercise for the reader.

Please note that this technique is also designed to support a multi-site drupal set up, but hasn’t been extensively tested in that context. It should work, but no guarantees!

, , , ,

Book Review: Pro Drupal Development

Nov 20th

Posted by James Stewart in Book Reviews

It’s surprising given drupal’s popularity that there aren’t more books covering it in detail. Site launches and contributions by the likes of lullabot and bryght have pushed the CMS’ profile and recent releases have emphasised the Web 2.0 potential, but a quick look at amazon reveals only four related titles. Of the four, Pro Drupal Development is definitely the most developer focussed.

This isn’t a book for a drupal newbie. Going in you’ll want to have spent at least a little time setting up a drupal site or two, and while there’s no need to be a PHP guru the authors do presume you’re not going to need help understanding their code samples. They focus on drupal’s internals, with a lot of time spent writing modules, understanding the user, node, menu, theme and related systems, and a little attention for performance optimisation.

There’s a lot of ground to cover and most chapters are short, giving just the essentials on each area. You’ll probably want to pause from time to time to try out the code samples unless you’re already experienced at writing drupal modules. Having written a number of modules and run into various problems I found I was able to focus on the new information and how it would have affected my approach, but if this is new ground the structure of the book may make it rather overwhelming.

The writers are keen to encourage their readers to read the book in order, and some chapters certainly do build on their predecessors, but the real strength of this is likely to be as a reference guide. A quick once-through will help newcomers to module development get a sense of how everything fits together, but chances are you’ll then want to refer back when you actually encounter problems that a given chapter can help with.

I was disappointed not to see more coverage of testing drupal code. As I mentioned yesterday, the lack of attention paid to automated testing in the drupal community frustrates me and it seems that for a book like this to not to provide some coverage of sensible testing regimens is a missed opportunity. Similarly, it would be really good to have some coverage of deployment tips, particularly relating to upgrading active sites. Both of those seem to me like core topics for any book purporting to provide a guide for professionals, but the priorities of the book mirror those of the drupal community where neither topic appears to be a significant concern.

For any experienced developer who needs to get to grips with the insides of drupal and/or write custom extensions, this book will be invaluable. The style won’t suit those looking for a broader scope or lengthier tutorials, but it will help you get to grips with each of the major components quickly and provide enough information to set you on your way. Hopefully it will also trigger further writing about drupal, which may cover more ground and help developers bring some other vital practices to their drupal work.

Disclaimer: I was sent a copy of this book for review by the publisher. You can find it at apress, amazon US, amazon UK and all sorts of other places.

, , , , , ,

Protecting static files when deploying with Capistrano

Oct 3rd

Posted by James Stewart in Snippets

There’s one project I work on where the client wants to be able to edit HTML files in the root folder, but I want to be able to deploy with capistrano. It’s a pain to have to log into the server and check whether those files have changed before each deployment so last night, I added the following to my deploy.rb:

desc "Make sure we don't overwrite manual static file changes"
task :before_deploy do
  captured = false
 
  run "svn status #{deploy_to}/current/public" do |ch, stream, data|
    if stream == :out and data.chomp.match(/^M/)
      captured = true 
    end
  end
 
  if captured
    run "svn commit #{deploy_to}/current/public -m 'Storing manual changes to static HTML before deploy'"
  end
end

It’s not the most elegant, but it’ll make sure that any files the client has changed (which were already in the repository) are committed, and it makes the deployment process just that little bit easier.

(thanks to Jonathan Weiss for a recent blog entry that reminded me to implement this.)

, ,
« Older Entries