Posts tagged sysadmin

Selected (belated, extended) Saturday Links

Mar 28th

Posted by James Stewart in Notes

The past two weeks haven’t really left time to compile my selected links, though there have been many. A few days at SxSWi (on which more, later) followed by travelling with the family and the inevitable work backlog moved blogging way down the priority list. So here’s a mammoth selection to get me caught up. Particularly interesting has been the discussion around the future of newspapers (represented here by Clay Shirky, Steven Johnson and Russell Davies), which seem to have finally pushed beyond “how t ind a good business model for papers” to looking at where the real value for society lies and how we can preserve and extend that in a changing landscape.

  • Making a jQuery Plugin Truly Customizable » Learning jQuery – Tips, Techniques, Tutorials

    Some nice tips for managing options, and a reminder to find _useful_ customisations not just load with customisation options without much thought about/consultation with other potential users

  • iPhone Coding For Web Developers

    Presentation slides from the internet's Matt Biddulph

  • Rack::Test released: Simply test any Rack-compatible app — Bryan Helmkamp

    There's a _lot_ to like about increased adoption of rack. "With Rack::Test, we hope to make it easy for frameworks to encourage their users to write tests by making it trivial to provide a testing environment. We’d like to foster compatibility between Ruby web app testing environments (especially important as ideas like multi-framework apps become more prominent). The philosophy is the library should stay small and extendable so frameworks can layer on additional functionality they want to offer without modifying Rack::Test’s core behavior or resorting to monkeypatching."

  • Newspapers and Thinking the Unthinkable « Clay Shirky

    "That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen …. Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify." … and a a lot more

  • russell davies: newspapers and all that

    "If we are going to create a new news ecosystem involving advertisers (and a lot of people would be grateful for that money) then we're going to have to do something about that institutional bifurcation between content and commerce. We're going to have to design the relationship between the two with the care of a good experience designer." – a response to Ben Hammersley asking if anyone talking about the future of newspapers had talked to anyone in advertising

  • Streams, affordances, Facebook, and rounding errors – Laughing Meme

    "Simon Willison asked this week about best practice for architecting activity streams. And the answer is, “It depends.” Depends on the scope, scale, access patterns, and affordances you’re building — your contract with your users.

    Which is a long way of saying think hard about the promises you make to your users, implicitly or explicitly.

    And, Facebook, my friend, what the HELL are you thinking? You managed to negotiate the best deal in the business, talk about a racket, and you threw it away for a piece of Twitter’s pain? Are you stupid? Well, best of luck with that."

  • SXSW Interactive Videos and Podcasts | SXSW.com

    Most of the sessions were recorded and this is the place to get hold of them.

  • SXSWi: Location-based service is the trend at Austin, Texas |

    "Predictably, location-based services were a major feature this year, with launches that included Foursquare, a social, location-based game by the Dodgeball creator, Dennis Crowley, and a new Facebook application for the location management tool Fire Eagle. While early adopters such as the SXSWers have been exploring location-based services for some time, it is inevitable that more consumer and privacy-friendly versions will start to creep into the mainstream."

  • stevenberlinjohnson.com: Old Growth Media And The Future Of News

    "I think it’s much more instructive to anticipate the future of investigative journalism by looking at the past of technology journalism. When ecologists go into the field to research natural ecosystems, they seek out the old-growth forests, the places where nature has had the longest amount of time to evolve and diversify and interconnect. They don’t study the Brazilian rain forest by looking at a field that was clear cut two years ago." … and …" Measured by pure audience interest, newspapers have never been more relevant. If they embrace this role as an authoritative guide to the entire ecosystem of news, if they stop paying for content that the web is already generating on its own, I suspect in the long run they will be as sustainable and as vital as they have ever been. The implied motto of every paper in the country should be: all the news that’s fit to link."

  • On running a panel

    A mixup over bus times meant I didn't make it to Andrew's panel at SxSW, but I heard many good things. It's really great to see this kind of debriefing-in-public going on. Hopefully it'll make for a stronger set of talks and panels next year.

  • Guardian API Maps – Home

    "This is a site that lets you search the Guardian's new API and add location information to articles. All the place data we collect is being made available to anyone who wants it."

  • Foursquare, Hot New Phone App, Is Dodgeball on Steroids | The New York Observer

    Quite a few people seemed to be playing with Foursquare at SxSW but most of the Brits were excluded as we didn't want to use that much data and it wasn't available in the UK iTunes store. One to watch, though.

  • A few notes on the Guardian Open Platform

    I saw Simon present the Guardian Platform at SxSW and it looks like a great achievement. Waiting to see what developers build on it, and how they roll some of the ideas back in

  • Taking remote imagery offline to Nigeria :: High Earth Orbit

    Andrew's notes on trying to source good map data for use in Nigeria. It's a useful overview of a variety of services and ways to use them, though highlighting the absence of really accessible, high-quality data.

  • Pulse Laser: The Utility of the Unfinished

    "One technique that S&W has been using recently to illustrate design work is placing sketches or wireframes in situ. Whilst wireframes themselves are incomplete artefacts, designed to be work in progress, they still suffer for being uniformly incomplete. Wireframes themselves can be almost too beautiful, and this means that it becomes all-too-easy to criticise them as only wireframes, rather than as part of a product that exists in the world. Contextualising the sketches into the photograph places the design into the world. This enables the design to be understood within the world, and also (importantly) to highlight the seams between the unfinished design and the finished world around it"

  • Spike: a log file viewing & (if we’re being generous) analysis tool for Rails developers.

    Looks like a handy addition to the toolkit

  • Generation Open | FactoryCity

    "Sharing and giving away all that you can are the best defenses against fear, obsolescence, growing old, and, even, wrinkles. It isn’t always easy, but it’s how we outlive the shackles of biology and transcend the physicality of gravity." – Perhaps an overly optimistic piece, but it connects together a number of current themes and we can hope…

  • Testing Facebook with Cucumber | opensoul.org

    For those faced with the unpleasant task of writing facebook apps, some people are working on making sure they can be thoroughly tested.

  • scraplab : instant sinatra deployment with heroku

    A lot of people seem to be excited about heroku lately, and it does look like a nice simple way to put up quick ruby apps. Must play soon.

  • How to speed up gem installs 10x « The Budding Rubyist

    Handy little tip, particularly for server environments: turn off ri and rdoc generation in your .gemrc file, and speed things up considerably

  • Facebook in 2010: no longer a walled garden – O'Reilly Radar

    A more positive spin on facebook's changes from David O'Recordon, who suspects they're going to pull down the walls around their garden and become a proper citizen of the open web.

  • Facebook blinks, copies Twitter, still gets it wrong. – broadstuff

    Critical commentary on facebook's recent changes. I'm not sure I entirely agree with statements like "By 2009 it was clear no one gives a sh*t about the Social Graph" but facebook really do seem to be finding that their approach is overly complex and quickly trying to shift to a more twitter-like "web of flow" (to steal Stowe Boyd's phrase)

  • Acquia Search goes public beta | Acquia

    Hosted solr for drupal: "Acquia Search can be installed as a module on any Drupal 6 site, and enhances a site's search experience with faceted search navigation, content recommendations, and configurable results weighting, all delivered through a redundant hosted service infrastructure.".

  • Oauth using pecl/OAuth

    Looks like a nice simple way to interact with oauth from a PHP app

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Selected Saturday Links

Mar 7th

Posted by James Stewart in Notes

Big themes this week have mostly revolved around twitter, facebook, and openness. Some have focussed on facebook redesigning to embrace a more twitter-like “web of flow” approach, and others on the fact that they’re jumping on various open web bandwagons. It’s been interesting to see some tie in with the government transparency thinking going around, as particularly noted by Chris Messina on FactoryCity. Meanwhile there are quite a few nice new tools emerging, and I really must try heroku one of these days.

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Deploying a Drupal Site with Capistrano 2

Jul 23rd

Posted by James Stewart in Notes

A little over a year ago I wrote up some instructions for deploying drupal sites using capistrano. It’s proved a popular entry, still getting a good bit of traffic, but in the time since I wrote it Capistrano 2 has joined us and my techniques have moved on, so it seemed high time I updated the instructions with some new ones.

As before, I’m going to presume that anyone reading this already has capistrano installed and has shell access to their server. If you need help with the former, I’d recommend stopping by the Capistrano website, and for the latter you should probably talk to your hosting company.

My approach is to keep each site’s assets (modules, themes, etc) within that site’s folder, and then store each site in version control (git or subversion). In common with capistrano-based rails deployments, I then have the site’s files and images stored in a shared folder and symlinked into the site, so that they can be preserved between deployments.

To get started go into the folder for the site you want to be able to deploy (eg. /path/to/drupal/sites/mysite.com) and type at the command line:

mkdir config
capify .

That will create the basic files for deployment, which you will then need to edit with your configuration details. I start by opening the file config/deploy.rb and deleting everything in it, so as to start with a blank slate. I then put in some overrides to capistrano’s default deployment methods so that they work better with drupal:

set :asset_folders, %W(images files)
 
namespace :deploy do
  desc "Link the asset folders from the shared folder into our site"
  task :finalize_update, :except => { :no_release => true } do
    logger.info 'finalizing update with custom method'
    run "chmod -R g+w #{latest_release}" if fetch(:group_writable, true)
 
    asset_folders.each do |asset|
      run "rm -rf #{release_path}/#{asset}"
      run "ln -nfs #{shared_path}/#{asset} #{release_path}/#{asset}"
    end
  end
 
  desc "Set up the expected application directory structure on all boxes"
  task :setup, :except => { :no_release => true } do
    run <<-CMD
      umask 02 &&
      mkdir -p #{deploy_to} #{releases_path} #{shared_path}
    CMD
    asset_folders.each do |asset|
      run "mkdir #{shared_path}/#{asset}"
    end
  end
 
  task :set_permissions, :except => { :no_release => true } do
    # do nothing
  end
 
  task :restart do
   # do nothing 
  end
end

You then need to set some basic configuration, so also in deploy.rb place:

set :deploy_to, '/path/to/your/drupal/sites'
 
# Make sure this is the domain name of your app as it is 
# what your sites folder will be named
set :site_name, 'staging.scodigo.com'
role :web, "your.server.com"
set :current_path, "#{deploy_to}/#{site_name}"
set :shared_path, "#{deploy_to}/shared/#{site_name}"
set :repository, "svn://your.svn.com/path/to/repos/trunk"

With that in place you’re all ready to go. But there was an extra step I decided to add this time around. Since I am frequently setting up new staging/test sites and often need to upgrade them to new versions of drupal, I wanted to be able to install and update the drupal files using capistrano.

To do that, I wrote a recipe that will grab the tarball for a given release, unpack it and install it, without wiping out an existing sites folder. To add that you will need to put the following code into your deploy.rb file:

namespace :drupal do
  desc <<-DESC
  Grab the specified version of drupal and install it
 
  This presumes that the variables drupal_version and drupal_path have been set
  DESC
 
  task :install do
    run "cd #{shared_path} && curl -O http://ftp.drupal.org/files/projects/drupal-#{drupal_version}.tar.gz"
    run "cd #{shared_path} && tar xzvf drupal-#{drupal_version}.tar.gz"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}/sites"
    run "cd #{drupal_path} && rm -rf !(sites)"
    run "mv #{shared_path}/drupal-#{drupal_version}/* #{drupal_path}"
    run "mv #{shared_path}/drupal-#{drupal_version}/.htaccess #{drupal_path}/.htaccess"
    run "rm -rf #{shared_path}/drupal-#{drupal_version}"
  end
end
 
after "deploy:setup", "drupal:install"
set :drupal_path, File.join(deploy_to, '../')
set :drupal_version, "5.8" # Or whatever version you want to install

(You can download a copy of the combined deploy.rb file here or view it as a pastie here)

Now, to set up your drupal site you just need to change directory to your site and type and command line:

cap deploy:setup

and to deploy a new version:

cap deploy

In practice, I tend to use these techniques in combination with the Capistrano Multistage extension, so that I can deploy a site to a staging server first, and then a production server. I’d highly recommend that approach, but for now will leave it as an exercise for the reader.

Please note that this technique is also designed to support a multi-site drupal set up, but hasn’t been extensively tested in that context. It should work, but no guarantees!

, , , ,

Book Review: Practical Ruby for System Administration

Dec 24th

Posted by James Stewart in Book Reviews

book sleeve for Practical Ruby for System AdministrationIf you’ve encountered Ruby primarily through Rails and know it chiefly as an elegant tool for writing web applications it’s easy to miss its longer history as a tool for systems administration. Before Rails made Ruby the language-du-jour sysadmins bore much of the responsibility for keeping it alive, with the result that it has a suite of libraries helpful for server monitoring and a range of other administrative tasks.

Author André Ben Hamou is clear that his book is not an exhaustive guide to using Ruby for systems administration. Rather than try to cover every possible context he provides an introduction to the language and some of its key libraries intended to give a feel for how it might be used and why it leads to succinct and expressive solutions. A number of the more important libraries for working with network protocols and files are covered, and there’s a good introduction to rubygems and how they can be used and created.

Having not done much work with Ruby on the command line I found the first couple of chapters, which cover command-line switches that can help with one-liners for file processing, particularly informative, though I suspect I’ll be referring back to them for a while until the different options take hold. As with the book as a whole those chapters are clear and to-the-point, helped by a presumption that the reader has a good understanding of the problem space and some experience with using scripting languages to simplify their life.

Don’t go into this book expecting to come away ready to work as a sysadmin. That’s not its intention. Nor is it a comprehensive guide to ruby, and you’ll probably still want a good language reference to go with it. But it provides a number of helpful hints and a good sense of how robust scripts can be built quickly and simply with ruby, and there are likely to be a few helpful tricks for most readers.

Disclaimer: I was sent a copy of this book for review by the publisher. You can find it at apress, amazon US, amazon UK and all sorts of other places.

, , , ,