Aug 09

Cucumber, wordpress and database_cleaner

I’m up to my usual using-ruby-tools-to-test-other-environments tricks, using cucumber and my wordpress activerecord classes to do acceptance testing against a highly-customised wordpress install.

I’m hoping to write a bit more about that soon, once I’ve put it through its paces a little more and cleaned up some of the code, but I wanted to quickly mention one of the key pain-points and an extremely handy solution.

One of the things I enjoy most about testing in rails is the handy tasks to prepare a blank database and the transactions that ensure the database is returned to that state after each test. Obviously that wasn’t going to work cleanly with wordpress since it doesn’t use ActiveRecord, but Matt kindly pointed me in the direction of Ben Mabey’s database_cleaner gem.

With that installed it’s as simple as adding:

require 'database_cleaner'
require 'database_cleaner/cucumber'
DatabaseCleaner.strategy = :truncation, {:except => %w[wp_options]}

to my


file and my database is reset on each step. Combined with a few hooks to manage the configuration and load in the schema at the appropriate point, it’s turning into quite a nice little testing environment.

Jul 09

Talking to WordPress with ActiveRecord

As mentioned in yesterday’s announcement I’m pulling some content across from this blog (running on wordpress) into the new Ket Lai site (a merb app). I’ve found myself doing similar things a few times lately, such as on Only Connect (on which more, soon) and so have built out a selection of ActiveRecord models to help me talk to a wordpress database from a ruby app.

At Matt‘s urging (he’s been using them to move data from a legacy site), I’ve finally put those models up on github. Being a single file they arguably should have been a gist, but I’m reserving the right to reorganise them in future.

They’re far from complete in that there are lots of validations I could have added in, named scopes that would probably be handy, and loads of convenience methods that some might like, but hopefully they’ll be of use to someone and evolve over time.

(on the subject of wordpress, I’m glad to see the addition of changelogs to the plugin directory – being asked to upgrade plugins without any idea what’s changed has long been a bugbear of mine, so hopefully this will resolve some of that)

Mar 09

Selected (belated, extended) Saturday Links

The past two weeks haven’t really left time to compile my selected links, though there have been many. A few days at SxSWi (on which more, later) followed by travelling with the family and the inevitable work backlog moved blogging way down the priority list. So here’s a mammoth selection to get me caught up. Particularly interesting has been the discussion around the future of newspapers (represented here by Clay Shirky, Steven Johnson and Russell Davies), which seem to have finally pushed beyond “how t ind a good business model for papers” to looking at where the real value for society lies and how we can preserve and extend that in a changing landscape.

Mar 09

Selected Saturday Links

Big themes this week have mostly revolved around twitter, facebook, and openness. Some have focussed on facebook redesigning to embrace a more twitter-like “web of flow” approach, and others on the fact that they’re jumping on various open web bandwagons. It’s been interesting to see some tie in with the government transparency thinking going around, as particularly noted by Chris Messina on FactoryCity. Meanwhile there are quite a few nice new tools emerging, and I really must try heroku one of these days.

Aug 08

Announcing image_associations

Discussions of multi-model forms and nested models in Rails has been revived recently, with various changes appearing in Edge Rails, plugins like attribute_fu getting a lot of attention, and the release of ActivePresenter. It looks like when the dust settles we’ll have a nice new set of ways to simplify our code.

One thing I frequently find myself doing is associating multiple images with a given model. My news story might have a banner image, and a series of other attachments, which I could specify with:

class Story < ActiveRecord::Base
  belongs_to :banner_image
  has_many :story_attachments

class BannerImage < ActiveRecord::Base
  has_attachment # I'm using attachment_fu

class StoryAttachment < ActiveRecord::Base
  has_attachment # I'm using attachment_fu

(or I could use has_one in place of belongs_to there, your tastes/requirements may vary)

What quickly becomes a pain is assigning the images to the models, and having rejected fat controllers I often end up writing accessors on my models to manage that for me:

class Story  data)

  def valid_file?
    # etc.

That quickly gets dull, so I’ve wrapped it up in a plugin I’m calling image_associations. With that I can write:

class Story < ActiveRecord::Base
  belongs_to_image :banner_image
  has_many_images :story_attachments

and get the accessors for free.

It may only be of use to me, and it’s pretty crude (there’s no support for deleting attachments, for example) but it’s been working nicely and has DRYed up my code in a satisfying way, so I thought I’d throw it out there. You can find it over on github.

(I’m not promising to keep supporting it, but it is used in an active project so chances are bug fixes will make it in. And of course, anyone is welcome to branch the project and twist it in their own ways)

Feb 08

Using the Django ORM as a standalone component

Django logoI’ve found myself working in Python lately, both for a new project and while preparing a review of the Django book. Working in Ruby I’ve become used to relying on ActiveRecord whenever I need to talk to a database (whether inside Rails or not) and after a little time refamiliarising myself with MySQLdb I realised I was going to need a decent ORM for this project. As well as being road-tested and well documented, I was also looking for something that could either generate its models on the fly, or had a tool to generate them from an existing schema.

I checked out SQLAlchemy and SQLObject, both of which looked like worthy contendors, but couldn’t find the generator I was looking for (if you know of one, please do let me know in the comments!) so I switched over to Django. I couldn’t find much information on using their ORM standalone, so thought I should share what I discovered. Please bear in mind that my python is rather rusty–if there are better ways to do any of this I’d be very pleased to hear about it.

The tool to generate your model classes from an existing database is django-admin.py which should be in your path if you’ve got Django installed. To get that up and running you’re going to need to set a few options.

First up, you’ll need a settings.py file specifying your database details. Mine looks like:

DATABASE_NAME = 'mydatabase'
DATABASE_USER = 'myusername'
DATABASE_PASSWORD = 'mypassword'

Once that is done and saved, then from the command line you should be able to call:

django-admin.py inspectdb --settings=settings

and the models will be echoed to stdout. To redirect that to a file called models.py you’ll need:

django-admin.py inspectdb --settings=settings > models.py

I found a few areas where the generated objects threw errors. We have a column called ‘try’ which is a python keyword, so that required a small change to the code. And a number of models have foreign key relationships with other models that are declared after them, so a little reorganising was called for. It’d be really nice to have the script handle that, but it wasn’t a big deal to make the changes by hand.

With that done, I tried a very simple new script:

from models import *
a = MyModel.objects.all()
print a

But ran into a few issues:

The ORM is relying on the environment variable DJANGO_SETTINGS_MODULE to tell it where to find your database credentials. You’ll need to set that to point to the settings.py file you created earlier.

You can’t have your models.py file in the same folder as your test script. Doing so throws a “IndexError: list index out of range” error. Instead make a new folder called, say, orm and put models.py in it, along with an empty file called __init__.py which will tell python to treat the folder as a module. You can then update your test script to:

from orm.models import *
a = MyModel.objects.all()
print a

With that done, you should be able to use the ORM just as you would in your Django view code.

Jan 08

Book Review: Practical Ruby Gems

Practical Ruby Gems sleeveFor those who aren’t aware, ruby gems refers a way of packaging up code so it can be easily distributed for other developers to use, and a tool to help with the distributing and/or installing that code. Find out more at wikipedia.

On first glancing at this book, I wondered how you could fill a full-length book on the topic of gems. While getting the tools installed on some systems requires care, and there’s space for a couple of chapters on packaging your own libraries as gems, both topics have been covered alongside other topics in numerous volumes. What I’d missed was that contents not only covers both of those topics, but also looks at 26 different gems and explains how you might use them in your projects.

The coverage of setting up and using ruby gems at the start of the book, and on packaging and distributing your own gems at the end of the book are brief but cover the basics well enough. It might have been helpful had the latter included a little information on how to include other libraries that need to be compiled and managing the cross-platform issues that raises. Chances are anyone planning to do that is well capable of reverse-engineering a gem spec file to work out what’s needed, but it would have added some weight and helped these chapters stand out from their equivalents in other volumes.

The gems profiled cover a wide range of uses: databases, certain web services, parsing HTML, web frameworks, recurring events, PDF generation, and more. Most of them are pretty well known libraries, but few of us will have had a chance to try them all out and newcomers to the ruby community looking for some support in their projects may well find some useful tips within.

For each gem there is an introduction, some basic code samples, a lengthier code sample (with commentary) and conclusion. That repetition means this isn’t a book many will want to wade through in one go. I found that after a while I needed a break, and if reading the book it may be best to pick out a gem that particularly interests you, read the relevant section and then write some code of your own before moving on to another.

A number of times I wondered if it would be preferable to drop some of the best known gems (there are plenty of ActiveRecord examples floating around) in favour of a little more depth. In particular it would have been interesting to see a dissection of how a few of the gems work, as a sign that anyone can contribute, to see what can be learned from techniques used, and to better understand how the gems in question can be used. Such explorations might serve to break up the text a bit, as well as providing useful insights into ruby development and perhaps broadening the appeal of the book a little.

Overall, Practical Ruby Gems was a helpful read and there are a few new gems I’m going to be exploring as a result of my reading, but you may well want to take a close look before investing in a copy. You may broaden your ruby knowledge in the reading, but you’re unlikely to deepen it significantly.

Disclaimer: I was sent a copy of this book for review by the publisher. You can find it at apress, amazon US, amazon UK and all sorts of other places.

Dec 07

Rails 2.0 : Very Soon Now!

All reports, and the evidence of the subversion commit log, says Ruby on Rails 2.0 will be with us very shortly. Apparently the announcement has been delayed until the gems have properly circulated. For those who’ve not been following, Rails 2.0 isn’t going to seem like a huge step forward as it’s mainly focussed on cleaning up existing features, and moving quite a few out of the core and into plugins.

I’ve converted quite a few projects over to be 2.0-compatible over the past few weeks and I’ve always come away feeling like my projects are cleaner as a result. It’ll be interesting to see what the benchmarks say about the performance impact of the new release.

Some of my favourite features of the new release are:

New view filenames and improved mime type support

Rather than filling up your views folders with .rhtml, .rxml, and .rjs files you’ll now be looking at filenames of the form myaction.html.erb, myaction.atom.builder, etc. If you ever found yourself working with the likes of myaction_atom.rxml, myaction_rss2.rxml, and so on, you’ll understand that the new naming system makes life simpler and your code simpler. In general, it really feels like the support for different formats that has been creeping into rails for a while now is finally coming of age with 2.0.

Pseudo-mime types

Closely related to the previous point, it’s nice to see an officially supported way to present different content to different devices just as you’d supply different formats for different requests. That’s written up well in this piece on building an iPhone UI for your Rails 2.0 application, and would have come in very handy when I was doing my work on intercepting microformats in rails input. Put simply you can define extra aliases for a given mime type, eg:

Mime::Type.register_alias "text/html", :my_extra_format

then use a before filter to identify a request as your pseudo-mime type:

  before_filter :detect_input_format

  def detect_input_format
    if request_matches_my_criteria?
      params[:format] = "my_extra_format"

and then the resulting action will render the view myaction.my_extra_format.erb (or myaction.my_extra_format.builder, etc.).

If the long-predicted explosion of the mobile web does indeed come in 2008, or more people adopt the facebook application approach, I can see this feature getting a lot of attention.

ActiveRecord Query Caching and Serializations

Lots of databases support native query caching, but it’s good to see it going in at the ActiveRecord level as it’ll sit a little closer to your app and be something you can rely on. I’ve seen a good speed up on some apps that I’ve been running on Edge.

ActiveRecord objects can now be created from XML and serialized to json much more easily, making inter-operability still easier.

And a lot more

The new initialization process is great, being able to specify :moved_permanently in redirects has cut out code I never liked, fixtures make a lot more sense, the view cleanups are great, and there’s a lot of stuff I never used which is no longer taking up memory in my processes. My experiences with Edge Rails lately mean I’m very pleased with Rails 2.0, am glad that this release is so focussed on clean-up, but I’m also looking forward to seeing what radical changes and refactoring come along once this version is out the door.

You can learn more about Rails 2.0 by checking out this post on the Riding Rails blog, the videos at railscasts.com and Ryan Daigle’s posts on Edge Rails. I’m sure a lot more will emerge over the next few weeks.

UPDATE (later that day): DHH has announced it and it’s official: Rails 2.0 is now out.

Dec 07

DataMapper – Competition for ActiveRecord?

When Ruby on Rails first hit the scene, what attracted many of us to it was ActiveRecord. By providing a declarative syntax for describing relationships, validations, and callbacks it provided an elegance to model code that makes programming a lot more fun. Over the past couple of years ActiveRecord has received a lot of love, with has_many :through and improved caching options being the key additions I’ve enjoyed. But despite all that it’s given, ActiveRecord is clearly not the last word in Object Relational Mappers and it’s been good to see increased attention for some alternative Ruby ORMs such as DataMapper.

DataMapper shares a lot of syntax with ActiveRecord but promises a few new features, such as a more succinct find syntax, eg.

  # Find first entry in my_objects table
  MyObject.find(:first) # ActiveRecord
  MyObject.first        # DataMapper

  # Find all entries in my_objects table
  MyObject.find(:all)   # ActiveRecord
  MyObject.all          # DataMapper

and perhaps more usefully, improved finder options. Edited from the DataMapper site:

# 'gt' means greater-than. We also do 'lt'.
Person.all(:age.gt => 30)

# 'gte' means greather-than-or-equal-to. We also do 'lte'.
Person.all(:age.gte => 30)
Person.all(:name.not => 'bob')

# If the value of a pair is an Array, we do an IN-clause for you.
Person.all(:name.like => 'S%', :id => [1, 2, 3, 4, 5])

# Does a NOT IN () clause for you.
Person.all(:name.not => ['bob','rick','steve'])

You can approximate that functionality with some Rails plugins, and of course you can use the :conditions parameter to an ActiveRecord finder to do the same queries, but there’s something nice about being able to do it all in Ruby. (You can also fall back to ActiveRecord like syntax)

Most uses of DataMapper I’ve seen so far have been standalone, or in conjunction with merb, but I decided to give it a try by converting an existing rails app. Given how much similarity there is in syntax it seemed like it would be fairly easy to convert a few models and see how it compared. Unfortunately, it’s not that easy.

It’s pretty rare to build a rails app without plugins, and most plugins that affect models do so by extending ActiveRecord::Base. Since by using DataMapper you’re breaking away from ActiveRecord::Base, those plugins won’t work without invasive surgery. Similarly, DataMapper doesn’t support AR’s validation syntax and while it will soon have validations of its own, it looks unlikely that they’ll be compatible. It might well be possible to write a plugin that makes many other plugins work by examining ActiveRecord::Base.extended_by, and that could provide a translation layer to allow ActiveRecord validations to work with DataMapper, but that begins to be a lot of work for very little return.

Where DataMapper may really help Rails developers is in introducing more competition into the Ruby ORM world. By taking some radically different design decisions, particularly with regards to migrations and column specifications DataMapper allows us to see a different, but similarly elegant way of doing things, and hopefully ideas will percolate back into ActiveRecord just as AR has clearly inspired DataMapper. Personally I am looking forward to using DataMapper in my next non-rails ruby project, but found the exploration a useful reminder of how much of rails’ success is in its coherency as a stack.

Oct 07

Book Review: Pro ActiveRecord

Right at the start of Pro Active Record the authors address a possible problem some may have with it: that there’s not enough in Active Record to warrant a full book. They point out that the basics are well covered as sections elsewhere but that this is the first book to really dig into working with legacy schema and other ‘advanced’ uses. That’s fair enough, but after reading the book I am still left with the question of why, then, they dedicate the first half to covering ActiveRecord’s most basic concepts?

Judging from postings on the rails email list, there’s certainly a lot of confusion about ActiveRecord, associations, observers, how to work with legacy table names and primary keys, and so on. But in a book with a title prefix of “Pro” I was expecting to jump straight into the nitty gritty of topics like compound/composite primary keys and performance tuning, probably with some real world examples, and maybe with a serious exploration of AR’s internals. As it is, such topics only get a quick treatment in the final chapter (the compound/composite primary keys section is a paragraph referring users to http://compositekeys.rubyforge.org).

It’s almost always instructive reading other developers’ code and it would be unfair to claim that I didn’t spot a couple of tips that may prove useful, but they were passing things. And sometimes I found myself wondering what happened to the tech review process, particularly in the coverage of the has_one association, where not only is the variable naming confusing, but they seem to be calling the each method on a single ActiveRecord instance.

I’m left wondering what the audience is for this book. The title and blurbs suggest it’s pitched at people who want to go deeper into ActiveRecord than they have before, but the content is better suited for someone with some database experience who wants to pick up ActiveRecord to write some scripts. As it is, if you’ve worked with ActiveRecord before your time will be better spent writing plugins and exploring the internals for yourself, and if you’ve not you’ll get most of the same material from a decent Rails book and some time exploring.

Disclaimer: I was sent a copy of this book for review by the publisher. You can find it at apress, amazon US, amazon UK and all sorts of other places.