Mar 07

Versatile RESTful APIs Beyond XML

An article I wrote has just been published over at InfoQ. It’s called Versatile RESTful APIs Beyond XML and shows how easy it can be to extend Rails’ RESTful behaviour to input and output resources not only as XML but also as JSON and Microformatted HTML.

The article builds on some posts on this blog, such as Intercepting Microformats In Rails Input, but offers a bit more context. The timing of the article fits nicely with a post on the microformats-rest list about Rails, REST and microformats, so hopefully we’ll see more discussion of these concepts over the coming weeks.

Mar 07

RESTful Rails Development PDF

I’d been meaning to take a look at Ralf Wirdemann, Thomas Baustert, Florian Görsdorf and Adam Grove’s Restful Rails Development PDF for a few days, and time travelling over the weekend gave me that opportunity.

I didn’t go through it all in great detail, as it’s fairly introductory material, but if you’re wondering how to get started with RESTful rails it provides a great overview to get you going. You can find it here.

(via rubyinside.com)

Feb 07

Relax over REST

Mark Nottingham has a good post running through a few topics on which people get needlessly caught when designing RESTful applications. If you’re new to working on RESTful application design (as many rails developers are) it’s worth checking out to save yourself needless anguish.

Thankfully for Rails developers at least some of the issues he identifies will be a little simpler than they might be for people designing systems from scratch. In particular, while there are a few URL design choices (numeric IDs, other parameters, or a hybrid? nested vs. flat?) the conventions are good and changing isn’t all that hard.

And while some sort of schema definition is important to ensure that server and client can be sure they’re speaking the same language, for those using tools like ActiveResource which take full advantage of dynamic languages, there’s a little more flexibility than there may have been before.

Feb 07

Intercepting microformats in rails input

In Input formats and content types in Rails 1.2 I mentioned a project I’ve been working on that will provide a RESTful service interface which accepts its input in a number of formats, including microformatted HTML.

For certain types of data microformats provide a great way to receive input as they don’t require your clients to learn a new schema to send you data. They can take the same semantically rich HTML they’re displaying on their website and POST it to your endpoint. Or they can use a tool like Ryan King’s hcalendar creator to generate some sample input.

But intercepting and interpreting that data isn’t quite so simple as JSON or XML. Those formats have well defined content types that we can use to identify them, and use the parameter parsers I described in my earlier blog entry. Microformats are HTML and so will come with an HTML content type, just like other forms of input.

It would be nice if there were a simple way (short of running them through a full parser) to identify POSTs whose bodies contain microformats, but so far I haven’t come across one. What we can do is to override rails’ default handlers to parse the raw post data and see if it looks like regular form input. If it doesn’t, we can presume it’s meant to be considered to be microformat data and we can do some parsing using the excellent mofo (on which more, later). My code at present is:

microformat_interceptor = Proc.new do |data|
  parsed = CGI.parse(data)

  if parsed.collect { |key, value| key + '=' + value[0] }.first == data
    { :event => HCalendar.find(:text => data) }

Mime::FORM = Mime::Type.lookup("application/x-www-form-urlencoded")
ActionController::Base.param_parsers[Mime::FORM] = microformat_interceptor
ActionController::Base.param_parsers[Mime::HTML] = microformat_interceptor

With that code in environment.rb, a request with hCalendar data in its post body will look to our actions as if it had an HCalendar object in params[:event]. If we extend a few of our event model’s methods, our controller can treat this input just as it would XML or a standard query string.

Obviously this is a work in progress, its output is very simple, and it’s not all that versatile, but so far my tests stand up well, and it makes the code considerably more elegant.

For a little more on microformats and APIs, check out Drew’s “Can Your Website be Your API?” and the REST page on the microformats wiki.

Feb 07

Input formats and content types in Rails 1.2

One feature of recent releases of Rails I hadn’t spotted before is the ability to define your own parameter parsing based on content type. I’m working on an application that will employ a RESTful API and that I hope will take its input in either standard http parameters, microformatted HTML, XML or JSON.

I don’t really want to have to write custom code within the controllers to interpret the input based on content type, so I started looking for how rails parses XML input and came across the following in the actionpack changelog:

    # Assign a new param parser to a new content type
    ActionController::Base.param_parsers['application/atom+xml'] = Proc.new do |data|
      node = REXML::Document.new(post)
     { node.root.name => node.root }

    # Assign the default XmlSimple to a new content type
    ActionController::Base.param_parsers['application/backpack+xml'] = :xml_simple

Looking at the actual source code it appears it’s actually being implemented slightly differently, with the Mime::Type object being used as the key, rather than the text content type. Since the json content type is already defined (with a reference to the object in Mime::JSON), JSON can (usually) be parsed as YAML, and the :yaml symbol is a shortcut to a YAML parser, handling it transparently is almost as simple as adding:

ActionController::Base.param_parsers[Mime::JSON] = :yaml

or if we wanted to be a bit more explicit:

ActionController::Base.param_parsers[Mime::JSON] = Proc.new do |data|

to environment.rb. Building these APIs is even easier than I’d thought!

Sep 06

Corrected bus routes on Rails

In the process of building my bus route app, I realised that half the data for bus stops is missing. While the site’s developers have done a good job of providing clear data on half the stops, if you want to see stops going in the other direction, you have to use a drop-down box that triggers an AJAX request and repopulates the table.

A little digging shows that the call is to:


which returns an HTML table with the relevant stop data. So in a sense, there are permalinks for each set of stops, but it’d be nice if they were more clearly advertised, particularly since the site as is won’t work for those without javascript switched on.

The other gotcha is that it seems the internal IDs for some routes don’t match their route numbers. If you try and retrieve the westbound stops for Route #14 the call is actually to:


and when you make requests for route 13, the routeID passed is 14. The same disparity continues, suggesting that they’ve (sensibly) added primary keys to their database other than the route number. It turns out that ID is embedded in the markup within a comment showing the direction and the ID. For Route #50 that is:

<div id="stopListWrapper">
<!-- E -> 19 -->
<div id="stopList">

Since the document is already being parsed using hpricot, we can get that with:

internal_route_id = doc.at("div#stopListWrapper").children[1].to_s.match(/\-\> (\d+) \-\-\>/)[1]

(get the div, note that the comment is the second child, and get the data with a regular expression)

I’ve updated my scraper and the service to grab data based on the correct IDs. The HTML views will follow suit shortly.

Sep 06

Bus routes on Rails

Following on from my previous entry about scraping bus route data from The Rapid’s website, and to begin to demonstrate the possibilities it opens up, I’ve set up a simple web service to provide route and stop data. It’s based on the new REST style from Edge Rails, and routes are scoped by city to allow for future expansion. To get data on Route 1, GET:


To get a list of the stops within 1.5 miles of a given longitude and latitude, GET:


Using Edge Rails, setting up the application was remarkably simple. Three models, three controllers, appropriate use of respond_to blocks, and the right entries in config/routes.rb:

map.resources :cities do |cities|
  cities.resources :stops
  cities.resources :routes

This was the first time I’ve used nested routes so it took a few minutes to work out the correct syntax for the link_to calls. When using nested routes like those above, you must declare first the ID of the city and then the ID of the stop or route, eg:

I’m not making any guarantees about the long term availability of the service, but if anyone wants to make use of it, let me know and we can probably work something out. I’ll probably be making use of it myself.

Feb 05

Services_Technorati Proposal

After a few days’ distraction I’ve now submitted a draft proposal for Services_Technorati. The attention.xml support still isn’t working as it should, but other than that it seems to hold together. Getting it up there hopefully means a little help with testing, and perhaps a bit more impetus to keep working on it.

You can find the PHP source and a quick example here.