In the process of building my bus route app, I realised that half the data for bus stops is missing. While the site’s developers have done a good job of providing clear data on half the stops, if you want to see stops going in the other direction, you have to use a drop-down box that triggers an AJAX request and repopulates the table.

A little digging shows that the call is to:

http://www.ridetherapid.org/includes/ajax_return.php?mode=routestops&direction={direction}>&routeID={routeid}

which returns an HTML table with the relevant stop data. So in a sense, there are permalinks for each set of stops, but it’d be nice if they were more clearly advertised, particularly since the site as is won’t work for those without javascript switched on.

The other gotcha is that it seems the internal IDs for some routes don’t match their route numbers. If you try and retrieve the westbound stops for Route #14 the call is actually to:

http://www.ridetherapid.org/includes/ajax_return.php?mode=routestops&direction=W&routeID=13

and when you make requests for route 13, the routeID passed is 14. The same disparity continues, suggesting that they’ve (sensibly) added primary keys to their database other than the route number. It turns out that ID is embedded in the markup within a comment showing the direction and the ID. For Route #50 that is:

<div id="stopListWrapper">
<!-- E -> 19 -->
<div id="stopList">
...
</div>
</div>

Since the document is already being parsed using hpricot, we can get that with:

internal_route_id = doc.at("div#stopListWrapper").children[1].to_s.match(/\-\> (\d+) \-\-\>/)[1]

(get the div, note that the comment is the second child, and get the data with a regular expression)

I’ve updated my scraper and the service to grab data based on the correct IDs. The HTML views will follow suit shortly.

Recommend this post:

Following on from my previous entry about scraping bus route data from The Rapid’s website, and to begin to demonstrate the possibilities it opens up, I’ve set up a simple web service to provide route and stop data. It’s based on the new REST style from Edge Rails, and routes are scoped by city to allow for future expansion. To get data on Route 1, GET:

http://projects.jystewart.net/buses/cities/1/routes/1

To get a list of the stops within 1.5 miles of a given longitude and latitude, GET:

http://projects.jystewart.net/buses/cities/1/stops/?longitude=X&latitude=Y&distance=1.5

Using Edge Rails, setting up the application was remarkably simple. Three models, three controllers, appropriate use of respond_to blocks, and the right entries in config/routes.rb:

map.resources :cities do |cities| 
  cities.resources :stops
  cities.resources :routes
end

This was the first time I’ve used nested routes so it took a few minutes to work out the correct syntax for the link_to calls. When using nested routes like those above, you must declare first the ID of the city and then the ID of the stop or route, eg:

< %= link_to 'My Route', route_url(city, route) %>

I’m not making any guarantees about the long term availability of the service, but if anyone wants to make use of it, let me know and we can probably work something out. I’ll probably be making use of it myself.

Recommend this post:

After a few days’ distraction I’ve now submitted a draft proposal for Services_Technorati. The attention.xml support still isn’t working as it should, but other than that it seems to hold together. Getting it up there hopefully means a little help with testing, and perhaps a bit more impetus to keep working on it.

You can find the PHP source and a quick example here.

Recommend this post: