• Making a jQuery Plugin Truly Customizable » Learning jQuery – Tips, Techniques, Tutorials

  Some nice tips for managing options, and a reminder to find _useful_ customisations not just load with customisation options without much thought about/consultation with other potential users

• iPhone Coding For Web Developers

  Presentation slides from the internet's Matt Biddulph

• Rack::Test released: Simply test any Rack-compatible app — Bryan Helmkamp

  There's a _lot_ to like about increased adoption of rack. "With Rack::Test, we hope to make it easy for frameworks to encourage their users to write tests by making it trivial to provide a testing environment. We’d like to foster compatibility between Ruby web app testing environments (especially important as ideas like multi-framework apps become more prominent). The philosophy is the library should stay small and extendable so frameworks can layer on additional functionality they want to offer without modifying Rack::Test’s core behavior or resorting to monkeypatching."

• Newspapers and Thinking the Unthinkable « Clay Shirky

  "That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen …. Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify." … and a a lot more

• russell davies: newspapers and all that

  "If we are going to create a new news ecosystem involving advertisers (and a lot of people would be grateful for that money) then we're going to have to do something about that institutional bifurcation between content and commerce. We're going to have to design the relationship between the two with the care of a good experience designer." – a response to Ben Hammersley asking if anyone talking about the future of newspapers had talked to anyone in advertising

• Streams, affordances, Facebook, and rounding errors – Laughing Meme

  "Simon Willison asked this week about best practice for architecting activity streams. And the answer is, “It depends.” Depends on the scope, scale, access patterns, and affordances you’re building — your contract with your users.

  Which is a long way of saying think hard about the promises you make to your users, implicitly or explicitly.

  And, Facebook, my friend, what the HELL are you thinking? You managed to negotiate the best deal in the business, talk about a racket, and you threw it away for a piece of Twitter’s pain? Are you stupid? Well, best of luck with that."

• SXSW Interactive Videos and Podcasts | SXSW.com

  Most of the sessions were recorded and this is the place to get hold of them.

• SXSWi: Location-based service is the trend at Austin, Texas |

  "Predictably, location-based services were a major feature this year, with launches that included Foursquare, a social, location-based game by the Dodgeball creator, Dennis Crowley, and a new Facebook application for the location management tool Fire Eagle. While early adopters such as the SXSWers have been exploring location-based services for some time, it is inevitable that more consumer and privacy-friendly versions will start to creep into the mainstream."

• stevenberlinjohnson.com: Old Growth Media And The Future Of News

  "I think it’s much more instructive to anticipate the future of investigative journalism by looking at the past of technology journalism. When ecologists go into the field to research natural ecosystems, they seek out the old-growth forests, the places where nature has had the longest amount of time to evolve and diversify and interconnect. They don’t study the Brazilian rain forest by looking at a field that was clear cut two years ago." … and …" Measured by pure audience interest, newspapers have never been more relevant. If they embrace this role as an authoritative guide to the entire ecosystem of news, if they stop paying for content that the web is already generating on its own, I suspect in the long run they will be as sustainable and as vital as they have ever been. The implied motto of every paper in the country should be: all the news that’s fit to link."

• On running a panel

  A mixup over bus times meant I didn't make it to Andrew's panel at SxSW, but I heard many good things. It's really great to see this kind of debriefing-in-public going on. Hopefully it'll make for a stronger set of talks and panels next year.

• Guardian API Maps – Home

  "This is a site that lets you search the Guardian's new API and add location information to articles. All the place data we collect is being made available to anyone who wants it."

• Foursquare, Hot New Phone App, Is Dodgeball on Steroids | The New York Observer

  Quite a few people seemed to be playing with Foursquare at SxSW but most of the Brits were excluded as we didn't want to use that much data and it wasn't available in the UK iTunes store. One to watch, though.

• A few notes on the Guardian Open Platform

  I saw Simon present the Guardian Platform at SxSW and it looks like a great achievement. Waiting to see what developers build on it, and how they roll some of the ideas back in

• Taking remote imagery offline to Nigeria :: High Earth Orbit

  Andrew's notes on trying to source good map data for use in Nigeria. It's a useful overview of a variety of services and ways to use them, though highlighting the absence of really accessible, high-quality data.

• Pulse Laser: The Utility of the Unfinished

  "One technique that S&W has been using recently to illustrate design work is placing sketches or wireframes in situ. Whilst wireframes themselves are incomplete artefacts, designed to be work in progress, they still suffer for being uniformly incomplete. Wireframes themselves can be almost too beautiful, and this means that it becomes all-too-easy to criticise them as only wireframes, rather than as part of a product that exists in the world. Contextualising the sketches into the photograph places the design into the world. This enables the design to be understood within the world, and also (importantly) to highlight the seams between the unfinished design and the finished world around it"

• Spike: a log file viewing & (if we’re being generous) analysis tool for Rails developers.

  Looks like a handy addition to the toolkit

• Generation Open | FactoryCity

  "Sharing and giving away all that you can are the best defenses against fear, obsolescence, growing old, and, even, wrinkles. It isn’t always easy, but it’s how we outlive the shackles of biology and transcend the physicality of gravity." – Perhaps an overly optimistic piece, but it connects together a number of current themes and we can hope…

• Testing Facebook with Cucumber | opensoul.org

  For those faced with the unpleasant task of writing facebook apps, some people are working on making sure they can be thoroughly tested.

• scraplab : instant sinatra deployment with heroku

  A lot of people seem to be excited about heroku lately, and it does look like a nice simple way to put up quick ruby apps. Must play soon.

• How to speed up gem installs 10x « The Budding Rubyist

  Handy little tip, particularly for server environments: turn off ri and rdoc generation in your .gemrc file, and speed things up considerably

• Facebook in 2010: no longer a walled garden – O'Reilly Radar

  A more positive spin on facebook's changes from David O'Recordon, who suspects they're going to pull down the walls around their garden and become a proper citizen of the open web.

• Facebook blinks, copies Twitter, still gets it wrong. – broadstuff

  Critical commentary on facebook's recent changes. I'm not sure I entirely agree with statements like "By 2009 it was clear no one gives a sh*t about the Social Graph" but facebook really do seem to be finding that their approach is overly complex and quickly trying to shift to a more twitter-like "web of flow" (to steal Stowe Boyd's phrase)

• Acquia Search goes public beta | Acquia

  Hosted solr for drupal: "Acquia Search can be installed as a module on any Drupal 6 site, and enhances a site's search experience with faceted search navigation, content recommendations, and configurable results weighting, all delivered through a redundant hosted service infrastructure.".

• Oauth using pecl/OAuth

  Looks like a nice simple way to interact with oauth from a PHP app

Implementing a (not quite public yet) design recently I had need of a way to specify a banner image for each post. While wp has pretty good support for adding various media into the body of posts, this needed to sit outside the post body.

I whipped together a quick plugin to handle uploading a banner and storing its details in the metadata for the post. It was a simple process, nicely self-contained, except that the post edit form doesn’t have the appropriate enctype=”multipart/form-data”. I looked around for any hooks that would allow me to cleanly add attributes to the form tag, but in the end resorted to editing wp-admin/edit-form-advanced.php to add it.

I’d hoped that there’d be time to find a cleaner way to do all this before telling people about it—perhaps some javascript that hooks into the existing media selector but allows it to populate a custom data field?—but it hasn’t, so I’m throwing it out there to the wider world as-is. The code is at github. Feel free to take it and use it as-is, to fork it and update it to be a better wordpress citizen, to email me patches to apply to my copy, or even to employ me to spend more time cleaning it up! Either way, it deserves to be out in the open and hopefully it’ll be of us to somebody besides me.

You can now find the comparison chart at: http://jystewart.net/process/resources/rails-geo-plugins/

A few updates have been lost along the way as they were solely made on the wiki, but hopefully it’s still of use. Since I published the original reviews and chart my attention has wandered a little from the geo plugin scene, so please do flag up any new plugins, changes in features or fixes that I may have missed. I’m going to be trying to check through all the existing listings to update them but that may take a while, so comments here may well encourage me to focus more quickly.

The feature set of GeoX is fairly straightforward. It supports a number of geocoding back ends: obviously google and yahoo are covered, but also mapquest’s relatively recent API. The standard lookup process is much like that provided by several other plugins—the sample given is:

require 'geox'
 
# uses the google engine
geocoder = GeoX::Geocoder.new(:geoengine => GeoX::Google) 
location = {:address => '701 Ocean St', :post_code => '95060'}
geocode = geocoder.geocode(location)
 
# geocode will be a hash containing the geocode data returned from the server
puts geocode.inspect

What is most notably lacking is any ActiveRecord integration such as that offered by acts_as_locateable or acts_as_geocodable. That is apparently by design as the intention is for the library to be usable outside rails but the plugin relies on the rails-specific blank? method and personally I prefer the graticule/acts_as_geocodable split where functionality equivalent to that of GeoX is bundled in the graticule gem and then ActiveRecord hooks are provided by the plugin. It is definitely nice to have geocoding handled manually when models are added or updated, but more than that it is very useful to have support provided for searching based on location.

Probably the most intriguing aspect of the plugin is the facility to compare the specificity of two different sets of coordinates. That’s achieved by returning an object type appropriate to the specificity. So if you have street-level data, the object returned is an instance of GeoX::Street. Operator overloading allow you to compare the objects:

puts "Street wins!" if GeoX::Street.new < GeoX::Block.new

The README file suggests putting the API keys in the file lib/geox_api_keys.rb within the plugin. That’s probably not the best of ideas, but it would be easy enough to rework it so that the configuration is done in the usual rails way and the plugin can be included with svn:externals or piston.

I can’t see myself switching to GeoX from the options I currently use and it’s a shame that its distinctive features weren’t submitted as patches to existing projects rather than adding yet another plugin to the increasingly crowded scene, but it’ll be interesting to see how it evolves over the coming months. ]]> http://jystewart.net/process/2007/10/rails-geo-plugins-geox/feed/ 3 http://jystewart.net/process/2007/03/rails-geo-plugins-geokit/ http://jystewart.net/process/2007/03/rails-geo-plugins-geokit/#comments Mon, 12 Mar 2007 17:52:59 +0000 James Stewart http://jystewart.net/process/2007/03/rails-geo-plugins-geokit/ More >]]> There’s quite a bit of overlap between GeoKit and acts_as_geocodable/graticule, as the latter pair were based on GeoKit. But it provides at least one feature (IP-based location lookup) that they don’t, so I decided to give it a whirl.

Since my main geographically related projects are both now based on plugins that I’m pretty happy with and which suit them well, I decided to resuscitate an old sample piece. A few months back I wrote about scraping the Grand Rapids bus routes site and put up a toy application utilising the resource features in then-edge Rails. I’ve been meaning to return to that project to test out some features in ActiveResource, but in the meantime it seemed like it might be useful to be able to search for the nearest bus stop.

The plugin comes with an extensive README file and getting up and running is very straightforward:

class BusStop < ActiveRecord::Base
	acts_as_mappable :default_units => :kms, 
		:default_formula => :flat, 
		:distance_field_name => :distance,
		:lng_column_name => 'longitude', :lat_column_name => 'latitude'

I had to add the column_name parameters to signify which database columns I was using, as the plugin defaults to using ‘lat’ and ‘lng’ for brevity. The inclusion of the default_units parameter is a nice one, but it would also be nice if the plugin provided an accessor method to convert distance on the fly to help developers localise their apps.

With that done I get access to a suite of methods for doing location searches. So if I wanted to find the nearest bus stop to Common Ground Coffee Shop I could call:

BusStop.find(:nearest, 
	:origin => '1319 Fulton St. East, Grand Rapids MI 49503')

That address actually failed in the google geocoder (probably because I had yet to enter my api key), but GeoKit automatically fell back to geocoder.us and got the co-ordinates.

Where it seemed to fall over was when I tried to limit by bus route:

BusStop.find(:nearest, :origin => '1319 Fulton St. East, Grand Rapids MI 49503',
	:include => :route_stops, 
	:conditions => 'route_stops.bus_route_id = 6')
 
ActiveRecord::StatementInvalid: Mysql::Error: Unknown column 'distance' in 'field list': SELECT DISTINCT bus_stops.id, distance FROM bus_stops  LEFT OUTER JOIN route_stops ON route_stops.bus_stop_id = bus_stops.id WHERE (route_stops.bus_route_id = 6) ORDER BY distance ASC  LIMIT 1

Andre Lewis, one of the plugin’s developers (along with Bill Eisenhauer), tells me that GeoKit doesn’t currently support the :include option or conditions on join tables. He’s hoping to add that soon, but for now you’re limited to a single table.

Implementing the IP-based geolocation was very easy. A before_filter is included which can be included in a controller with the declaration:

geocode_ip_address

will return a GeoLoc object containing details of the lookup source, the address and the co-ordinates. I chose to add my own before_filter so I could find the nearest bus stop:

class ApplicationController < ActionController::Base
  before_filter :find_nearest_stop
 
  def find_nearest_stop
    @nearest = BusStop.find_nearest(:origin => get_ip_address)
  end
end

(where get_ip_address is a method provided by the plugin). I had to deploy to a remote host to actually test this as 127.0.0.1 obviously doesn’t work as a source IP address for these purposes, and it’s likely that a number of users working through proxies of one sort or another will find the information less than accurate. But so long as that’s clear, it’s a nice feature to add to local information services, potentially getting users to relevant information more quickly.

I do like the separation between gem and plugin that acts_as_locateable offers, the fact that geocodes are stored in a separate table, and full join support for queries. But the IP translation from GeoKit is also a nice feature. Both are highly capable solutions, so it’s likely that once GeoKit adds full join support, choosing between them may well come down to personal taste.

(Nb. Those following along with these posts may be interested in GeoKit co-author Andre Lewis’ book Beginning Google Maps Applications with Rails and Ajax. I’m hoping to check it out soon)

The API is a little different in that it returns its data as javascript strings, but otherwise it’s simple enough to send a GET for a given address and get back the data. To see how easy it is, I decided to code up an extra geocoder for graticule that would use this service.

The main change I had to make was to allow for extra arguments to the locate method. For the geocoders distributed with the gem there’s just one argument: the address. For this service to retrieve data outside the USA it requires more structured data, so I added an optional argument for the locate method and built my query based on its existence. The code turned out to be very simple:

require 'graticule'
module Graticule
 
  # A library for lookup of coordinates with http://geo.localsearchmaps.com/
  class LocalSearchMapsGeocoder < RestGeocoder
 
    def initialize
      @url = URI.parse 'http://geo.localsearchmaps.com/'
    end
 
    # This web service will handle some addresses outside the US
    # if given more structured arguments than just a string address
    # So allow input as a hash for the different arguments (:city, :country, :zip)
    def locate(address, args = {})
      if args.empty?
        get :address => address
      else
        get args.merge(:street => address)
      end
    end
 
    def check_error(js)
      raise AddressError, "Empty Response" if js.nil? or js.text.nil?
      raise AddressError, js.text.match(/alert\('(.+)'\)/)[1] if js.text == "alert('location not found');"
    end
 
    def parse_response(js)
      location = Location.new
      coords = js.text.match(/map.centerAndZoom\(new GPoint\((.+?), (.+?)\)/)
      location.longitude = coords[1]
      location.latitude = coords[2]
      location
    end
  end
 
end

So now I can use graticule and specify an address such as:

g = Graticule::LocalSearchMapsGeocoder.new
location = g.locate '14, Avenue Claude Vellefaux', 
	:zip => '75010', :city => 'Paris', :country => 'France'

and find out that that address is at latitude 48.873016, longitude 2.369934. Unfortunately it is still quite a limited service; when I threw in some addresses in China, Malaysia and even New Zealand I got Graticule::AddressError exceptions. But such is the current state of geocoding, and getting some of Western Europe on top of North America is progress.

Another extension for graticule that might be nice would be a way to specify countries that a geocoder does/doesn’t support. That way the gem could intelligently choose a geocoder that will work out for the specified address.

Update (March 21st): This code is now included in the latest release of the graticule gem.

I like that separation. Having the generalised code in a gem and the rails-specific hooks in a plugin makes a lot of sense and makes it much easier to use the core code in non-rails ruby apps, and having a single gem that supports multiple services allows for built-in failover should the preferred geocoder be unavailable.

Much of the functionality of the plugin is already integrated into my application, but not with quite so many options. In such cases I really enjoy installing plugins; there’s something very satisfying about going through my application deleting code.

The plugin adds two tables to your database. The first, geocodes, holds longitudes/latitudes for given addresses, while the other, geocodings, polymorphically links those geocodes to your existing models. In my case, this meant re-geocoding all the locations already in my database, but since I’m operating on a fairly small data set, that was a pretty simple case of iterating across them all and re-saving them. For those operating with very large databases, you may want to write a more sophisticated migration to handle that.

The trickiest thing was re-coding my search queries to use the new database. acts_as_geocodable offers a number of neat methods for running queries such as (from their documentation)

  event.distance_to "49423"
 
  Event.find(:all, :within => 50, :origin => "97232")
 
  Event.find(:nearest, :origin => "Portland, OR")

But I wanted a way to build more sophisticated searches so I could, say, limit by title and order by distance. It turns out that’s pretty easy too:

Location.find(:all, :origin => 'Grand Rapids MI', 
	:conditions => ['title LIKE ?', 'My Title'] :order => 'distance')

The one place where I had a problem was when trying to use the last of the examples.

Location.find(:nearest, :origin => 'Portland, OR')

blew up with:

ActiveRecord::StatementInvalid: Mysql::Error: Incorrect parameter count in the call to native function 'RADIANS': SELECT locations.*, geocodes.*,
            (ACOS( SIN(RADIANS()) * SIN(RADIANS(geocodes.latitude)) + COS(RADIANS()) * COS(RADIANS(geocodes.latitude)) * COS(RADIANS(geocodes.longitude) - RADIANS()) ) * 3963.1676)  AS
            distance FROM locations  JOIN geocodings ON
              locations.id = geocodings.geocodable_id AND
                geocodings.geocodable_type = 'Location'
              JOIN geocodes ON geocodings.geocode_id = geocodes.id  ORDER BY distance ASC LIMIT 1

but when I used a full address (my home) in the same query, I got an appropriate result. It looks as though perhaps if it fails to get an appropriate pair of co-ordinates for the specified location, it tries to perform the query anyway, with an exception resulting.

I also found some problems when trying to use the plugin with locations outside North America, but that is a limitation of the geocoding services and not of the gem or plugin themselves. Hasten the day when enough data is open that global geocoding services can become a reality.

Working with acts_as_geocodable has so far been a very straightforward experience and has allowed me to rid my code of some pieces I’d always meant to refactor out. It’d be good to see the error I ran into handled more neatly, and perhaps an obvious API to take advantage of the failover options presented by graticule, but the plugin is still early in its life and shows a lot of promise.

Typically if I’m using an auto complete field in a form it’s because I want to link that record with an existing entry in another table, such as connecting an event with a location, or a book with a publisher, and that won’t work because the value obtained from the auto completion is the name of the linked entity, not its ID. I’ve tried various ways of managing that, either by overriding the standard methods or by adding some extra code in my model that will convert a name back into the associated model ID, but they all feel like hacks and I’d not had time to clean up the code and package anything as a plugin.

Thankfully Xavier Nora’s already done it with the model_auto_completer plugin. The plugin provides a set of helper methods that will allow you to use auto completion, but have the ID of the record found stored as a hidden field in your form, making processing considerably easier.

Initially I had a problem using the plugin, and in fact found my rails application falling over because the plugin wasn’t getting the right name for the foreign key and since I have other fields that allow entry of a new location, rails couldn’t tell if a post with:

location[]=123&location[name]=My%20Location

should use the first location data or the later data.

Looking through the code I found that instead of using:

foreign_key  = real_object.class.reflect_on_association(association).options[:foreign_key]

to get the foreign key it should be using:

foreign_key = real_object.class.reflect_on_association(association).primary_key_name

Having submitted a patch, I was very pleased to hear over the weekend that the change has been integrated into the plugin so I can let piston update back to the official trunk. With that change in place, the plugin is a great way to make data entry much more convenient.

I’ve been procrastinating on getting my first Ruby on Rails plugin (first anyone else might be interested in, at least) out the door, mainly due to some configuration problems with Subversion, but finally I have that all sorted out and the plugin ready to go.

loads_from_amazon is an ActiveRecord Mixin that works with the Ruby/Amazon library to let you populate an ActiveRecord object with the details of a product listed on amazon. In the README I use the following example:

class Book < ActiveRecord::Base
  acts_as_amazon
  maps_to_amazon_attribute :authorlist => 'authors', :combine => ';'
  maps_to_amazon_attribute :title => 'product_name'
  maps_to_amazon_attribute :isbn => 'asin'
  maps_to_amazon_attribute :publisher => 'manufacturer'
  maps_to_amazon_attribute :pubdate => 'release_date'
end
 
@book = Book.load_from_amazon(isbn) # Loads but does not save
@otherbook = Book.load_from_amazon!(other_isbn) # Loads data and saves to database

So feel free to grab it from svn (http://projects.jystewart.net/svn/rails/loads_from_amazon/trunk) and let me know what you think.