Posts tagged google
Quick update on Heathrow Tower
Feb 13th
While there haven’t been any visible changes to my Heathrow Tower project in the past couple of weeks beyond my throwing in a few greetings in other languages to break things up a bit. Having put some of the statistical plans on hold as the snow last week prevented any data gathered from being anywhere close to representative, I’ve gradually been building up the database behind the scenes so I can start to do some of the more intricate things I’d like to do.
The key data I wanted was the airport codes for the various flights, and geographic data for those airports.
Firstly I found FlightAware.com who will provide all sorts of data from a flight code, but unfortunately the first time I tried making a request to their site using HTTP Client I spotted a comment in the HTML referring visitors to http://flightaware.com/about/termsofuse.rvt which states:
You will only access the FlightAware web site with an interactive web browser and not with any program, collection agent, or “robot” for the purpose of automated retrieval of content.
So I started looking at airline sites. United have a relatively straightforward URL scheme that responds to a GET and returns data that can be scraped. eg:
http://www.ua2go.com/flifo/FlightSummary.do?date=20090201&fltNbr=959
BA and Virgin, on the other hand, both require cookies to be enabled in order to get results from their flight trackers and don’t advertise any other URLs for flight data. Once I’d realised that only one of those three carriers was going to be helpful, I decided not to keep checking airlines.
So, a little frustrated, I tried just typing flight codes into google. And lo and behold… most of them give useful results. It doesn’t get them all, of course, but it’s enough that out of the 838 flight codes in the database, 695 are fully identified. Of those not identified, a number seem to be flights that were diverted to Heathrow but don’t normally go there.
So with some sense of the airports served, I also want to know about the airports themselves. Wikipedia’s pretty good there, with geo data for them all in an easy to capture form. Some, like Heathrow itself, are very easy to find:
http://en.wikipedia.org/wiki/LHR
while others are a bit trickier. But with some manual intervention I was able to get all of that data. The manually produced mappings and the code for pulling in the data can both be found on github.
More updates as time allows…
Ecampaigning Forum: Notes on Open Space sessions
Apr 12th
While my live blogging efforts focussed on the more formal sessions at ecampaigning forum, most of the event’s time and content was spent in groups following the Open Space methodology. The gatherings for people to suggest sessions were instructive in themselves as they gave considerable hints as to the key concerns of ecampaigning practitioners.
How to engage with the big social networking sites, whether to create your own, organising around big events (such as G8 summits and climate conferences) and ways of managing decentralised/coalition campaigns were some of the big themes, but the sessions covered a wide range beyond that such as engaging with young supporters, or older supporters, choosing content management systems, operating on a tight budget, pooling resources/tools and one hastily agreed discussion of twitter. What follows are a few notes on things that struck me.
The twitter session drew a mixture of existing users, aware onlookers, and newcomers. A lot of time was spent exploring existing uses of the site with examples such as teamtibet’s usage to co-ordinate protests around the olympic flame and Downing Street’s account. Most people seemed taken with its potential for short term co-ordination, but many questions arose about its potential for long term campaigning beyond informing core supporters of news updates. Being seemingly the longest-serving twitter user there, it was interesting to hear responses to a tool I’ve quickly come to take for granted
A recurring theme was the adoption of drupal by a number of the big agencies. Most seem keen to contribute code back to the community, along the lines of AI and CivicActions‘ assets module. I’ve mentioned my mixed feelings about drupal before but am hopeful that through events like this we might be able to resolve some of the issues that frustrate me.
I brought up Russell Davies’ 2008 – the year of peak advertising in conversation over breakfast on the first day and that phrase recurred a few times. There’s a general awareness that the last few years have brought lots of opportunities to attract attention by simply being quick to adopt some new “web 2.0″ tool, but that won’t last. It didn’t seem like there was a sustained discussion or much sense of where to go next, but working hard to attain attention has been the life of campaigners for a long time and so perhaps this is just another step in that journey?
There’s clearly a growing sense of how hard it is to influence big summits where the final communique is often planned months in advance. Gatherings of world leaders are a great opportunity for media coverage and to present the “actionable moments” that Ben Brandzel spoke of, but they’re now when the real chance for change occur. It’s vital to find ways to turn the energy around these summits into sustained, directed action after the final communique is published, planning the next steps before the events themselves take place.
In the session on pooling resources and tools a number of questions came up about the ethics of collaborating with big players like google (who have just been on a big outreach programme for their new Google Earth offering for NGOs). The data provided and the tools offered by the likes of Google can be a great boon to charities operating on tight budgets, but at the expense of ceding a lot of control and a lot of attention data (and with providers like facebook there are concerns about things like this). It was obvious that there is some desire to develop open source tools that provide similar tools, but it’s not clear whether the resources are there. Mention was made of open street map and I brought up the theyworkforyou api, and it definitely would have been interesting to have had people who could present on the usage of that; some concerns remain as to how ready those tools are for non-geeky end-users, which would be easy to resolve if someone were to direct the right resources.
I’m looking forward to seeing what other people bring up in their notes on the event, and what themes come out in the ongoing discussion. You can see my photos on flickr, find some content on technorati and check out the conference wiki for more. All my posts on the topic are gathered under the ecf08 tag.
Demos/Google: How Has The Internet Changed British Democracy?
Oct 27th
On Tuesday evening I found myself at a google/demos event, How Has The Internet Changed British Democracy?. Unlike most discussions about the net and democracy I’ve attended, the panel here was very ‘establishment’, consisting of Demos Director Catherine Fieschi , Spectator Editor Matthew D’Ancona, Stephan Shakespeare, the founder of online polling agency YouGov, and BBC Political Editor Nick Robinson. Tom Watson MP was detained in the House of Commons, or he would have joined the panel.
Given the establishment panel it was refreshing to hear a general concensus that while there have been lots of indications of change, the real potential of the net has not yet been realised in British politics. Catherine Fieschi concluded her introductory statements noting that she doesn’t think the internet has changed democracy “yet in our societies, but I think it’s gone some of that way in others.”
A recurring theme, first clearly voiced by Spectator Editor was that we should be looking for a genuinely new form of politics for the internet age, and not simply how the net “facilitates and speeds up” the old forms. There was a general consensus that Gordon Brown’s recently announced “new politics” has (in the words of Catherine Fieschi) “a distinctly 20th century feel” but while many ideas were thrown around, what was naturally lacking was any description of what a 21st century version might look like.
Nick Robinson discussed in some detail how he came to start blogging, first in the form of an “election diary” some years back, and more recently again on the BBC News site. He pointed out that as TV reporter he didn’t get much feedback and that by providing a feedback mechanism the blog had helped his reporting. In particular he pointed to the resignation of former Liberal Democrat leader Charles Kennedy, when there was a sense among some members of the public that reporters had know of Kennedy’s drinking problem and were covering it up. Robinson claimed that he had been unaware of that sense, and that the blog made him aware of it and so able to address it directly.
Robinson was particularly concerned that in an age of on-demand television and increasing narrow-casting it is all too easy for viewers to “opt out” of the political news he believes is vitally important. He sees blogging and other new media as a way to engage a new audience. His eagerness to engage is impressive but I found myself wondering if a better way to handle a situation where people are only watching a narrow selection of news is to try to make sure that reporters work harder to convey the contexts of (and hence the interweaving between) their stories. He hinted towards this in suggestions that he wanted the BBC to make better use of amazon-style recommendation systems for news but I think it bears further development to help people dig deeper into the contexts their interests exist within.
Tom Steinberg of mySociety challenged the panel to discuss a “non-media” aspect of the political process they could see changing in the coming years as a result of the internet and while they shared a focus on the somewhat obvious element of direct accountability (as demonstrated to some extent by mySociety’s projects) the key interest seemed to be in the position for co-creation in the consultative and drafting process. We have tools to gather opinion and to collectively edit documents, can that be harnessed to change the way the government writes Green Papers?
Perhaps the strongest sense I took away was that government needs to wake up to the possibility and need for stronger feedback loops. At the mySociety Disruptive Technologies session earlier in the month Stef Magdalinksi led a discussion that spent quite some time on the need for campaign facilitators to provide feedback loops so that campaigners can see the impact of their actions and all involved can refine their work.
The same possibility exists for government who could be proactive in their responses to petitions especially, as Nick Robinson pointed out, those gathered through the Downing Street Petitions system. Government can use these systems to identify people with specific interests and to supply them with information, whether persuasive or simply educational, to deepen that conversation. Similarly people can be identified to receive alerts about consultations, to be invited to discussions, and so on. It may be possible for expertise and interest at the grassroots to more effectively bubble up into policy making.
It would be naive to think that major change that will happen quickly or withour pressure, but the possibilities are enormous and this session showed that there are people close to the policy making process who are thinking seriously about the possibilities.