With its complex yet penetrating arcs and careful unravelling of a fictionalised but well-rooted version of Baltimore, The Wire quickly became my favourite television show of recent years. So following its recent conclusion I’ve naturally been devouring every article I can find, not quite ready to let go.

The following paragraphs from a piece by executive producer David Simon struck me as an unintended example of why newspapers (his previous profession) have generally fared so poorly over the past few years, and a reminder of how easy it can be to miss the possibilities new technologies :

It would not have been easy for a veteran police reporter to pull all the police reports in the Southwestern District and find out just how robberies fell so dramatically, to track each individual report through staff review and find out how many were unfounded and for what reason, or to develop a stationhouse source who could tell you about how many reports went unwritten on the major’s orders, or even further — to talk to people in that district who tried to report armed robberies and instead found themselves threatened with warrant checks or accused of drug involvement or otherwise intimidated into dropping the matter.

These are areas that the technologies we use on the web should have helped papers become more effective. Improved scraping, syndication, and the tools that make sites like EveryBlock possible make it easier for people to track the data and offer new ways to identify potential grounds for investigation. The technology doesn’t remove the need for beat reporters, but it might let them find the leads more quickly, and help newspapers find the story. Instead, too many have simply looked to the short-term, aiming to raise ad revenue by becoming increasingly generic.

The story is the same outside the confines of the newspaper world. When approaching new web projects we need to make sure we’re not simply being reactive, but proactive. How can we take what we do well, and do it better?

Recommend this post:

The Economist is the latest organisation to announce it will be putting its archive online. Stretching back to 1843, that’s a huge mass of data coming online, even if (for now) it sits behind a very expensive pay-wall. The news comes three days after The Guardian and Observer announced that:

The first phase of the Guardian News & Media archive, containing the Guardian from 1821 to 1975 and The Observer from 1900 to 1975, will launch on November 3.

It will contain exact replicas of the original newspapers, both as full pages and individual articles. and will be fully searchable and viewable at guardian.co.uk/archive.

Readers will be offered free 24-hour access during November, but after this trial period charging will be introduced.

It’s not surprising that both services will carry charges. Putting all that material online is a costly business and there’s not yet much information to explain what level of traffic it’ll attract. Chances are the majority of people using these services will be based in academic institutions that will purchase licenses. For anyone with an academic account, this will be hugely useful.

It’s also another reason the web development community needs to take very seriously an issue the question of historical context on the web that Gavin has been talking about for a while now. Whilst these are closed archives most access will be through their own specific tools, but when they (inevitably, in my opinon) open up that’s an awful lot of data with very specific provenance that our search tools will have to mediate. Hopefully the developers working on these projects will be able to share their experiences and begin to establish best practice for mediating large, web-based historical archives.

What would be doubly fascinating would be to see what sort of social layers could emerge on top of these archives. Obviously if the archives were open that would provide a wealth of historical references that are URL addressable, making hypertext documents still richer. But could I search and annotate the data such that any mention of my ancestors is easily identifiable and tie that into my family tree? Could specific groups organise around topics within the archives, with, say, urban spelunkers gravitating to news of closed or buried buildings, or local communities able to map out their history in much more detail?

One thing is for sure, as more and more historical archives open up we’re all going to need to be much more imaginative about how we interact with online material. The well rehearsed “comments on news articles” model only works for recent occurrences or recently unearthed information, and there’s so much more we could do.

Recommend this post: