Notes
Coderack: Rack Middleware Directory
Jun 13th
With Rails 3 inching ever closer, there’s likely to be a good bit of attention on Rack Middleware over the coming months. It’s there as an option in Rails 2.3, but with Rails 3 it’ll get the attention it deserves.
Working on a few Rails 3 apps, I’ve been very pleased to find Coderack (via Robert Brook on twitter, I think). It’s a directory of middleware, packed with interesting tools.
The one discovery I’m so far using is StaticFallback, which “bounces or redirects requests to missing static files”. So you can sync up your development environment with your production database, and don’t have to worry about also pulling down all the assets that may have been uploaded to it. Very handy!
It’d just be nice if CodeRack could add an RSS feed of new middleware submissions…
JangoMail, lackadaisical security, and a workaround
Jun 9th
A client recently asked me to integrate their site with the JangoMail mass mailing system. I wanted to keep them happy so agreed to investigate, but was horrified by what I saw in the JangoMail API documentation.
JangoMail appears to be optimised for those with existing databases of email addresses they want to maintain and contact. For those wanting to keep those databases in sync they offer a script you can download and install on your server that they can call with details of various actions (user unsubscribed, user clicked link, job completed, etc) as well as to extract the list of email addresses they should send a given campaign to. So far, so good.
The problem is in the implementation. Once you have downloaded and installed the script on your server, they ask you for your database credentials and then send these, along with an SQL query, over HTTP (or HTTPS) for the script to execute. In the FAQ attached to their blog entry on the topic the obvious question “Is this method of connecting to my data secure?” is asked, with the response:
Yes. It is inherently secure if you opt to have JangoMail connect over https instead of http. It can be additionally secured by restricting the range of IP addresses allowed to connect to the custom script file. JangoMail’s range of IP addresses are: 209.173.141.193 – 209.173.141.255″
That answer is far from satisfactory. I refuse to give a third party my database credentials, still less to to execute arbitrary SQL received over an HTTP request, even if that is SSL and includes a password.
Surely if they want to keep the steps for the user to a minimum they could still provide an interface that takes credentials (via SSL)–along with appropriate other details like “enforce SSL?”–and generates a PHP script that contains those credentials embedded within it, along with code to generate the SQL from a set of parameters? It’s not a hard thing to do (witness the way wordpress/drupal/etc will generate a config file for you — it’s the same thing). That way JangoMail can take responsibility for making sure that the credentials are only ever sent a small number of times (and via SSL), and the endpoint can contain appropriate validation.
In this case I decided to take matters into my own hands and write a sane script to receive their input. There’s nothing forcing you to put genuine database credentials into their form, so instead I used those fields to provide a username and password I’d use to authenticate their request. In each of the boxes to enter the SQL for your queries I entered some code to generate JSON containing the relevant details.
With that done it’s a relatively trivial matter to parse the JSON, do any validation you may want to do and update your database accordingly. A rough-and-ready (and barely tested) script that does just that can be found in this gist. A perfectly satisfactory (if slightly laborious) solution for any competent web developer.
But of course JangoMail’s target market isn’t competent web developers. They’re clearly trying to target a general audience, and for that audience their lackadaisical approach to security is indefensible.
Week 140
May 30th
It’s been a week of tying up loose ends in an effort to get invoices out the door and perhaps have a focussed start to the new month.
The next step of the infrastructure for JLL is in place, a feasibility study for a document processing system (and associated quote) are being reviewed by a new client, I’m wrapping up a report on some work I’ve been doing for Eastside Educational Trust, a new site for a music industry client is awaiting some DNS amendments on the client’s end, and another project is with a client so they can consider what direction they want to take it in next.
I guess that’s why I’ve been feeling pretty tired this week. This Bank Holiday’s a welcome one!
Character encodings, Rails 3 and Ruby 1.9.2
May 21st
In a lengthy blog post detailing many of the intricacies and some of the politics relating to character encodings in Ruby, Yehuda Katz has a few paragraphs that left me more than a little excited:
The most common scenario where you can see this issue is when the user pastes in content from Microsoft Word, and it makes it into the database and back out again as gibberish.
After a lot of research, I have discovered several hacks that, together, should completely solve this problem. I am still testing the solution, but I believe we should be able to completely solve this problem in Rails. By Rails 3.0 final, Rails application should be able to reliably assume that POSTed form data comes in as UTF-8.
and
When using Rails 3.0 with Ruby 1.9.2-final, you will generally not have to care about encodings.
If it does indeed work out that way, that’s a whole category of bug reports I’ll finally be able to say goodbye to.
Week 138
May 14th
Another week, another chance to reflect on where it went.
The big news of the week was the launch of the Ninja Tune XX website mentioned below. I’m still very pleased with how much we got done with such a short lead time. With quite a bit of attention coming to the site via twitter, facebook and a variety of music blogs (including pitchfork), I’ve been keeping a close eye on traffic and resource consumption. We had an early memory spike, which seems to have been resolved by removing the dependency on the RedCloth gem (I didn’t trace any specific problem to the gem, I just realised we didn’t need it any more and was pleased by the result), and a slight issue due to a mis-match between the versions of mongoid and bson_ext we were using, but those were easy fixes and the site continues to move along nicely.
Other than that, I spent quite a bit of time doing something I’ve not done for many years: watching a virus scanner work its way through my system. I’m working with a client who are hosting their apps internally and so need VPN access to update them, but the sysadmin wouldn’t grant that until I performed that scan. Naturally everything went smoothly other than that my computer was painfully slow for six hours!
There was a very helpful day with Ben and James working through next steps for the James Lind Library. We desparately needed to step back and take a long view at the site’s content, and were finally in a position to do that. After some drawing on the whiteboard, and some more in OmniGraffle we’ve got some ideas that I’m really looking forward to presenting to the client and moving on to implement. The more time I spend working on the backend for the site and immersed in the data, the more I want to get our stamp on it and open it up so that some of the wonderful old (really old) content can come to the fore.
We’ve entered another of those periods where there are lots of small projects to juggle, and lots of prospects, but nothing quite solidifying it. There’s some consideration of how best to land some longer term projects that might provide a little more stability as there are a couple of products that I’d really like to complete but it’s providing very hard to schedule them around lots of moving pieces. And I’m finally getting round to talking to solicitors about some sample contracts, a task I’ve been putting off for far too long…