App Engine Remote API calls

Not sure how I missed this but apparently App Engine (as of 1.1.9) supports remote access to your live data store. This means you can create administration applications more easily by running them locally, rather than within the limitations of the live platform. You can even run a local python prompt with access to your live datastore which is pretty neat.

Services Vs Applications: Does Rails Encourage SOA Better Than Django?

Building larger applications tends to mean splitting your codebase up some how into manageable chunks. I’m quite interested in what I see as different approaches in the Rails and Django communities:

Django tends to recommend building Reusable Apps and we have sites like Django Pluggables to catalog what’s available. You then grab a few of these applications from the web or write your own, add them run them all together as part of a single application. Pinax is probabaly the poster child for this approach. The 0.5.1 release for instance appears to have 41 individual reusable apps, many written by other people and projects.

The Rails community tends to talk more about RESTful service orientated architectures, with things like ActiveResource making this sort of thing easier. So rather than your manageable chunks being within your application they’re separate instances in their own right.

I’d be interested in hearing from more people about their experiences, in particular if you’ve gone against the grain so to speak.


RewiredState was awesome. 100 or so geeks plus a smattering of government types gathered in the shiny new Guardian offices in Kings Cross on Saturday to hack (the Government).

Some events like this are more productive than others, and the end of day demos included some realy impressive stuff. See for your self on the projects page

My own little project even won a prize (an invisible bottle of Champagne no less). My complain was that if you want to report an issue about the over 7000 government websites you have to do it per site. All of the sites do their own thing, which might be a nice contact form, maybe an email address or in some cases a postal address hidden on a page that’s not linked to from anywhere.

My solution was pretty simple - a centralised issue reporting and navigating tool. So you go along to the site you have an issue with and hit a bookmarklet (or a badge on the site if the site in question have been nice enough to add one). You fill in a very simple form which appears before your eyes and everything is tracked on a nice shiny website.

The advantage of all that is transparency. You can see which sites people have issue with, and also ideally which issues get addresses or at least acknowledged by the support staff for the site in question. The hack had comments so others could follow up on individual issues. It would be simple enough to have league tables and the like as well, or add tagging for a little bit of categorisation - I did only have a few hours though.

screenshot of FeedbackGov website

After a few nice comments I’m going to have to finish it off and get it up somewhere I think. All I really need to do is clean up the bookmarklet code (which was a hack in more ways than one) and add a bit of sanity checking to data entered into the system.

A massive well done to everyone involved is in order as well, especially James who I remember talking to the idea about ages ago. Congratulations all - and hopefully be back next year.

Content to Markup ratio bookmarklet

Stoyan Stefanov just released an excellent little bookmarklet to calculate a content to markup ratio

It’s interesting browsing around a few sites and comparing ratios:

App Engine for Python Developers

I gave a short presentation about App Engine last night at the Cambridge Python User Group and as always it’s available on Slideshare.

I find App Engine a great way of just writing (Python) code and putting it online quickly, and the presentation was aimed at being something of a fanboy ode. During questions afterwards we talked about some of the limitations and cases where App Engine isn’t as well suited, and at least a few people were heading off planning on having a play.

The event was good fun and I picked up a few Python related tips and tricks from the other presentations and general conversation. Thanks for everyone who organised and attended the event and to RealVNC for hosting the talks.

Quick dmigrations Update

I’ve finally been getting round to doing a bit of work on dmigrations, the Python based database migration tool we developed at work.

dmigrations offers a simple but flexible way of managing changes to the database in your Django projects. It is a replacement for Django’s built in syncdb command.

I still have some work to do to merge in a number of modifications we’ve been running internally for a while but I’ve triaged the issues list that had been left alone for too long and merged in a fantastic collection of patches from a few people including Alexander Robbins and Chris Lamb. Thanks hugely to everyone for these.

Everything is available in trunk in SVN at the moment ahead of me cleaning a few thinks up and updating the download version. I’ve added a file as well ahead of hopefully adding it to the Python Package Index.

I obviously wasn’t the only one who had decided to do some work on the project in the last couple of days. msaelices has just submitted a patch which adds a PostgreSQL backend. I don’t tend to use PostgreSQL personally so if anyone does want to have a look at running dmigrations with the patch I’d love to hear about it.

HTTP Debugging Server in Django

I’ve been busy building and playing with various HTTP clients recently, mainly due to more playing with RESTful web services. I took a couple of hours out to build something to make my life easier - namely a very simple logging HTTP server in Django.

All the application does is accepts HTTP requests and log the results to a file. I’ve been using it to make sure the requests I’m sending from elsewhere are correct, before pointing the client at a web service that actually does something useful. It supports POST, GET, PUT, DELETE, etc. So far so simple.

screenshot showing us tailing the logs

I ended up using Django mainly because I did most of the work on the train, and I know my way around it pretty well by now. I did originally thing it might have been better to use something simpler but I did end up learning a few new tricks that I’ll use for future projects.

  • In theory Django doesn’t need a database. While this is true for the most part, the in-built test runner does require a database. One thing I’ve talked about before is the usefulness of creating your own custom test runner. If anyone else runs into this specific problem have a look at the file.
  • The logging is handled via Python’s inbuilt logging module. It has some nifty features (like log rotation, maximum file sizes and different logging levels) but is a little fiddly to get working correctly. Tales abound of log messages appearing multiple times, or permission errors causing problems. At least for this project I think I have everything working. I’ll try using this module on a couple of other projects and maybe spin it off into a small reusable app.

For something that took only a few hours to write it was both good fun and a useful learning tool. And Django proved itself more useful that I though for smaller services like this.

Startup Hubs in the UK - Where To Put One?

I always enjoy reading things by Paul Graham, even if I’d like to be able to disagree with his conclusions on occasion. It’s not just that I love Hackers and Painters, but also he often says things like the following that make me smile:

hackers are much more constrained by gentlemen’s agreements than regulations. If they shake your hand on a promise, they’ll keep it. But show them a lock and their first thought is how to pick it.

His latest short essay concerns startup hubs again, something that I’m actually pretty interested in, end up talking to people about, but hardly ever writing anything on. Mike over at Techcrunch Uk elicited some good comments last year with a post on the subject. As usual Graham focuses on the US, but what about translating his idea of buying a Startup Hub and applying it to the UK? He sets several criteria for success:

  • Do you have good weather?
  • Do people live downtown, or have they abandoned the center for the suburbs?
  • Would the city be described as “hip” and “tolerant,” or as reflecting “traditional values?”
  • Are there good universities nearby? Are there walkable neighborhoods?
  • Would nerds feel at home?

Lets assume for the moment that the idea would work. Take £15-20million and invest in 30 or so well chosen startups on the condition they move into town. The question is then: where in the UK would this be most likely to work? And are their any UK specific criteria that would increase the success rate?

Some people might be thinking “but no UK city is going to invest £20million in a scheme to encourage startups”. Leeds is doing just that. Or rather Yorkshire Forward are putting up £35million along with Leeds University putting up £31milion. What are they doing with all that money? They are going to build a big building with it - which is exactly what Graham says won’t work.

GitHub Changelog

It turns out the guys at GitHub publish the commit messages from their work on GitHub itself. If you really have to keep up you can even subscribe to an Atom feed.

OK, so this might not be hugely useful. But it is funny, and a just a nice sign of a small company being open in an interesting way. A few of my favourites:

mmm pizza… guard against bad SHAs in Walker#commit

really need to get a handle on this insane testing setup

I guess ff3 linux is the worst browser ever made

dont care

Epydoc Ant task

I quite like Epydoc for generating Python API documentation, even if the interface looks a little dated and could do with a lick of paint.

For most project I use an Ant build script to generate documentation when needed. You could wrap the basic commands in a make file or a bash script if you prefer that sort of thing though. The only trick is to make sure you have everything you need on your Python Path. In Django projects, or App Engine projects, you’ll probably find the default runner script plays with the path somewhat. The following example adds the current directory to the path along with an ext directory where I stash external modules, which should see it run cleanly without any import errors.