Django Powered

A short break from blogging ends with a new site design. As with all these things their will no doubt be a few kinks still to work out and I’ll be adding to the design a little over the coming months. The main reason for all this change? A move to a custom CMS build using Django. This was something of an excuse to play around with Django outside work and I’m pretty happy with the results. I have all the bits of wordpress I actually used, plus a few bits I didn’t have before. More importantly I have something I want to hack on. Wordpress is a kick ass blogging tool, but keeping it updated or adding new features never seemed much like fun.

It seems like it’s the week for Django related site launches in the UK. We released the new Capital Radio site on the world last week and Nat beat me by a day or so with her new Django powered site. Look for more in the future too I would wager.

I have a whole range of posts brewing about what I found out along the way, both building a personal blog and building a large site with a big team. Keeping up with the bleeding edge of Django ahead of the 1.0 release took some doing (I’m running the latest Trunk release here at the moment). Deciding to use a combination of Spawning and Nginx for serving is a nice break from Apache as well. But that is all for later when I have a little more time.

Of Hacking, Continuous Integration and Django

I’ve not written anything here for a good few weeks, my tweeting has slowed down some and I’m behind on my feed reading. I’m going to blame the new job and the daily commute from Cambridge to London I think. I’ve definitely not been any less busy that usual:

Hacking

We had an internal hackday on Thursday and Friday last week where lots of us over at GCap/Global downed tools and build cools stuff for a couple of days. This is exactly the sort of reason I took the job in London - for the opportunity to build interesting things quickly with other smart people. I got to play around with an event driven, music orientated, API hack and a more useful but less sexy documentation hack. Yes, I said documentation hack. I might be able to release the latter all being well but I need to finish it off first and kick the tyres on some internal projects

Django

As a development team we’re using Django for everything which is proving to be huge fun. It’s a decent size project and we’re pushing Django (and in particular new-forms admin) in interesting ways. Having worked previously with PHP, ASP.NET and Rails I’m loving lots of bits of Django. I mentioned the template system before but their are lots of other things to appreciate. Some of this just comes from working with people like Simon and Rob who know Django pretty well. Some of it just from being able to write Python every day.

Continuous Integration

When not working on Django, or writing HTML, CSS or Javascipt (now mainly using JQuery), I’ve been busy pushing the benefits of Continuous Integration. As someone who is actually pretty bad at writing unit tests I like the process of working with Cruise Control as a gatekeeper. I also like automation in general so have been busy with Ant scripts and some Twill scripts as well for more functional testing. Django’s test suite is pretty nifty and easy to use. The official documentation is a pretty good starting point but I’m still on the lookout for some more in depth best practices articles.

So, between writing lots of code I’ve had less time to write words other than internal documentation. I’m finding the change pretty refreshing at the moment but want to keep up with writing every now and again. Who knows how that will work out? I have a feeling that I might blog more geeky code related stuff but time will see how that plays out.

Where are the Rock Star Web Project Managers?

In maybe a more constructive manner than yesterday I started wondering where the rock star web project managers hang out? I think we’re all aware of something of a celebrity culture within web circles. Their are a hardcore of people who’s blogs, books and conference appearances we’ve all seen several times over. And in the main I think this has had a positive effect on everyone involved. People like Jeremy, Molly and Simon have at different times acted as pretty useful barometers and yard sticks for lots of people. But these people are invariably designers and developers - not product managers or project managers.

The only person I can think of who has talked a little about the topic of project management is Meri with her new book Principles of Project Management. The topic occasionally comes up in conversation, or is mentioned by the designers and developers noted above. And their are lots of blogs (more often by developers it seems) about Agile, XP and Scrumm. But what about the practice of web product management? You can point to countless blogs written by designers and developers at the likes of last.fm, flickr and Yahoo. But where are the managers?

So, my question to you is: do you know of any great web product or project managers that blog about the discipline?

@media: Designers Vs Developers

Another successful @media conference comes to a close and as usual interesting things were said and hopefully everyone learned something. As usual I have a few more things on my must find time to play with list. More on those if they happen.

But one part of the conference I felt needed addressing straight away was the first days panel. The Hot Topics panels bring together a few of the days speakers to answer questions posed by the audience. This year each day had it’s own panel discussion, with the first days session having a design theme. So far so good, and before I getting going I have to say I think Jeffrey Veen did a sterling job of prodding and prompting the session along. The rest of the panel composed of Andy Clarke, Dan Rubin, Bronwyn Jones and Indi Young.

The panel fielded a few interesting questions (and The Beatles in-joke was highly entertaining) but the whole thing took an odd turn part way through - odd in the sense that everything suddenly became very anti-developer.

Andy Clarke seemed to be the ring leader here saying things like (I’m paraphrasing here) “I hate daily stand-ups, we should do Agile”. Dan Rubin was involved too, seemingly saying all developers prefer the rigid nine to five because they’re logical people, while designers want to be able to work whenever they choose. This and other comments caused a few good friends and web developers on the front row to literally shake their fists in anger. The audience got involved too, cheering the anti-developer or anti-engineer statements on. In my mind damage had just been done to out our industry.

Buidling web sites or applications is a pretty multidisciplinary exercise, and in any team environment good communication is often the difference between executing well and failing badly. Conferences like @media are a great place to come together with people from different disciplines and similar jobs in different organisations and share stories. I’m not saying either Dan or Andy haven’t had bad experiences with developers or stand-ups. I’m not saying bad individuals or bad process don’t exist (I have scars from both). I’m saying that generalising these experiences and then promoting them to impressionable designers is dangerous.

I have the good fortune of working with a great designer, Alex Lee, at GCap at the moment. He’s involved in our stand-up meetings every morning and without him there we would waste untold amounts of time and effort. If he turned up on monday and said “I’m not going to come to stand-ups anymore, Andy Clarke says they aren’t cool”. We have a massive problem. My fear is this is exactly what’s going to happen somewhere tomorrow.

I’m something of a hybrid; I’ve run the whole gamut of design and development activities during my career to-date (I’ve even touched on project management) and have worked in small and medium sized agencies, as an independent freelancer and now in a decent sized in-house team. All of these roles pose different communication challenges which require different solutions. What works for three people in a distributed team doesn’t work for fifteen people working in-house. Some types and sizes of team work best with stand-ups, others work best being in the same small room together. Sometimes you need a centralised audit trail of everything that has happened on a project, other times it’s overkill. Somethings everyone has to be working on the project at the same time, at other times as long as tasks get done it really doesn’t matter.

I’m not saying I like all of those working conditions - I’m saying that if you understand where they work best you can work in the environment that suits you. Andy Clarke runs a boutique agency doing high quality work, working with a very small team where individual flexibility and minimal process is definitely the best approach. I now work in a cross disciplinary team of maybe sixty people and it’s a different ball game completely.

This isn’t an attack on Andy, or Dan, or even an anti-designer rant. The web is a young industry filled with young people. We’re learning as we go and stealing what we can from other disciplines. Also not that many of us really like the idea of project management, never mind the reality. Agile methods like Scrumm are hot at the moment but that’s not to say we won’t find something that better suites our discipline in the future. Iterative development, while often annoying to designers, appears to be producing better results. If designers, and developers, want to change how we work together for the better, then get interested in project management and lets have a discussion in public. If you want to complain about individual experiences then that’s why you have a blog.

Newcastle Web Week

The North East is stealing a march on London this week with all sorts going on for those who like a real world get together. Although not quite as busy as London Web Week next week we have the Thinking Digital conference kicking off tomorrow, followed by a Geek Dinner on Friday night and then BarCampNorthEast over the weekend. I’m even planning on popping out tonight with a few people in town for the festivities. As if fate somehow got involved the weather is also pretty darn nice today too.

If you didn’t already know about these events and you’re in the area their just might still be the chance to come along. We have a few places left for barcamp in particular, you just need to register over on eventwax. And the Geek Dinner on friday night at The Pitcher and Piano is open to all, at least until it fills up.

It really is great to see thing happening at this end of the country, and a real shame I’m leaving the area so soon afterwards. I really hope the local community gets behind all these events and they act as a catalyst for interesting things. I’ve always been of the mind that if you can get enough smart, interested people together in one place then good things will happen. It’s why I enjoy going along to conferences, hackdays and barcamps. It’s also one of the main reasons I joined GCap.

Once I’ve recovered from the next two weeks (which may take a while as next week I’m in London for @media, BarCampLondon4 and anything else I can get along to) I’ll hopefully write more about trying to organise a barcamp and, if I really get time to think, about knowledge workers outside and inside London.

But in the meantime leave a comment if you’re going to be around in Newcastle over the next week. It should be a great opportunity to meet new and old friends alike.

Design Strategies for a Distributed Web

I just finished my presentation on the last day of the Xtech conference in Dublin. I’d chosen to ramble on about the advantages, problems and a few solutions of building applications atop of lots of APIs. The presentation is now up on slideshare at slideshare.net/garethr/design-strategies-for-a-distributed-web/.

Lots of interesting conversations have been occurring all week and a few people have mentioned a near barcamp feel at times. It’s a pretty small, clued-up, technical audience and as always some of the best bits have been conversations in the corridors. Jeremy has been live blogging many of the talks which should be a worth a read later on. Although with three tracks most of the time that’s only some of the great talks on offer.

Moving to London, Working For GCAP

On departing pubstandards in London a few weeks ago Dirk Ginader said something along the lines of:

According to Dopplr you’ve spent 40 odd days in London in the last 8 months. Don’t you think you might as well just move here?

I’m not saying that had anything to do with it, but I’m happy to announce I’m moving to London in a permanent manner very very soon. I’m taking up a job at GCAP, working with quite a few people I know from various web goings on; Hi to Simon, Ross, Brad, Ed and Stuart amongst others.

Freelancing for the last year has been great. I’d been able to work on the variety of projects I’d wanted after leaving my previous job, and I’ve had more time to myself when I wanted to get involved with other projects. WaSP, SXSW, Highland Fling, BarCamp, Thinking Digital to name a few things I’ve been up to in the past 6 months. But something about working with a super smart team of people won me over. The fact that I’ll get to focus on Python and Django is an added bonus.

Patrick apparently closed the book on this happening some time ago. I’m just hoping someone got the date about right and won the money. I’m sure I’ll write more about the whole thing as time permits and lots of specific topics spring to mind. I have a number of events in the next month to get along to before all this happens however; if you’re along at Xtech, Thinking Digital, London Web Week, BarCamp NorthEast, BarCamp London 4 or @media then remember to say hi. My addiction to attending web events can only get worse with me being nearer London I fear.

A Really Simple Capistrano Recipe

Build scripts. I think I’ve probably mentioned before; everyone knows they should but not many actually do. I’m not talking about your large in-house development teams or your sexy web startups; both probably have a good-enough build process for different reasons. But smaller teams, web design agencies or freelancers often rely on FTP and a prayer. One of the problems is definitely finding suitable documentation. It’s not that their isn’t a lot of good quality documentation - it’s the suitability of it for those with only a passing interest and a limited systems administration experience.

Capistrano is a tool for automating tasks on remote servers. It can be used for all sorts of useful things but we’ll concentrate on it’s utility for web site deployment. It’s been popular with the Rails crowd for a while and is written in Ruby, but it can be used for deploying sites in Python, PHP, Django, whatever you happen to be using at the time.

You’ll first need to install Ruby and Capistrano. The Capistrano site has installation instructions which should get you up and running. You’ll need to be able to use a console on your machine and have a passing acquaintance with your webserver. I’ve also assumed that you’re using source control, mainly because if you’re not you should start their. Automation becomes much more useful and practical on the back of a good source control system.

Right, lets get started. Create a file called capfile somewhere on your local machine. The following two code snippets should go in that file.

<code>set :username, "<username>"
set :host, "<host>"
set :path, "<path>"
set :restart, "/etc/init.d/apache2 restart"
set :checkout, "svn export --force"
set :repo, "<repository address>"</code>

Just replace the variables (marked with angle brackets) with your own values. The path is the full path on the remote server where you want to deploy your files to. Note that I’ve abstracted out the webserver restart command and the source control checkout command. The above example values assume you’re using apache and subversion (and want to do a subversion export). It should be easy enough to change these for your preferred combination of source control system and webserver.

<code>desc "Remote checkout and restart of webserver"
task :deploy, :hosts => "#{username}@#{host}" do
  run "#{checkout} #{repo} #{path}; #{restart}"
end</code>

If your platform of choice doesn’t require a server restart (say PHP) you can always remove the ; #{restart} portion of the command.

Now we’re all set up just bring up a console in the folder containing the capfile and run the following command:

<code>cap deploy</code>

After prompting for a password to access the remote machine this should display the output from the commands being run. Depending on the size of the repository you’re checking out

Of course for most projects you would want to expand this basic recipe. Maybe you have database updates to run or need to deploy to multiple machines at once? You’ll might have machine specific configuration files which aren’t in the source control system. You will also probably want to be able to revert back if something goes wrong and you might want to only change files that have been altered. Capistrano has lots of powerful features built in that will let you do just that, and once you get started with this sort of build scripting you’ll find lots of areas to improve. And you’ll find lots of good tutorials online that will take you further.

The difficult part with project automation and build scripts is often the getting started in the first place; especially if you’re not the server admin type. Unfortunately the documentation and articles written on the subject tend to be somewhat arcane and aimed at the more hardcore developer. Hopefully this simple, usable, example convinces someone else to give it a try.

Resourceful Vs Hackable Search URLs

I often end up pondering URL design given a moment and something that keeps coming up is hackable search queries. But first a very quick primer on the idea of resourceful design.

REST is a series of architectural principals more than a defined architecture. The Resource Orientated Architecture builds on those ideas with a series of concrete guidelines put down by Sam Ruby for designing RESTful systems. The simple version is that you try to design your system around resources represented by URLs.

I’d thoroughly recommend reading RESTful web services whenever you get a moment as this subject is covered in detail.

Flickr isn’t a truly resourceful design but it does have many of the hallmarks. For example the URL that describes me is at:

<code>http://flickr.com/people/garethr</code>

When it comes to searching on flickr we have:

<code>http://flickr.com/search/?w=all&q=pubstandards&m=text</code>

The pattern of using a query string argument named q to pass a search string is pretty common. One of the guidelines mentioned as part of the ROA discussed query strings:

Query string parameters are appropriate if they are inputs to a Resource which is an algorithm. Otherwise, these values should be moved into the URI.

Search is definitely algorithmic. Now you could maybe argue that a global search should be done on the root of a site, with specific resource searches on the resource in questions. eg. /people/?q=. This would likely work fine but require some behind the scenes complexity as well as probably not being as obvious to the end user. Global searches are in many cases much more common that restricted searches and even in resourceful designs the root of the site (ie. the home page) might not act as a list of available site resources. A notable exception might would have to be the excellent BBC Programmes site which is basically one big semantic catalogue.

But we have another kind of URL that’s cropping up for search results, one that treats the URL much more like a fundamental part of the user interface for search. An example from a site I use all the time is The Accessible UK Train Timetable which allows for URLs like the following:

<code>http://traintimes.org.uk/newcastle/london</code>

You can basically squash all the search parameters from the form into the URL, meaning you can easily bookmark search results. Note however the actual content is likely to change. The above example for instance would use the current time to get a list of trains from Newcastle to London. In an hours time the results will be different.

Another good example would be the new Yahoo! UK TV listings which has URLs like these:

<code>http://uk.tv.yahoo.com/listings/bbc-two/2008-04-17/</code>

<code>http://uk.tv.yahoo.com/listings/itv1/2008-04-17/21-00/</code>

Again this is really a search query, or at least specifying the time and date is. In some ways it’s the return of the command line - allowing searches to be run very quickly from a textual interface.

Now, both these approaches treat URLs with the respect they deserve. But they do have the potential to clash somewhere in the middle if care isn’t taken. The Accessible Train Times site is a single purpose site which just does searches while BBC Programmes does feature a search engine but it’s just the global BBC search which takes you off site. And if that wasn’t enough potential competition then a question raised by Simon at The Highland Fling regarding URL design and the search engine optimisation crowd go me thinking too. From being a somewhat niche area of interest URLs might just become a sort after part of a good website design - fought over by the varying disciplines of modern web design and development.

DSLs for HTML and CSS - The Future, or Just Plain Wrong?

After my previous post about Django and the web standards community a number of the comments picked up on the fact I mentioned haml under the title Other Craziness. Ok, so I was being a little over-poetic but I decided this warranted a closer look.

Haml is a markup language that’s used to cleanly and simply describe the XHTML of any web document without the use of inline code. Haml functions as a replacement for inline page templating systems such as PHP, ASP, and ERB, the templating language used in most Ruby on Rails applications.

A quick example should help. The following haml code…

<code>%div.special#primary Hello, World!</code>

…is compiled to the following HTML:

<code><div class="special" id="primary">
  Hello, World!
</div></code>

Depending on your application this could be at runtime or as part of a build step. Although primarily associated with Rails because haml is also available as a command line utility you could in theory use it with any framework or language.

My initial take on this was to call haml an abstraction of HTML but Nathan Weizenbaum, one of haml’s developers, put me straight:

Haml doesn’t really abstract HTML. Not in the same sense that, say, Rails helpers do. Since Haml has a one-to-one mapping to HTML, I view it more as an alternate syntax for HTML than an abstraction.

Lots more examples for anyone interested can be found on the haml documentation site.

After some research and some playing around with the command line version of the haml engine I decided to see what Twitter thought about the situation. Little did I realise what I was letting myself in for:

Tom Morris kicked things off:

bq.:http://twitter.com/tommorris/statuses/784959480 I’m not sure why everyone insists on clumsily reinventing HTML every few weeks (eg. wiki syntaxes, of which there are hundreds)

Brad Wright thought:

bq.:http://twitter.com/intranation/statuses/784963589 Sass is just stupid, since you’re basically writing exactly the same CSS just in shitty YAML style.

And followed with:

What’s the point of abstracting HTML? It’s not that hard

And Mark Norman Francis chipped in with:

EVIL. And not in a good, kittenish way.

A few people echoed Ross Bruniges sentiment that haml and sass are just:

bq.:http://twitter.com/rossbruniges/statuses/784954916 front-end code for those who don’t really care all that much about it and would rather create databases

I have to admit to this being my initial reaction on hearing about and looking at haml, hence the remark from the previous post. But that’s not to say everyone was negative.

Mike Stenhouse stepped in and said:

bq.:http://twitter.com/mikesten/statuses/784952890 Love haml - it’s all I use these days. More readable, dynamic and hackable. Took me a while to come around to it though…

Some of the comments were about how the use of haml might alter the dynamic of a team, to either positive or negative effect - depending on your point of view.

Mark Ng saw it as a cunning way of getting rid of the front-end guy.

bq.:http://twitter.com/markng/statuses/784952935 at first, they look elegant. Then it becomes obvious how they remove designers from the process of making markup.

Where as Olly Hodgson say it maybe as a route to get the dyed in the wool back-end writing decent markup.

bq.:http://twitter.com/OllyHodgson/statuses/784957324 They look interesting. With proper training it might be a good way to get back-end programmers creating decent HTML (shock horror!)

At present haml is very much pitched at the Rails community from whence it came. Many of the examples demonstrate benefits compared to ERB, and haml is of course written in Ruby and available as a Rails plugin. Being perceived as part of that community has obvious benefits but also some subtle costs, in particular regarding those people that don’t like Rails very much.

I’m not really convinced of the benefits in all fairness. The something else to learn barrier only gets magnified when working within a team environment. You now have to train new recruits of whatever skill level in another syntax. One that they might be able to write quickly enough but can they understand from the briefest of glances at a template? HTML might not be great here but it is familiar to everyone. Their is also the programmers abstraction. What if I can’t get the markup I want out the other side of the black box? Yes it’s open source so I can hack the box open but that causes even more problems. And while I quite like meaningful whitespace (for instance in Python), in templates which fail if it’s not quite right I see a major problem for those whom a text editor is not their best friend.

I am however interested to see whether the problems people have with haml are with haml in particular or with the overall approach of alternative syntax’s for HTML and CSS. Are DSLs (Domain Specific Languages) needed for CSS and HTML? and if so is this a possible avenue for innovation on top of slow moving standards?