PHP Asset Packager

Performance used to be something other people thought about. If you were working on a high traffic site for a large company, chances are they would throw inordinate amounts of expensive hardware at the problem. If you had a personal site only if you got really popular would you need more than a shared host. But the number of web applications being launched by small companies or individuals from their bedrooms is raising the awareness of the importance of performant websites.

Credit to YAHOO! here. Google and Amazon might be the first companies that spring to mine when it comes to thinking about infrastructure but YAHOO! have been publishing lots of useful, practical and interesting information in this area. Steve Souders, the Chief Performance YAHOO! has a new book going into detail about YAHOO!s performance rules. The recent release of the YSlow extension and another upcoming book from Stuart Colville and Ed Eliot cap things off nicely.

According to YAHOO!

Combined files are a way to reduce the number of HTTP requests by combining all scripts into a single script, and similarly combining all stylesheets into a single stylesheet. It’s a simple idea that hasn’t seen wide adoption. The ten top U.S. web sites average 7 scripts and 2 stylesheets per page. Combining files is more challenging when the scripts and stylesheets vary from page to page, but making this part of your release process improves response times.

While all that has been going on I’ve been working with a few people on a Rails project and come across a great plugin; asset_packager from Scott Becker. Inspired by the Vitamin article by Cal Henderson (of Flickr and another good book fame), asset_packager compresses and merges CSS and Javascript based on a prescribed configuration.

I’m also working on a few PHP projects and wanted the same experience in these as well, so I’ve gone a put together a quick release of php asset packager. At the moment this is a CSS only release, with Javascript features coming later, along with all the rest of the features of the Rails plugin and anything else I can think of or need.

First off you define your stylesheet setup. I used YAML as my goal was to stick as close as possible to the Rails plugin.

stylesheets:
- base:
  - screen
  - header
- secondary:
  - foo
  - bar

The above simply states that you want to merge screen and header CSS files together to form a base stylesheet, and merge foo and bar together to create a secondary CSS file.

All you then have to do is include the following:

// define("ENVIRONMENT", "production");

echo Asset_packager::stylesheet_link_merged();

If the ENVIRONMENT constant is set to production the line includes the merged stylesheet links, otherwise each file is included seperately. This again mirrors the behaviour of the Rails plugin. A command line script is also included in the source to actually merge (and tidy) the CSS files.

The vague plan is to add the rest of the features and then maybe package up as a plugin for whatever software or frameworks I’m working with at the time. At the moment that probably means wordpress, symfony and code igniter. Everything is open source and available from svn over on the Google Code site. Contributions, in the way of code, bugs, documentation or thoughts welcome.

My Trac Ticket setup

I’ve been busy really getting to grips with Trac recently and thought I’d post up a few details. Trac for those that haven’t come across it is a wiki, issue tracking systems and source code browser all rolled into one. It’s open source and written in Python.

I’ll start off describing my current ticket setup, along with the code I use for reports. In future posts I’ll hopefully describe thinks like setting up users and permissions in a flexible way. I’ll leave describing installation to others, mainly because it’s a pain. I’ll also assume you’ve installed the WebAdmin plugin, or are familiar with trac-admin.

Create a new Ticket

By default Trac comes set up with severity, milestones, ticket types, priority and components and a few default options in each. The problem with all these options is that it takes more time and effort to add and manage issues - so they don’t get logged. Unless you know you absolutely need them I find it easier to remove most of these options in the first instance.

Trac new ticket

I prefer using Trac soley for issue tracking. It suffers badly in my opinion if used for more project management related activity. Other tools do that job better (and also do bug tracking poorly).

I keep priority in, but by default it’s blank and only has one other options - High. I only assign High priority to issues which are causing major problems, either preventing a successful build or stopping something happening that’s meant to.

Available reports

Trac reports list

Trac reports are simple (or not so simple) SQL queries with a few bits of special syntax. I have four reports set up; Open, Closed, Important and My Issues:

Open, not too suprisingly, displays all the open issues. As mixing different components here would be confusing we divide the list with headers for each components

SELECT
  component AS __group__,
  (CASE priority WHEN 'High' THEN '1' ELSE '' END) AS __color__,
  id, summary, owner,
  time AS created,
  component AS _component, 
  priority AS _priority,
  changetime AS _changetime,
  description AS _description
FROM ticket t
LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority'
WHERE status IN ('new', 'assigned', 'reopened') 
ORDER BY priority DESC, p.value, t.type, time

Closed is the reverse of Open (I’m starting to state the obvious here) but with a few differences in terms of the listing. I’m more bothered here about seeing a timeline of closed issues. Good for keeping a eye (or a feed) on to see progress.

SELECT
  (CASE priority WHEN 'High' THEN '1' ELSE '' END) AS __color__,
  id, summary, owner, component, resolution,
  priority AS _priority, 
  changetime AS modified,
  description AS _description
FROM ticket t
LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority'
WHERE status NOT IN ('new', 'assigned', 'reopened') 
ORDER BY changetime DESC

My Issues lists only those issues which are assigned to me (or the currently logged in user) and open.

SELECT
  component AS __group__,
  (CASE priority WHEN 'High' THEN '1' ELSE '' END) AS __color__,
  id, summary,
  time AS created,
  component AS _component,
  priority AS _priority, 
  changetime AS _changetime,
  description AS _description
FROM ticket t
LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority'
WHERE status IN ('new', 'assigned', 'reopened') AND owner = '$USER'
ORDER BY priority DESC, p.value, t.type, time

Important is an overview of critical issues. Software can have small bugs and be perfectly usable in the vast majority of cases. But some issues either effect lots of users or prevent the system working at all. These High priority issues need addressing first, before anything else.

SELECT
  component AS _group__,
  (CASE priority WHEN 'High' THEN '1' ELSE '' END) AS __color__,
  id, summary, owner,
  time AS created,
  component AS _component,
  priority AS _priority, 
  changetime AS _changetime,
  description AS _description
FROM ticket t
LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority'
WHERE status IN ('new', 'assigned', 'reopened') AND priority = 'High'
ORDER BY priority DESC, p.value, t.type, time

Interesting bits here include the following line which makes all High priority tasks have a red border:

(CASE priority WHEN 'High' THEN '1' ELSE '' END) AS __color__,

As a side note, I hadn’t written proper SQL for a while (all that hacking on APIs). Suprisingly good fun as it turns out.

Conclusions

The main benefit of this setup is simplicity. Too many bug tracking systems are either way too complicated, or come with defaults which are on the complex side (I include Trac in this last category). Garret Dimon has been writing about the development of a new bug tracking system with a focus on simplicity. Fixx and Lighthouse are other products looking to fill this niche. That’s not to say larger teams don’t need to store more information than this, just that by starting small you can actually find out what you’re missing - rather than guessing and making the whole process of adding and fixing bugs a particularly painful one.

The Future of The Future of Web Apps

Continuing the catch up, as mentioned, here’s some thoughts on the recent Future of Web Apps Expo conference.

It’s fair to say the last Future of Web Apps (FOWA) event got a little stick due to what was seen as too many paid sponsors on stage lacking interesting things to say. Some designers and developers on the coal face also felt all the talk about startups just wasn’t relevant to their day jobs. So you have to hand it to the gang at Carson Systems +Carsonified for coming back with an even bigger and better event which addressed most of these concerns.

Simon and Brian came in to help with the speakers and at some point the event went for two tracks - the developer stage and the entreprenuers stage. At the same time the sponsors (supplemented by lots of smaller uk startups) moved into a large expo area outside the two stages. All this meant a move to ExCel which is a bit big, soulless and out of town for my liking but hey, how many places can deal with that many people with space to spare?

In the interests of fairness I have to say I ended up helping out, getting there early on the first day and standing behind the desk (with Cristiano and a few other nice people) for the mad rush at registration. I hadn’t worked on a large event for quite a while, I’d forgotten how much fun it can be! I don’t think that makes me biased though.

Highlights for me included Matt Mullenweg on all sorts of interesting architecture bits and pieces learned from wordpress.org, Heidi Pollock with practical tips for mobile web development, Paul Graham on startups in general (and why we all need to be in Silicon Valley apparently), the endlessly entertaining Simon Wardley on ducks (and commodisation of online services) and Matt Biddulph of Dopplr on practical tips for playing nice with social networks. Chatting with Steve Sounders and hearing about how he got the title of Chief Performace YAHOO! was also good fun. The book is on it’s way as I write.

Dave Morin from Facebook was a disappointment. He basically showed everyone facebook (which I would imagine 99% of the audience are already on), talked up some huge numbers and then dodged questions about walled gardens and open data.

I didn’t make it into the entreprenuers stage as much as I should have done. Mike Stenhouse spent most of his time in there I think and had nice things to say about most of the talks. I caught Leisa Reichelt and Dick Costolo, both of whom gave particularly interesting talks to round off the second day of the conference. Although at least some people I spoke to don’t like having more than one track (the same topic of conversation cropped up at @media) I much prefer the choice of sessions. Missing the odd clash is a small price to pay for the larger number of interesting talks.

Overall; lots of interesting content from lots of interesting speakers - many of whom I hadn’t seen speak previously (always a good sign). As for constructive criticism I’d have to say that the half hour slots of the developer track were probably a little too short in some cases and the conference was lacking in real, hands on, practical tech sessions.

One final thing that can’t go without mention; Diggnation. I had no idea what this was all about and quite frankly I’m not sure I do now. I’m still carrying the mental scars. I don’t think anyone can explain that in words so, here’s the video.

Mashup Demo

Long time no post. Lots of excuses, time away (including a holiday) and work catching up with me. Hopefully expect a few quick fire posts to catch up and then back to normal.

I was in London a couple of weeks back, mainly to help out at FOWA (which can get it’s own post) but also to generally catch up with lots of people about interesting stuff. After using my Monopoly knowledge of London to find the apple store (something some Londers took issue with) I heard about an event round the corner thanks to the magic of the internets (in this case Twitter, Upcoming and Dave Stone).

Mashup Demo was billed as “a simple and effective way for startups, growing businesses, enterprises and corporates to present themselves to an audience of professionals who would include:- investors, media journalists and bloggers, potential partners and industry influencers”. When first chatting with one of the exhibiting companies they looked me up and down and said “so, are you a blogger?”. Apparently I didn’t look much like a VC. Who knew?

Fifteen lightning presentations made up the bulk of the event, with me taking notes using my handy N800 (another blog post probably), and then followed a good amount of time to wander about and chat with everyone, probably about 100 people all in all. The companies presenting represented lots of the hot topics at the moment shall we say. Anyone not mentioning mobile, or social networking (or both!) had come to the wrong party.

Local Search had a strong showing from welovelocal.com, Rumble, and tipped.co.uk. The thorny problem of local was one of the really interesting discussion topics at the BBC innovation labs earlier in the year and it’s good to see interesting ideas of how to solve it. Although personally I don’t need another social network and also want to be able to define what I mean by local (not just be given a location, or have it based on a pool of friends, or whatever the latest looking glass is).

Identity was suprisingly popular with both Bondai and meecards talking up their applications. Obviously with all the cool kids playing with OpenId and the alpha geeks tinkering with OAuth identity is technologically interesting, but both these companies have spotted commercial opportunities as well. Be interesting to see how this all plays out. I’ve rambled on before about what I called Banks for Data (as apposed to money, which is just data anyhow when you think about it) but not really considered early movers and entrepreneurs in this sector.

A couple of interesting takes on agregators dropped in as well. fav.or.it was pitched as trying to popularise feeds for everyone else, but looked to me to actually be more of a useful tool for the information hungry crowd. It’s innovative features (inline commenting looked nice) would likely not be that interesting to people with only a few feeds and a few minutes but for people with hundreds of feeds look interesting. Brandwatch was similar to an idea a friend and me kicked around a while back, but never put together. Simply put it finds stories relating to your brand (company, product, whatever) and ascertains the importance and whether it’s positive or negative. You can then track all that over time with useful graphs and charts and the like. At the moment they are indexing about 500,000 sites and it still uses some manual filtering I think. Even so I know of clients past who would definately find the service offered interesting.

A few others caught my eye; an ad network for bloggers based on their blog rolls called rollSense, a service simliar to slideshare but for business documents called Edocr and testcard.tv. Testcard.tv does some really pretty impressive agregation, stitching and searching of online video. Sort of a glimpse of the future, but probably not as pretty. Optimising that for the Wii or PS3 (or keyboard/remote control) would be awesome.

After chatting with a few of the companies, and other people in the crowd, I popped for sushi with the aforementioned Dave for discussions of the freelance life I’ve got myself in to. The event was pretty darn interesting. That the companies presenting were, for the majority, proper startups was interesting. And the fact the whole event was geared around product centric companies and the surrounding eco-system interested me. I get to similar business events up and around newcastle, but these tend to be more service company orientated it seems. All in all Mashup Demo kicked my stay down in London off nicely.

Getting going with Symfony

I’ve tinkered with a rather large number of these web application frameworks, in Ruby, Python, PHP and C#, at some point over the last few years. But I’ve never really settled on any of them - mainly because my day job didn’t need me to and also because I like playing with new toys. I even tinkered with my own homage to Web.py in PHP which a few people have picked and are running with.

Right now though I’ve been doing some work with a few people on a Rails project. Their are some seriously nice things in Rails that I don’t see elsewhere; really good testing support baked in for one, and some serious deplyments for another.

Symfony is another PHP framework but one that only supports PHP5 (a good thing in my book). It’s basically a slick integration of lots of existing mature PHP tools plus some good old MVC and tried and trusted Software Patterns. I also discovered from a few YAHOO’s that they have started using it on some levels within YAHOO.

My interest piqued I found a couple of related presentations on Slideshare (my new favourite web app), and the site has a high quality online documentation which can also be purchased in dead tree format.

Having a play around with Symfony it took me a little while to get everything up and running, but I’d kept some handy notes for later. Hopefully they might be useful to someone other than me.

I’ve dropped in a few variables denoted by braces where relevant. The paths are from my MAMP and PEAR based environment and I’m using a MySQL database, though you could just as easily use Sqlite.

First we set up a virtual host in Apache. You have to do this manually at the moment. I already have a script for setting up virtual hosts under apache on OS X which I might extend to do this for me if I end up using Symfony alot.

<VirtualHost 127.0.0.1>
  ServerName 
  DocumentRoot "/Applications/MAMP/htdocs/{project}/web"
  DirectoryIndex index.php
  Alias /sf /Applications/MAMP/bin/php5/lib/php/data/symfony/web/sf
  <Directory "/Applications/MAMP/bin/php5/lib/php/data/symfony/web/sf">
    AllowOverride All 
    Allow from All
  </Directory>
  <Directory "/Applications/MAMP/htdocs/{project}/web">
    AllowOverride All
    Allow from All
  </Directory>
</VirtualHost>

cd /www
mkdir {project}
cd {project}

Now we dive into the actual symfony commands. We’ll just setting up both a frontend and a backend for this app which I think is likely to be pretty typical. I quite like the idea of seperate entities for these.

symfony init-project {project_name}
symfony init-app frontend
symfony init-module frontend {module_name}

Next we have to configure our database connection by editing /config/databases.yml

all:
    propel:
      class: sfPropelDatabase
      param:
        dsn: mysql://{username}:[email protected]:3306/{database_name}

That done we can set up our data definitions in /config/schema.yml. Read the documention for all the details but you can probably get the general idea.

propel:
 item:
   id:
   title: varchar(200)
   description: longvarchar
   created_at:

And yet another quick change in /config/propel.ini

propel.database.createUrl = mysql://{username}:[email protected]:3306
propel.database.url = mysql://{username}:[email protected]:3306/{database_name}

The next set of commands build our model files, create SQL for out chosen database and then create the tables in the database.

symfony propel-build-model
symfony propel-build-sql
symfony propel-insert-sql

We’ll quickly create a backend using the scaffolding. Note that item here is the model name which is taken from the schema.yml config file above.

symfony init-app backend
symfony propel-generate-crud backend items Item

All in all not bad going. And that’s basically the same for most of the apps you’re likely to develop. It could probably be a little easier in the database and apache config areas in fairness. Rails’ use of an inbuild webserver really helps here. But the use of YAML to define the model is, in my mind, more straightforward that using Ruby in Rails migrations. I’ve an app in the works at the moment and I’m looking at using Symfony. If that pans out I’ll blog more about the bits I’m interested in - the inbuilt unit testing framework high on my list.

Brighton Rock(s)

Back from a madcap weekend in Brighton for the dconstruct BarCamp Brighton double header. If I was to write down everything I can remember we’d be here for a week. If I was to write down simply everything then we’d be here longer still. Information packed just doesn’t cover it.

I’ll likely sort some more detailed posts on some particular topics that piquet my interest in the coming weeks or months, for instance user centred design vs agile methodologies (thanks to Leisa Reichelt), some more rambling posts on web application frameworks, microformats for write APIs, OAuth and countless other things I’ve no doubt forgotten.

One thing you can get now is my BarCamp presention which I’ve uploaded to SlideShare. I’ve not used this much previously but I’m smitten and will be uploading a few other past presentions in the next few weeks as soon as I can find them. If you’re interested in RESTful architectures or Rabbits then this might float your boat:

I also took lots of photos (some with Norm’s very nice 1.4f 50mm lens, thanks Norm) whick I’ll get on Flickr when I get a moment. Expect lots of portraits of people (maybe including you?) for once instead of odd abstract architecture and textures.

Final thing I’ll say for now is a big thankyou to anyone or everyone who sorted all this out. The 600-700 geeks descending on brighton all got fed, watered/beered and educated without anything appearing to go wrong (except when the hotel bar ran out of beer - I blame Steve Marshall). I’m looking at you clearlefties and Glenn and everyone from Madgex. Good job everyone involved.

Newcastle Geekvenues

More interesting Newcastle centric news, although this time partly releveant to everyone else - well, as long as you live in Oxford, Brighton, Leeds or Manchester that this.

Natalie Downe has spent some of her copious spare time (having left the agency way of life and gone all freelance too) cooking up Geekvenues. It’s an attempt to map (using Google Maps) any and all geek related venues around the UK. Local meetup places perhaps, regular venues, or just geek friendly pubs with wifi.

I’ve been adding some venues for Newcastle and they can be found over at newcastle.geekvenues.com/. If anyone knows of any other ones let me know. And if anyone is thinking “I want it to cover my neck of the woods” then get in touch with Nat, ideally with a link to a KML file of locations. I’m looking at Alan and John from Edinburgh in particular here.

OpenCoffee Newcastle

Open Coffee

OpenCoffee Newcastle is a new, open, informal and regular meetup for technology entrepreneurs, designers, bloggers, developers, geeks, investors and anyone else who’s interested.

We are going to meet every Thursday morning, between 10am and 12pm, at The Side just off Newcaste Quayside.

The first event will be held on September 13th. So come and share ideas, demo, get to know each other or just have a coffee.

OpenCoffee Newcastle is part of the global community effort over at opencoffeeclub.org. Here’s the blog post that started it all

With all that out of the way a little bit of background and some geekier links. I’d been meaning to get an OpenCoffee event up and running for a while and after a couple of chats in nice pubs with Sam and a trip to The Side we had a date and a vague plan. Worst case is it’s me and Sam just meeting up but hopefully a few more people will make it along. We should have wireless as well for any demos.

Note I’ll likely be somewhat wired. I might be up in edinburgh the night before for FOWA roadtrip and in any case I’ll have been up all night at barcampbrighton.

You can register for the event on upcoming or even register for the group. Or you could just turn up. See you there! Any questions post a comment (either here on on upcoming) or drop me an email.

HTML. An alternative to Word?

When I’m writing code, be it CSS, Javascript, Ruby, whatever, I don’t like big bulky IDE’s. Give me TextMate or Vim any day. Going all freelance I now have the occasional need to produce nice documents and being in charge means I get to decide what I use to produce them. I never really liked working in Microsoft Word so decided against it. But rather than sensibly going for an obviously alternative I decided to just use HTML, or rather more specifically, Textile.

An example of a current document

As an old time textpattern I learned to love the simple text transformation language and it’s inclusion in the 37signals apps made me smile and more likely to use them. Why use it over HTML you may ask? For me I think about HTML like a browser, as a tree of nodes. I end up spending quite a bit of time writing content then going backwards and wrapping what I just wrote in markup. This is fine for your typical site, but for writing documents it’s hard to get into a flow. With textile I can flow in one direction.

So far I’m liking it alot. I can seperate content from presentation (Word styles are a myth), easily repurpose my content (text, html, pdf) and write documents in anything I like, including Word if I chose to.

It’s been a pretty good excuse to play with various bits of CSS3, including multi-columns, advanced layout, generated content and the various print offerings. Given I can control the rendering engine then produce a PDF to distribute I can use anything that the latest browser builds can deal with.

I’ve not got all the workflow steps automated yet, I’m using Firefox to print documents to PDF (via Preview) at the moment which works fine but is still a manual process I can get rid of using something like css2xslfo and fop. I’m also pretty sure the PDFs won’t be accessible, although the original markup certainly should be.

As I produce more complex documents I’ll likely miss things like automated table of contents generation, footnotes and page numbering but, then again, I can write that in Javascript (or on the server) if I so choose. I was impressed listening to HÃ¥kon Wium Lie at @media about the process they used to format his last book using CSS and PrinceXML. I’m half tempted to spend time rolling all this together into a web app. It would be pretty good fun really; APIs, XSLT, CSS, PDF, some nice UI problems, etc. Only problem I see would be entering a market contested by Microsoft, Google, Sun and er, lots of others.

Some Thoughts on Good Conferences to Attend - UK

Ben Metcalfe just posted a run down of some thoughts on conferences to attend. With all my travels this year I thought I’d join in.

I’ve spend quite a bit of time this year travelling the UK for various get togethers, conferences, meetups and the like and had great fun doing so. I go mainly because nowhere else do I find quite such a critical mass of smart interesting people to have a pint with and discuss the intricacies of the web. For 2007, in roughly chronological order: WebDD, Barcamp London, Future of Web Apps, Highland Fling, Refresh Edinburgh, @media, Hackday. I’ll be at dConstruct, Future of Web Apps (again) and possibly Future of Mobile and a Rails meetup in London as well to finish off the year.

I won’t go on about how cool Barcamp was. I actually think the whole unconference thing sits alongside the more formal conference scene quite nicely rather than acting as a direct challenge. Safe to say if you have yet to make a Barcamp then be on the lookout (although if you could not pick brighton please. I really want to go and it’s going to be hard enough to get tickets anyway!)

Highland Fling was excellent. Alan did a bang up job of picking a pretty focused topic and getting a gang of safe hands in to do the talking. Everything seemed to follow on from the previous talk, or catch a thread only touched on earlier in the day. This turned out to be down to the specific topic rather than some pre-conference collusion which I thing bodes well for other such topic centric mini-conferences. It was all rounded off (as all conferences must be) with a rousing talk from Andy Clarke and a panel chaired by Jeremy Paxman Keith.

media is still the daddy of the UK web standards scene. Personally I want more advanced materials and I'd love to see more audience interaction, but the scale of it makes that a little difficult I acknowledge. I'll keep going as long as I learn something new and come away feeling a little bit inspired. I'll be going next year. Maybe it's not going to hit the heights of the very firstmedia a couple of years back but that was my first trip out into the world. Oh, and patrick throws good parties in pubs with beer.

I think dConstruct is basically a grown up Highland Fling. It was great fun last year so here’s hoping it’s as much fun this year. Again, the idea of a particular topic is in place. I’m still thinking about @media Ajax if I can get enough work together to pay for it (note to people trying to keep up - I now work for myself as a freelancer) and I have a sneaking suspicion I’ll want to get along to Future of Mobile in a couple of months too.

My plan for next year is to try and get out of the country and hopefully to some of the events in the US and Europe that offer something different. SXSW is an obvious choice, but I’d be interested in any other ideas on top of Ben’s list. I’d also like to think in a year of so I might even end up talking at some nice event or other - but I need to pick a topic and stick to it I reckon instead of flitting between whatever I feel like that afternoon. Oh and probably practice.