Hacker Posts

Mar 21, 2009 · 2 minute read

I’ve been playing again with App Engine, and going back to an on/off pet project that I’ve build variations of for a while.

It’s basically a pretty straightforward aggregation platform, taking content from a number of feeds and creating relationships between the items. It’s mainly an experiment in creating a decent size site on App Engine - it can be surprising how many urls you can get out of a good corpus of data:

400 items, each with a unique url
each item has a variable number of tags generated, at the moment that amounts to about 1000 unique urls
each month and year of content gets a url so that’s another 4 or so
each day for which we have items we also get a url, so that’s another 90 or so

So starting off with 1500 or so pages isn’t bad. The site also grows over time as new items are posted or I add more feeds. Everything else I’ve done with App Engine has been more application focused so seeing how a content site performs is interesting in and of itself.

The data in question I used for this first experiment was the feeds I could find from the top posters on Hacker News. Hence the name Hacker Posts. That left me with 35 feeds:

I have a couple of other communities or events that I’d like to do the same thing around, as well as a few features I want to add to the underlying software. The nice thing with App Engine is rolling new instances out is as simple as running a command.

More than seven

Hacker Posts