Dec 28, 2013 · 2 minute read
Hyde is a brazen two-column Jekyll theme that pairs a prominent sidebar with uncomplicated content. It’s based on Poole, the Jekyll butler.
Built on Poole
Poole is the Jekyll Butler, serving as an upstanding and effective foundation for Jekyll themes by @mdo. Poole, and every theme built on it (like Hyde here) includes the following:
- Complete Jekyll setup included (layouts, config, 404, RSS feed, posts, and example page)
- Mobile friendly design and development
- Easily scalable text and component sizing with
rem units in the CSS
- Support for a wide gamut of HTML elements
- Related posts (time-based, because Jekyll) below each post
- Syntax highlighting, courtesy Pygments (the Python-based code snippet highlighter)
In addition to the features of Poole, Hyde adds the following:
- Sidebar includes support for textual modules and a dynamically generated navigation with active link support
- Two orientations for content and sidebar, default (left sidebar) and reverse (right sidebar), available via
- Eight optional color schemes, available via
Head to the readme to learn more.
Hyde is by preference a forward-thinking project. In addition to the latest versions of Chrome, Safari (mobile and desktop), and Firefox, it is only compatible with Internet Explorer 9 and above.
Hyde is developed on and hosted with GitHub. Head to the GitHub repository for downloads, bug reports, and features requests.
Oct 13, 2013 · 3 minute read
Originally published on Medium.
We have a bunch of internal mailing lists at work, and on one of them someone asked:
we’re looking into monitoring/logging tools…
I ended up writing a bit of a long reply which a few people found useful, so I thought I’d repost it here for posterity. I’m sure this will date but I think it’s a reasonable snapshot of the state of open source monitoring tools at the end of 2013.
Simply put, think about four elements and you won’t be far off on the
technical front. Miss one and you’re probably in trouble.
- metric storage
- metric collection
- monitoring checks
For logs, some combination of syslog at one end and elasticsearch and
Kibana at the other are probably the state of the open source art at
the moment. The shipping around is more interesting; Logstash is improving constantly, Heka is an similar alternative from Mozilla, and Fluentd looks nice too.
For pure metrics it’s all about Graphite, which is both awesome and
perilous. Not much else really competes in the open source world at
present. Maybe OpenTSB (is you’re into a Hadoop stack.)
For collecting metrics on boxes I’d probably look at collectd or diamond both of which have pros and cons but work well. Statsd is also useful here for different types of metric collection and aggregation. Ganglia is interesting too, it combines some aspects of the metrics collection tools with an integrated storage and visualisation tool similar to Graphite.
Monitoring checks is a bit more painful. I’ve been experimenting with Sensu in hope of not installing Nagios. Nagios works but it’s just a bit ungainly. But you do need somewhere to write checks against metrics or other aspects of your system and to issue alerts.
At this point everyone loves dashboards, and Dashing is particularly lovely. Graphiti and Tasseo for Graphite are useful too.
For bonus points things like Flapjack and Reimann provide some interesting extra capabilities around alert control or real time monitoring respectively.
And for that elusive top of the class grade take a look at Kale, which provides anomaly detection on top of Graphite and Elasticsearch .
You might be thinking that’s a lot of moving parts and you’d be right. If you’re a small project running all of that is too much overhead, turning to something like Zabbix might be more sensible.
Depending on money/sensitivity/control issues lots of nice and not so
nice commercial products exist. Circonus, Splunk, New Relic, Boundary and Librato Metrics are all lovely in different ways and provide part of the puzzle.
And that’s just the boring matter of tools. Now you get into alert design and other gnarly people stuff.
If you got this far you should watch all the Monitorama videos too.
Aug 11, 2013 · 5 minute read
Originally published on Medium.
I’m a big fan of the Platform as a Service (PaaS) model of operating web
application infrastructure. But I’m a much bigger user and exponent of
Infrastructure as a Service (IaaS) products within my current role
working for the UK Government. This post describes why that is, and
hopefully helps anyone else inside other large enterprise organisations
reason about the advantages and disadvantages, and helps PaaS vendors
and developers understand what I personally thing is a barrier to
adoption in that type of organisation.
A quick word of caution, I don’t know every product inside out. It’s
very possible a PaaS product exists that deals with the problems I will
describe. If you know of such a product do let me know.
A simple use case
PaaS products make for the very best demos. Have a working application?
Deployment is probably as simple as:
git push azure master
Your app has started to run slowly because visitors are flooding in?
Just scale out with something like:
heroku ps:scale web+2
The amount of complexity being hidden is astounding and the ability to
move incredibly quickly is obvious for anyone with experience of doing
this in a more traditional organisation.
A not so simple use case
Even small systems are often being built out of many small services
these days. Many large organisations have been up to this for a while
under the banner of Service Orientated Architecture. I’m a big fan of
this approach, in my view it moves operational and organisational
complexity back into the development team where its impact can often be
minimised by automation. But that’s a topic for another post.
In a PaaS world having many services is fine. We just have more
applications running on the Platform which can be independently scaled
out to meet our needs. But services need to communicate with each other
somehow, and this is where our problems start. We’ll keep things simple
here by assuming communication is over HTTPS (which should be pretty
typical) but I don’t think other protocols make the problem I have go
away. The same problem applies if you’re using a SaaS database for
It’s the network, stupid
Over what network does my HTTPS internal service call travel? The
internet? The internal PaaS vendor’s network? If the latter, is my
traffic travelling over the same network as other clients on the
platform? Maybe I’m running my own PaaS in-house. But do I trust
everyone else in my very large organisation and want my traffic on the
same network as other things I don’t even know about? Even if it’s just
me do I want internal service traffic mixing with requests coming from
the internet? And are all my services created equally with regards what
they can and cannot access?
Throw in questions like: is the PaaS supplier running on infrastructure
provided by a public IaaS suppliers who you don’t have a relationship
with and you start to question the suitability of the current public
PaaS products for building secure service based systems.
A journey into Enterprise Architectures
You might be thinking, pah, what’s the worst that can happen? If you
work for a small company or a shiny startup that might be completely
valid. If on the other hand you’re working in a regulated environment
(say PCI) or dealing with large volumes of highly sensitive information
you’re very likely to have to build systems that provide layers of
trust, and to be doing inspection, filtering and integrity checking as
requests flow between those layers.
Imagine that I have a service dealing with some sensitive data. If I
control the infrastructure (virtualised or not, IaaS provided or not)
I’ll make sure that service endpoint isn’t available to anything that
doesn’t need access to it via my network configuration. If I’m being
more thorough I’ll filter traffic through some sort of proxy that does
checking of the content; It should be JSON (or XML), it should meet this
schema, It shouldn’t exceed this rate, it shouldn’t exceed this payload
size or response size, etc. That is before anything even reaches the
services application. And that’s on top of SSL and maybe client
If I don’t control the infrastructure, for example when running on a
PaaS, I lose some of the ability to have the network protect me. I can
probably get some of this back by running my own PaaS on my own
infrastructure, but without awareness and a nice interface to that
functionality at the PaaS layer I’m going to lose lots of the benefits
of running the PaaS in the first place. It’s nice that I can scale my
application out, but if new instances can’t connect to the required
backend services without some additional network configuration that’s
invisible to the PaaS what use is that?
The question becomes; how to implement security layers within existing
PaaS products (without changing them). And my answer is “I don’t know”.
Why isn’t SSL enough?
SSL doesn’t help as much as you’d like to think here because if I’m an
attacker what I’m probably going to attack is your buggy code rather
than the transport mechanism. SSL doesn’t protect you from SQL injection
or unpatched software or zero-day exploits. If the only thing that my
backend service will talk to is my frontend application, an attacker has
to compromise two things rather than just ignore the frontend and go
after the data. Throw in a filter as described above and it’s really
three things that need to be overcome.
The PaaS/IaaS interface
I think part of the solution lies in exposing some of the underlying
infrastructure via the PaaS interface. IaaS is often characterised as
compute, storage and network. In my experience everyone forgets the
network part. In a PaaS world I don’t want to be exposed to storage
details (I just want it to appear infinite and pay for what I use) or
virtual machines (I just care about computing power, say RAM, not the
number of machines I’m running on) but I think I do, sometimes, want to
be exposed to the (virtual) network configuration.
Hopefully someone working on OpenShift or CloudFoundry or Azure or
Heroku or DotCloud or insert PaaS here is already working on this. If
not maybe this post will prompt someone to do so.
Apr 23, 2013 · 3 minute read
I’ve become increasingly interested in web application security issues over the last year or so. Working in Government will do that to you. And I’ve come to the conclusion that a) there are lots of good open source security tools, b) many of them are terribly packaged and c) most developers don’t use any of them.
I’ve been having related conversations at recent events I’ve made it along to, including Devopsdays London which featured some good open spaces discussions on the subject. Security is one of those areas that, for many organisations, is basically outsourced to third party penetration testing firms or consultants. Specialists definitely have a role to play, but with a move towards increasingly rapid releases I think in-house security testing and monitoring is going to get more and more important.
I’ve started to build a collection of tools on GitHub, along with a vagrant setup to test them out. Full instructions are available on that repository but the short version is you can run one command and have one virtual machine filled with security testing tools and, if useful, another machine running a vulnerable web application with which to test. The current list of tools runs to:
But I’ll add more tools as I discover them or as people file issues or pull requests.
What about Backtrack?
When I started investigating tools for security and penetration testing most roads led to Backtrack. This is a complete Linux distribution packed with a huge number of security tools, including many if not all of the above. Why then did I write puppet code rather than create a Vagrant box from Backtrack? Firstly, Backtrack is probably great if you’re a professional penetration tester, but the barrier to entry to installing a new distibution for most developers is too high in my view. And with a view to using some of these tools as part of monitoring systems I don’t always want a separate virtual machine. I want to be able to install the tools wherever I want. A good configuration management tool gives you that portability, and Vagrant gives you all the benefits of a local virtual machine.
As mentioned I’d like to expand how some of these tools are used to include automated monitoring of applications, maybe look at ways of extracting data for metrics or possibily writing a Sensu plugin or two. The first step to that is probably breaking down the monolithic puppet manifest into separate modules for each tool. Along the way I can add support for more operating systems as required. I’ve already done that for the wackopicko module which is up on the Forge.
I’m also soliciting any and all feedback, especially from developers who don’t do any security related testing but feel like they should.
Mar 23, 2013 · 2 minute read
I’ve not been writing many blog posts lately, but I have been doing quite a bit of writing elsewhere. One of the things I’ve had a hand in at work is the new Government Service Design Manual. This is the work of many people I work with as well as further afield. It’s intended to be a good starting place to find information about building high quality digital services.
The manual is in beta and we’re looking for as much feedback as possible on the whole thing. It’s already proving useful and a good way of framing the scope of discussions, but it has lots of room for improvement.
If you’re reading this post I’m going to wager you’re interest lies in or around devops flavoured content. The following are guides I’ve written in this area that I’d love any and all feedback on.
If you’re interested in the background to this endeavour then a couple of blog posts from some of my colleagues might be of interest too. First Richard Pope talks about how the manual came about and here’s a post from Andrew Greenway about this beta testing of the service standard.
The source for all this is on GitHub so if you prefer you can just sent a pull request. Or I’m happy to get emails or comments on this post. In particular if people have good references or next steps for these guides then let me know as several of them in particular are lacking in that area.
Mar 23, 2013 · 1 minute read
I had fun speaking at QCon in London earlier this month with a talk on the Cloud track entitled the Perils of Portability.
This had some Governmenty stuff in but was mainly part rant, part hope for the future of cloud infrastructure. I had some great conversations with people afterwards who felt some of the similar pain which was nice to know. I also somehow managed to get 120 slides into a 40 minute presentation which I think is a personal records.
The videos will be available at some point in the not too distant future too.
Feb 17, 2013 · 1 minute read
About a month ago I had the good fortune of speaking at the London Web Performance meetup. This was one of the first talks I’ve done about our work at The Government Digital Service since the luanch of GOV.UK back in October. The topic was all about moving quickly in a large organisation (The UK Civil Service is about 450,000 people so I think it counts) and featured just a hand full of technical and organisational tricks we used.
Feb 17, 2013 · 1 minute read
With only a week or so to go before the end of February, it’s looking like March might be a little busy.
- I’m speaking at QCon, in London on Wednesday 6th on Clouds in Government - Perils of Portability (which in hindsight is probably the silliest title for a talk I’ve ever used)
- On the 15th and 16th of March I’ll be at Devopsdays, again in London. I’ve been helping out with organising the event and I’m very much looking forward to going along after seeing all the work being put in.
- And last but not least I’m heading to Boston for the rather exciting Monitorama from the 26th until the 30th. Looking forward to meeting up in person with quite a few folks I’ve spoken to over the last year or two.
If you’re going to be at any of these events (QCon and Devopsdays still have tickets available I think) then let me know.
Jan 13, 2013 · 1 minute read
I had great fun back in November at the QCon conference in San Francisco. As well as currating one of the tracks and catching up with people in the area I managed to give the following talk.
In hindsight it might have been a bit odd to try and cover both Rails and Django examples in the one presentation but it was quite good fun putting together code examples using both of them at the same time. As well as a large set of tips, tricks and tools I settled on a few things that I think any web (or other) framework should support out of the box.
- A debug toolbar
- Transparent caching support
- Hooks for instrumentation
- Configurable logging
Dec 30, 2012 · 2 minute read
I’m a big fan of system packages for lots of reasons and have often ended up rolling my own debian package repository at work, or working with others that have done so. Recently I finally got round to setting up a personal package repo, at packages.garethrushgrove.com. More interesting than the repo is probably the tool chain I used, oh and the rather nice bootstrap based styling.
The source code for everything is on GitHub although not much documentation exists yet. In the middle are a few shell scripts that generate the repo. Around them is a Vagrant box (which makes it easier to build packages for different achitectures or distros) and some Rake commands
<code>bundle exec rake -T
rake recipes:build[recipe] # Build a package from one of the available recipes
rake recipes:list # List available recipes
rake repo:build # Build the repository</code>
The recipes commands allow for building new packages based on scripts. A few examples are included which use fpm, but you could use anything. The repo:build command triggers the debian repository to be rebuilt.
The vagrant configuration shares various folders between and guest and host which also opens up a few useful features. One is I can just drop any old debian package into the debs folder and run the repo:build command and it will be in my repository. The other useful capability is that the resulting repo is shared back to the host, which means I can then check it into Git and in my case push it up to Heroku.