The Impact of Amazon's new CloudFormation service

Let me put to rest the worst of the FUD. This was never a master plan by Amazon to wipe out Chef and Puppet in a hostile takeover of the configuration management territory. Opscode were part of the CloudFormation Beta, and deeper integration with Chef is very much part of the future roadmap. So don't worry - this is not an apocalyptic disaster - it's an overwhelmingly good and exciting development that promises to make the task of complex orchestration a little bit easier.

CloudFormation is a service that simplifies the process of firing up a complete AWS stack. Instead of making individual API calls to set up EC2 instances, elastic load balancers, scaling groups and other offerings, we simply make one call. This is great - because previously making these calls was a bit of pain. Your options ranged from using the AWS console, which is pretty unpleasnant, through using tools such the Java-based EC2 command line tools, through to scripting a series of calls with a library such as Fog or Boto.

Does that sound a lot like Chef or Puppet to you? No. Sure, knife has EC2 management capabilities because it wraps Fog, but that's peripheral, and is really just recognition of the fact that Amazon hadn't produced a fully featured and consistent way to drive their API.

The main point of confusion here is that people are equating provisioning and configuration management. Provisioning is going to the shop and buying a server. Racking it and cabling it. Putting it in the right VLAN. Giving it a port and an IP address and sticking an operating system on it. Outside of the cloud this is a pretty major undertaking, but the cloud makes all this very easy. Configuration management is policy driven. It's deciding what software goes onto the machine, how it's configured, how it should behave in certain circumstances, and enforcing that. You need both - CloudFormation provides the former.

Let's be clear - I'm not downplaying the significance or awesomeness of the service. What Amazon have done with CloudFormation is make it much much easier to do this at a stack level rather than for each individual component of an AWS infrastructure. Together with Elastic Beanstalk, Amazon are doing some important and innovative stuff in this space.

For me the area which is of most interest is the mechanism for creating these stacks. CloudFormation uses JSON templates to specify the infrastructure components and interdependencies. Amazon have provided some sample templates for provisioning popular opensource stacks such as Drupal, Wordpress and Redmine. I think this is what has caused all the excitement. However, it's important to remember that this is purely image-based - there's no ongoing management of the essential configuration of these machines.

What excites me about all this is that it's... JSON. We like JSON - JSON is used throughout Chef, and CloudFormation opens up lots of possibilities for creative interplay. Far from competing with or replacing Chef, CloudFormation plays directly to its strengths. Chef metadata can be passed from a JSON template, including role information, validation key and Chef server URL. The end result is a fully configured and managed AWS infrastructure, from scracth, with one call.

The other exciting thing is that this JSON can just be stored in a databag. This suddenly makes it really rather easy to manage and control some of the more complicated and powerful AWS services such as the queing service, or cloud watch alarms from the very heart of your configuration management tool.

So: is CloudFormation awesome? Yes. Exciting? Absolutely. Powerful? You bet! A replacement? A threat? Absolutely not - what we have here is the next generation in server automation and provisioning, in a form which slots in perfectly with next generation system integration and configuration management. Bring it on.

Show Comments