Abusing JSON: Why AWS CloudFormation sucks?!
· Software Engineering · Updated
By Jan Brennenstuhl · 3min read
AWS CloudFormation (CFN) is a file-based interface for provisioning AWS resources and in that way is Amazons answer on the ‘Infrastructure as Code’ paradigm. I am a huge fan of this approach and CFN is a big step into the right direction as it supports not just common DevOps ideas, by allowing devs to create, update and delete the operational infrastructure they need with their own tools (a text editor, version control and tests).
It also enables you to think different about your IT infrastructure and how to work with it (e.g. just think about immutable stacks). However, I see some issues that become more important if you work with CloudFormation on a daily basis …
JSON Lang, WTF?!
CloudFormation templates are simple JSON-formatted text files that (in theory) can easily be created and maintained by hand as nowadays every dev (and/ or ops) should be able to read and write valid JSON. However, AWS CloudFormation uses the originally static ‘lightweight data-interchange format’ as a scripting language, allowing e.g. the definition and usage of parameter and (somehow crazy) intrinsic function references.
In my eyes, this is a heavy misusage that leads to several problems regarding complexity and comprehensibility. You can find several deterrent examples that illustrate this right in the official AWS developers guide. If you still don’t think JSON is a syntax-bloated, hard-to-read data format just have a look on how to define DataPipelines using CloudFormation and tell me you like it and that you are able to easily understand all the definitions as well as their interactions. It took me some time…
But that’s not even the worst part. JSON also lacks some feature substance that some devs would regard heavily essential. In contrast to its technical superset YAML, JSON is not aware of e.g. comments or user defined data types. In my eyes, this is a huge downside and a massive argument against Amazons classical CloudFormation design.
But enough JSON-bashing for now. Let’s solve at least some of our AWS CloudFormation issues.
Currently there are a couple of tools around that sit on top of CFN as an abstraction layer and offer not just a YAML-based config approach, but also further functionality that comes in handy if you work with infrastructure as code during your everyday business.
One of those tools is CFN-Sphere, a lightweight Python-tool that aims to simplify AWS CloudFormation handling not just for infrasturcture automation but also for human interaction. Initially built by Marco Hoyer (a former colleague and friend of mine) CFN-Sphere comes with fundamental support for e.g. cross stack parameter references and automatic stack dependency resolution including circular dependency detection. Both are amazing features also for smaller AWS infrastructure definitions like the backup and staging environment for DynamoDB tables I am currently working on (More on that topic in one of the upcoming posts).
So if you are struggling with your with your CloudFormation config or if you have a certain feeling that CFN files having around several hundred lines of code are almost unmaintainable, make sure to check out CFN-Sphere and its stack template feature. And help making it even greater by becoming a contributor!
Jan Brennenstuhl is a Principal Software Engineer at Zalando SE, balancing security with friction for their customers. He built an IAM team and brought single sign-on (SSO) to Europe's largest e-commerce fashion platform.