This article presumes basic knowledge of AWS lambdas and AWS Parameter Store

Introduction 🔗

All systems are different, however one of very common problems which nearly all software has is how to deploy to different environments, usually starting in a local or development environment and ending in a production environment.

The 12 factor app [1] approach to deployment defines a number of approaches to solving various aspects of supporting multiple environments. This article focuses on two of those approaches: Configuration and Environmental Parity.

One of the key points of these approaches is that the same code should be deployed to every environment with differences in configuration (URLs, usernames, databases URI, feature switches). With AWS we can go a step further than that and not only have the same code across environments, but also the same infrastructure by using AWS Cloudformation (and/or SAM).

This is called Infrastructure as Code and is an extremely powerful way of managing your entire stack.

You won’t see any AWS console screens in this post. It’s code all the way. Everyone should be doing this and make sure you are version controlling your templates!

One common practice to support different environment is to set up different AWS accounts for each environment, for example, you might have the following environments; development, QA, pre-production and production which you would set up different AWS accounts. This might seem slightly strange at first, it definitely did for me. However, this gives you a huge amount of flexibility and control in managing your environments and is considered AWS best practice.

EDIT January 2019: A update to the above recommendation. We’ve found that we have hit certain account limits with the approach above (one of which is Filter Subscription limits). We’re now planning using different accounts for different part of our architecture and each environment. So for example, we might have a “website-dev”, “website-production”, “stockmanagement-dev”, and “stockmanagement-production” accounts.

For more on limits see here

Cloudformation 🔗

AWS Cloudformation and AWS Parameter Store can combine together to create AWS stacks which have 12 factor attributes.
Let’s look at an example of a basic Cloudformation template snippet which defines a lambda and API gateway which requires some database connection details (hostname/username/password)

There are a few problems with the above template. We would not be able to deploy to multiple environments and be able to support different usernames/passwords and hostnames. Never mind the fact that our username and password are stored in plain text living inside our source control.

So let’s make this template slightly better by introducing parameters to our template.

You can see now that there is no reference to the actual username/password or database host values within the template. We’ve now separated configuration and code and could stop there and expect variables to be passed into the template.

However, let’s go a step further and manage these variables via AWS Parameter Store.

Parameter Store is a AWS product that allows you to manage configuration items in our AWS stack. It allows create, read, update and delete operations on key value pairs. You define the name, type and value of the parmeter and can then look up the parameters using Cloudformation.

Looking at the above you can see we have changed the Parameter Type to a Parameter Store Type and and introduced a ‘Default’ property for each variable.

The Default properties value is the key to the Parameter Store variable to be looked up and populated.

Migration Path for existing Parameters 🔗

One of the gotchas with Cloudformation and Parameter Store is migrating already defined Parameters to use Parameter Store values.

So in our second example we have a number of parameters defined, if this template has already been deployed to AWS and we were to convert the type to Aws::Sam::Parameter::Value<String> and redeploy then unfortunately this will fail (February 2019 — issue raised with AWS)

In order to get around this you can use Dynamic references to look up the variables like so;

EDIT: September 2010: Resolving the parameters like this requires you to give a version number. We’ve found this to mean an extra maintenance overhead that we’re not prepared to accept. i.e. everytime we update the Parameter Store variable we’ll also need to update our cloudformation template to the new version.

A full example would look like this.

That’s it! So much simpler and you get rid of a load of boiler plate code too!

You can now set up Parameter Store variables for each account which contain environment specific information.

The gotchas 🔗

Parameter Store works well on the whole, but there are some gotchas that you need to be aware of.

  • Custom Domain Names must be lower case! Expect to receive a403 Forbidden for any upper case letters in the custom domain name. Also watch out for domain name length limits.
  • For the purposes of this article I used Parameter Store to hold the database username/password. This approach should not be used in practice — sensitive information should be stored in Secret Manager or Secure Strings.
  • If you have a existing Parameter and you change that to a Parameter Store variable you need to pay delete the stack, delete or rename the variable first. This also applies to changing the Default value of a Parameter. I’ve spoke to AWS about this and they admit this is far from ideal.
  • When putting a http as a string into Parameter Store beware that Parameter Store tries to be clever and look up that URL and populate the content. You need to wrap up the http in json when putting.
  • Storing secure strings — Cloudformation only support for certain Resources. So you can’t just pass in a ENV — you need to use one of the support types.

Reference 🔗

  1. 12 factor app: eBook ↩︎