Infrastructure as Code - a Summary

Adhere to DORA's Accelerate research, track Delivery lead time, Deployment frequency, Change fail percentage and Mean Time to Restore. Prioritize speed and quality by continuously deliver infrastructure changes through code and automated pipelines.

These is the final part of the Infrastructure as Code series.

Define everything as code. Continuously test and deliver all work in progress. Build small, simple pieces that you can change independently.

You need a Cloud Age mindset, exploiting speed to improve quality and building quality in to gain speed. Automating your infrastructure takes work, but doing it helps you make changes (and build better systems from scratch).

Assume that systems are unreliable. Make everything reproducible. Avoid snowflake systems that you dare not touch. Create disposable things. Minimize variation to simplify maintenance, even if you dockerize everything. Ensure that you can repeat any action.

Minimize configuration. If you need radically different stacks/environments/instances, define a different project/template that describes a majority of those differences.

Avoid creating unnecessary stages in your pipeline, as each stage adds time and cost to your delivery process. Only add tests that deliver value. Consider that you can test different things during different stages of your pipeline and that tests get more valuable as they test more components. Tests should focus on common issues, variable outcomes and combinations of code.

Use sound software development practices when developing infrastructure as code. Refactor components so that they can be isolated. Use mock services when needed to isolate a component for testing. If your tooling supports it, use role inheritance to define a common base set of features and then create specializations from those.

Don't keep your test environments around. Rebuild them continuously.

Things have a habit of changing on a server when you aren't paying attention. Be especially wary of "quick changes" that we're sure won't break anything, and therefore don't remember making. Patching a server is an anti-pattern. Rebuild it instead.

Managed Kubernetes Clusters are not a Cloud Abstraction Layer. Applications need to access other resources, such as storage and networking, which will be provided differently by different platforms. Also consider other services such as monitoring, identity and secrets management. Again, these are much easier to cope with when they're tied into your cloud infrastructure, which adds to the difficulty of moving a K8s application from one provider to another.

The goal of modularity is to make it easier and safer to make changes to a system.

Reuse increases coupling.

Maintain small, focused source code repositories. Optimize for change and for domain concepts. Instead of maintaining a file with all firewall rules, maintain settings close to its consumer (e.g. firewall rules for this particular application).

Through IaaC practices, infrastructure code can be designed, reviewed and reused across multiple environments and systems. You don't need a lengthy design, review and signoff exercise for each new server or environment, if you use code that has already been through that process.

Your code creates environments far more consistently than humans, even if they try to follow checklists.

Automated testing, including for governance concerns like security and compliance, gives people working on infrastructure code fast feedback. They can correct many problems as they work, without needing to involve specialists for routine issues.

People who aren't specialists can make changes to the code for potentially sensitive reas of infrastructure such as networking and security policies, having specialists sanity check and review these changes in code.

The infrastructure codebase and pipelines used to deliver changes to production instances can be organised based on their governance requirements so a security policy change goes through a review and signoff step not necessarily required for changes to less sensitive areas.

Make small changes and continually apply these changes through a pipeline. Refactor your infrastructure code in order to create seams. Feature toggles are a powerful technique to introduce changes. 

When making a change, consider expanding your API surface to accommodate the new functionality, shift traffic over to the new resources and then remove old resources.

Plan for failure. Make up a rudimentary plan where you address what you can come up with. Little is better than nothing. Then iterate.