Terraform Modules The Right Way

Introduction

Terraform is a powerful tool for managing infrastructure as code.
This article outlines best practices for designing Terraform modules and automating testing for Terraform.

Modules

A module is a container for multiple resources that are used together.
Key considerations while designing Terraform modules include:
- breaking down complex modules into standalone submodules
- including a README.md, CHANGELOG.md, versions.tf, variables.tf, and outputs on each module
- using namespaces for resources
- locking down Terraform and provider versions.

Automated Testing for Terraform

Terraform testing can be broken down into several types: static analysis, unit tests, integration tests, and cleanup tests.
Best practices for Terraform testing include:
- using remote state with versioning and locking
- using workspaces for multiple environments
- never saving TF state files in git
- including tests for each module
- adding an examples folder to use modules
- including in git pre-commits
- including in CI workflows.

What are Modules?

A module is a container for multiple resources that are used together. You can use modules to create lightweight abstractions, so that you can describe your infrastructure in terms of its architecture, rather than directly in terms of physical objects.

Key Considerations while designing terraform module.

Untitled

Terraform Module Structure

variables.tf: Module inputs
outputs.tf: Outputs of module and other modules that can use
README.md: Module documentation and use cases
CHANGELOG.md: Change logs and upgrade guides
examples: Use cases and examples for module usage
tests: for writing go tests with terratests
submodule: breakdown the complex module

$ tree complete-module/
.
├── README.md
├── main.tf
├── variables.tf
├── outputs.tf
├── ...
├── modules/
│   ├── nestedA/
│   │   ├── README.md
│   │   ├── variables.tf
│   │   ├── main.tf
│   │   ├── outputs.tf
│   ├── nestedB/
│   ├── .../
├── examples/
│   ├── exampleA/
│   │   ├── main.tf
│   ├── exampleB/
│   ├── .../

Untitled

Module releases

Use semantic versioning to release module.

Untitled

MAJOR: when you make incompatible API changes

MINOR: when you add functionality backward compatible

PATCH : when you make backward-compatible bug fixes

Module destructuring

Untitled

Module Creation - Recommended Pattern | Terraform | HashiCorp Developer

Monolithic Terraform Issues

one/several huge files
hard to find/debug
guess work
slower development cycles

Best Practices while creating a module

Include README.md , Changelog.md , versions.tf , variables.tf and outputs on each modules.
Breakdown complex modules into standalone submodules
Namespace all your resources.
separate modules, tests.
Lockdown terraform and provider versions.
TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache to reuse caches
Use null attribute when the field will be empty

Untitled

Include tests for each module.

Untitled

Add an examples folder to use modules.

Automated testing for Terraform

Untitled

Test types

Untitled

Static analysis
- Compiler / parser / interpreter
  - terraform validate
  - VS Code:
    - https://marketplace.visualstudio.com/items?itemName=MadsKristensen.Terraform
- Linter
  - conftest
  - terraform_validate
  - tflint
- Dry run
  - terraform plan
  - hashicorp sentinel
  - terraform-compliance
Unit Tests There is no pure unit testing for infrastructure code. Terratest deploy and undeploy Example : https://github.com/gruntwork-io/infrastructure-as-code-testing-talk
- Deploy
- Validate
- Undeploy
Other ways to validate
- Web services: web requests
- server: terratest ssh package
- cloud: terratest, cli ,api
- DB: SQL queries
Integration
- test parallelism
- test stages
- test retries
CleanUp
- cloud-nuke
- aws-nuke

Untitled

Terraform Best Practices

– Use remote state with versioning and locking; – Use workspace for multiple environments; – Use for_each instead of count if it’s possible; – Never save TF state files in git, they can contain sensitive information in plain text format; – Use modules for code reuse (DIY);

Integrating with workflows

Include in git precommits
- https://github.com/antonbabenko/pre-commit-terraform
  - checkov required for checkov hook.
  - terraform-docs required for terraform_docs hook.
  - terragrunt required for terragrunt_validate hook.
  - terrascan required for terrascan hook.
  - TFLint required for terraform_tflint hook.
  - TFSec required for terraform_tfsec hook.
  - infracost required for infracost_breakdown hook.
  - jq required for terraform_validate with -retry-once-with-cleanup flag, and for infracost_breakdown hook.
  - tfupdate required for tfupdate hook.
  - hcledit required for terraform_wrapper_module_for_each hook.
  - env0 to run terraform and terragrunt workflows
Include in CI workflows

Demo

Example : https://github.com/ashokpokharel977/terraform-aws-vpc

DevOps production readiness checklist: https://gruntwork.io/devops-checklist/

Conclusion

Terraform is a powerful tool, but it must be used correctly to be effective.
By following these best practices for designing Terraform modules and automating testing, you can ensure that your infrastructure will be reliable and secure.