Terraform Best Practices: Lessons from Real-World Team Projects

Learning Terraform the hard way

I've made most of the Terraform mistakes so you don't have to. Corrupted state files at 2 AM, modules that nobody could understand, CI/CD pipelines that deployed to production when they shouldn't have.

These practices come from fixing those mistakes.

Terraform is HashiCorp's declarative infrastructure-as-code tool: you describe the cloud resources you want in configuration files, Terraform computes a plan to reach that state, and a state file tracks what has actually been deployed so later runs can detect drift. Everything below assumes a team sharing that state, not one person running Terraform solo. The failure modes that matter show up when five engineers touch the same infrastructure, not when one person does.

Project structure that scales

The monolith trap

Many teams start with everything in one file. This works until you have 50+ resources, then it becomes unmanageable.

Better: environment separation

terraform/
├── environments/
│   ├── dev/
│   ├── staging/
│   └── prod/
├── modules/
│   ├── networking/
│   ├── compute/
│   └── database/
└── global/
    ├── iam/
    └── dns/

Each environment has its own state file. Changes to dev can't accidentally affect prod.

Separate state per environment also enables separate credentials per environment, which matters more than it sounds. The CI identity that applies dev doesn't need permission to touch anything in the prod subscription or account. If one set of long-lived credentials can apply to every environment, a leaked token or a compromised CI runner turns into an org-wide incident instead of a dev-only one.

Module design

Good modules are:

• Single purpose: one module, one job
• Configurable: expose what needs to vary
• Documented: README with examples
• Versioned: tag releases

A real module layout

A module tree is one thing; the files inside a single module are what people actually maintain day to day. A module meant to be reused across environments, and possibly across repos, should look like this:

modules/networking/
├── main.tf          # resources
├── variables.tf     # inputs: type, description, validation, sensitive flag
├── outputs.tf       # values other modules or root configs consume
├── versions.tf      # required_version + required_providers with version pins
└── README.md        # usage example, inputs/outputs table

`versions.tf` is the file teams skip, and the one that causes the most confusing bugs six months later. Pin both the Terraform core version and every provider version explicitly:

terraform {
  required_version = ">= 1.7.0"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.90"
    }
  }
}

Without that pin, a `terraform init` run months from now can silently pull a new major provider version with different defaults or breaking changes, and the first sign of trouble is a plan that wants to destroy and recreate half your infrastructure.

Pinning module sources against supply-chain drift

The same logic applies to where a module comes from, not just which provider version it uses. A module source like a Git URL with ref=main, or a registry reference with no version constraint, pulls whatever is on the default branch or latest tag at apply time. If someone (or something, in a compromised-dependency scenario) pushes a change to that branch between your last code review and your next terraform apply, that change gets deployed without ever going through review. Pin every module source to an explicit, immutable version:

# Registry module, pinned to a version range
module "vpc" {
  source  = "app.terraform.io/my-org/vpc/azurerm"
  version = "~> 4.2.0"
}

# Git-sourced module, pinned to a tag, not a branch
module "vpc" {
  source = "git::https://github.com/my-org/tf-modules.git//vpc?ref=v1.4.0"
}

Treat a module version bump the same way you'd treat a dependency bump in application code: read the changelog, run it through terraform plan in a non-production workspace first, and don't let ref=main or an unconstrained registry source reach a production configuration. HashiCorp's own reference on module sources and versioning covers the syntax for both registry and Git sources.

State management

Remote state is non-negotiable

Local state files cause team conflicts, no locking, and no backup. Use a remote backend with built-in state locking, such as S3 with DynamoDB for AWS or Azure Storage (native blob leasing) for Azure, so two engineers can never run apply against the same state at once.

State file security

Your state file contains sensitive data:

• Enable encryption at rest
• Restrict access to state bucket
• Enable versioning for recovery
• Never commit state to Git

These four points are the floor, not the ceiling. If you're on Azure, the full lockdown, Managed Identity auth instead of static storage keys, private endpoints on the storage account, RBAC scoped to a single container, and blob versioning for recovery, is its own topic with working Terraform code: see How to secure Terraform remote state in Azure Storage account.

Least-privilege provider credentials

The identity Terraform authenticates as when it runs apply deserves the same scrutiny as the state backend. A service principal with Owner at the subscription root, or an IAM user with AdministratorAccess, is the path of least resistance when you're setting up CI, and the path of maximum blast radius the day that credential leaks. Scope the identity to exactly the resource groups, subscription, or account that environment's Terraform actually manages, and keep that scope separate per environment as covered above.

Where the provider supports it, skip static credentials entirely. GitHub Actions can authenticate to Azure via OpenID Connect (azure/login with client-id, tenant-id, and subscription-id, no client-secret stored anywhere) and to AWS the same way (aws-actions/configure-aws-credentials with role-to-assume). The token is minted per run and expires when the job ends, so there is no long-lived secret sitting in a repo, a vault, or a pipeline variable for someone to steal.

Writing better Terraform code

Use variables wisely

Don't hardcode values. Use variables with descriptions, defaults, and validation rules.

Variable defaults and validation rules belong in variables.tf. Actual per-environment values belong in .tfvars files, but a .tfvars file is not a safe place for a secret. A terraform.tfvars committed to Git with a database password in it is the same mistake as hardcoding that password in a resource block, and Git history remembers it long after the line is deleted. Mark sensitive inputs with sensitive = true so Terraform redacts them from plan and apply output, and pull the actual value from a secret manager at runtime instead of storing it as a plain variable.

Data sources over hardcoding

Look up values dynamically instead of hardcoding AMI IDs or other values that change.

Meaningful resource names

"sg1" tells you nothing. "web_app_alb_security_group" tells you everything.

CI/CD for Terraform

Basic pipeline

1. Validate and format check on all branches
2. Plan on pull requests
3. Apply on main with required approval

For teams using Azure DevOps, our Azure DevOps Pipelines guide walks through setting up the full CI/CD workflow with approval gates, environment deployments, and service connections for Azure.

Security scanning

The tool landscape here shifted: tfsec was folded into Trivy, so "add tfsec" is dated advice on its own. Checkov, Trivy, and Terrascan are the three tools teams actually choose between today, and they don't flag the same things on the same codebase. We compared all three head-to-head, including a real case where a suppressed check silently stopped protecting a production storage account for months: see Terraform security scanning: Checkov vs Trivy vs Terrascan compared.

Policy as code: gating applies before they run

Security scanning checks the .tf files themselves. Policy as code checks the plan, the actual set of resource changes Terraform is about to make, against rules your org sets, and it can hard-block terraform apply if a change violates one. That distinction matters: a variable that only evaluates to a risky value in one environment's .tfvars file can pass every static scan and still produce a plan that opens a storage account to the public internet.

HashiCorp Cloud and Enterprise ship this as Sentinel, evaluated automatically between plan and apply. The open-source equivalent is Open Policy Agent, run via conftest against the plan's JSON output:

terraform plan -out=tfplan
terraform show -json tfplan > plan.json
conftest test plan.json

package terraform.policies

deny[msg] {
  resource := input.resource_changes[_]
  resource.type == "azurerm_storage_account"
  resource.change.after.allow_nested_items_to_be_public == true
  msg := sprintf("%s must not allow public blob access", [resource.address])
}

Wire the conftest test step in as a required CI check that has to pass before the apply job is even allowed to run, the same way you'd require tests to pass before a merge. For the broader pattern of wiring static analysis, dependency scanning, and policy gates into a pipeline beyond just Terraform, see DevSecOps: how to integrate security into your CI/CD pipeline.

Common mistakes

1. Ignoring drift

Run terraform plan regularly to detect manual changes.

2. Not using workspaces correctly

Workspaces are for temporary variations, not environments.

3. Storing secrets in variables

Never put secrets in code. Fetch from secret managers at runtime.

4. Massive blast radius

If one terraform apply can break everything, split into smaller configurations.

5. Long-lived, overly broad provider credentials

This shows up in practice when a team reuses one Owner-scoped service principal across every environment because setting up separate identities per environment felt like unnecessary overhead early on. It isn't overhead you skip for free forever: the day that credential leaks, a misconfigured CI log, a compromised dependency, a phished laptop with a .env file still on it, the blast radius is every environment at once instead of one.

Two tradeoffs every growing team hits

Monorepo vs multi-repo Terraform

Once you have more than a couple of modules and more than one team touching infrastructure, where the code lives becomes a real decision, not a default.

Workspace vs directory-per-environment

Terraform workspaces and separate directories both let you run the same configuration against multiple environments, but they solve different problems, which is why using workspaces for permanent environments is common mistake #2 above.

Team workflow: who can plan, who can apply

The technical controls above only hold if the workflow around them enforces the same rules. A practitioner checklist for the workflow itself:

• Every change goes through a pull request. No direct commits to the branch CI applies from.
• terraform plan runs automatically on every PR and posts the plan output where reviewers can see it, so review covers the actual infrastructure diff, not just the code diff.
• At least one reviewer who isn't the PR author approves before merge. Production-affecting modules require two.
• terraform apply runs only from the protected branch, only in CI, never from a developer's laptop against the shared backend.
• Applying to production requires a separate, explicit approval gate after merge (GitHub Environments protection rules, Azure DevOps approval checks, or Terraform Cloud apply confirmation), so merging code and authorizing an infrastructure change are two distinct decisions.
• Internal modules published to a registry (Terraform Cloud's private registry, an internal Git server, or Azure DevOps Artifacts) get semantic version tags, a changelog entry, and a README with an inputs/outputs table before anyone downstream is allowed to consume the new version.
• Any apply run outside this process, a documented break-glass procedure for genuine incidents, gets logged and reviewed after the fact, not silently absorbed as normal practice.

Quick reference: team guidelines

Frequently asked questions

What are the most important Terraform best practices for teams?

The three highest-impact practices for team Terraform environments are remote state with locking (prevents concurrent apply conflicts), environment separation with separate state files per environment (dev, staging, prod), and mandatory PR review with a CI plan step before merge. These three practices together prevent the most common categories of production incidents in IaC-managed environments.

Why should Terraform state not be stored in Git?

Terraform state files contain sensitive data including resource IDs, IP addresses, connection strings, and any secrets that Terraform touched during provisioning. State files are JSON and are stored in plaintext unless encrypted separately. Committing state to Git exposes secrets in version history, creates merge conflict issues when multiple engineers apply simultaneously, and provides no locking mechanism to prevent concurrent state corruption.

What is Terraform state locking and why does it matter?

State locking prevents multiple Terraform processes from modifying the state file simultaneously, which would cause corruption. When using a remote backend like Azure Storage with blob leases or AWS S3 with DynamoDB locking, Terraform acquires a lock before any operation that modifies state and releases it when done. Without locking, two engineers running terraform apply at the same time can interleave writes and corrupt the state file, requiring manual recovery.

Should I use Terraform workspaces for environment separation?

Terraform workspaces are designed for temporary variations on the same configuration, not for permanent environment separation. Using workspaces for dev, staging, and prod environments is a common mistake: it shares a single backend configuration, makes it easy to accidentally apply against the wrong environment, and provides no access control separation between environments. The correct approach is separate directories with separate state backends per environment.

How do I handle secrets in Terraform without hardcoding them?

Never put secrets in Terraform variables, .tfvars files, or any file committed to version control. Use a secrets manager: in Azure, reference Key Vault secrets using a data source at runtime; in AWS, use Secrets Manager or Parameter Store data sources. For CI/CD pipelines, inject secrets as environment variables from your secrets management system (GitHub Actions secrets, Azure Key Vault references in Azure DevOps) rather than storing them in pipeline YAML files.

How do I stop an unpinned module from silently changing what gets deployed?

Pin every module source to an explicit version: a version constraint for registry modules, an immutable tag (not a branch) for Git-sourced modules. Never leave a module source pointing at ref=main or a registry reference with no version constraint, since either one pulls whatever the latest commit or release happens to be at apply time, bypassing whatever review your last plan went through. Treat a module version bump like a dependency bump in application code: read the changelog and run a plan in a non-production workspace before rolling it into production.

What is the difference between Terraform security scanning and policy as code?

Security scanning tools like Checkov, Trivy, and Terrascan analyze the .tf configuration files themselves for known-bad patterns before anything is applied. Policy as code, using Sentinel or Open Policy Agent, evaluates the actual plan output, the resolved set of resource changes Terraform is about to make, against organizational rules, and can block terraform apply outright. A static scan can miss a risky value that only appears in one environment's .tfvars file; a policy check against the plan catches it because it sees the resolved values, not just the source code.