Infrastructure Drift: How to Detect It and What to Do About It

The drift problem

You've got beautiful Terraform code, well-organized modules, everything documented. Then someone makes a "quick fix" in the AWS console, and suddenly your code doesn't match reality.

That's drift. It starts small and grows until you have no idea what's actually running.

Infrastructure drift is the gap between what your Terraform, CloudFormation, or Bicep code describes and what actually exists in your cloud account. It happens any time a resource is created, changed, or deleted outside your infrastructure-as-code (IaC) pipeline, and left unnoticed, it grows until an outage, an audit, or an attacker finds it first.

Most teams treat drift as an operations hygiene problem: annoying, but not dangerous. That's the wrong lens for a security team. Drift is exactly how misconfigurations get back into production without review: a security group rule added through the console during an incident, a public S3 bucket toggled open while someone debugged an access problem, a firewall rule loosened "just for a quick test." None of these go through the review or policy checks your IaC pipeline enforces. They sit there, invisible to your code, until someone finds them: ideally an auditor, not an attacker.

Why drift happens

The usual suspects

1. Emergency fixes: Production is down, someone fixes it manually
2. Console convenience: It's faster to click than to write code
3. Automated processes: Auto-scaling modifies resources
4. Service integrations: AWS services create resources on your behalf
5. Lack of access control: Too many people with console access
6. Incident response shortcuts: A security rule gets loosened or an alert gets silenced to unblock troubleshooting, and nobody schedules the follow-up to reverse it

The cost of drift

• Security gaps: A manually added ingress rule or a public storage toggle bypasses the IaC review that would normally catch it, sometimes for months before anyone notices
• Outages: Terraform destroys manually-created resources it doesn't recognize, or fails applying against a state that no longer matches reality
• Compliance failures: Auditors find undocumented changes that nobody can explain, who made them, when, or why
• Lost time: Engineers spend hours debugging why staging and production behave differently, when the answer is drift nobody tracked

Why drift is a security problem, not just an ops problem

Security group and NSG rules are the clearest example. During an incident, someone opens port 22 or 3389 to 0.0.0.0/0 in the console to get emergency access, fixes the problem, and moves on. The rule stays. It never appears in a pull request, never gets a peer review, and never shows up in a Terraform diff unless someone runs a plan. Attackers scan for exactly this kind of rule constantly, and it bypasses every control your IaC pipeline was built to enforce.

Storage buckets follow the same pattern. An S3 bucket's block-public-access setting gets toggled off to unblock a migration or a support ticket, and the follow-up ticket to turn it back on gets deprioritized. Nothing in your code or PR history shows it. The bucket sits there, publicly readable, until a scanner (yours or someone else's) finds it.

The same goes for firewall rules loosened "just for a quick test," IAM policies widened to unblock a deploy, or a WAF rule disabled to rule out a false positive. Individually these look like small, defensible judgment calls. Across a team making dozens of them a month, they are how a well-reviewed baseline turns into an unreviewed, undocumented attack surface. That is why drift detection belongs on a security team's radar, not just a platform team's.

Detecting infrastructure drift

Terraform plan in CI, on a schedule

The baseline detection method costs nothing beyond compute time: run terraform plan -detailed-exitcode on a schedule, not just before deploys. Its exit code tells you what happened without parsing output: 0 means no changes, 1 means an error, 2 means Terraform found a difference between state and reality. A scheduled pipeline, every few hours for security-sensitive environments, nightly at minimum elsewhere, that fails the build on exit code 2 turns drift from something you discover during an incident into something you get paged for the same day. See our Terraform best practices for teams guide for structuring state and modules so scheduled plans stay fast and readable instead of thousand-line diffs nobody opens.

For teams on Azure DevOps, our Azure DevOps Pipelines guide covers how to configure the cron trigger and wire up Slack or Teams alerts so a drifted plan doesn't sit in a build log nobody checks.

driftctl and other drift-detection tools

driftctl, now maintained by Snyk as an open-source project, compares live cloud state directly against Terraform state and flags unmanaged or changed resources without a full terraform plan run, useful as a lightweight, read-only check you can run more often than your CI schedule. Firefly, env0, and Spacelift go further with hosted drift dashboards, historical tracking, and one-click remediation, at the cost of being commercial platforms rather than a CLI you drop into an existing pipeline. A five-person platform team is usually better served by scheduled terraform plan plus driftctl than by standing up a new SaaS platform.

Cloud-native drift and compliance tools

AWS Config continuously records configuration changes to your resources and evaluates them against rules you define, independent of whether the change came from Terraform, the console, or another AWS service. The managed rule cloudformation-stack-drift-detection-check catches drift on CloudFormation stacks specifically, while custom Config rules can flag a security group with an open ingress rule or a public storage bucket regardless of what created it, since Config evaluates the resource itself rather than a change history.

Azure Policy re-evaluates resource compliance against your policy definitions automatically every 24 hours, and faster (around 15 minutes) after a Resource Manager change. That gives you drift detection for resource-level configuration, NSG rules, storage account settings, tag enforcement, without extra tooling. For settings inside a VM's operating system rather than the Azure resource wrapper around it, Azure Machine Configuration (formerly Guest Configuration) extends the same audit-and-remediate model to OS-level drift. Our Azure Policy vs Defender for Cloud comparison breaks down which of Microsoft's two overlapping compliance tools owns which layer of this.

Remediation playbook: detect, triage, decide, prevent

Finding drift is the easy part. What separates a team that stays ahead of it from one that lets it accumulate is a repeatable process for what happens after the alert fires.

Step 1: Detect

Use one or more of the methods above: scheduled terraform plan, driftctl, or your cloud provider's native compliance evaluation. The goal at this stage is a reliable, low-noise signal that something changed outside your pipeline, not a perfect one.

Step 2: Triage severity

Not all drift deserves the same urgency. Score it before deciding what to do:

• Critical: anything that changes your security posture, an open ingress rule, a public storage bucket, a widened IAM policy, a disabled logging or WAF control. Page someone.
• High: capacity or availability-relevant changes, a resized instance, a changed autoscaling limit, a modified load balancer target group.
• Medium: configuration that affects cost or operability but not security or uptime, a changed instance type, an unexpected tag.
• Low: cosmetic or provider-managed drift, default tags a cloud provider adds automatically, computed fields that always show as changed. Usually safe to suppress in your detection tooling rather than triage every run.

Sorting into these tiers before deciding what to do keeps a genuinely dangerous change (the open security group) from sitting in the same queue as a tag nobody cares about.

Step 3: Decide: revert, adopt, import, or drop

You have four ways to resolve drift once you know its severity. Which one is right depends on whether the manual change was a mistake or a legitimate update your code has not caught up to yet.

• Revert: run terraform apply to restore the code-defined state. Use this when the manual change was unauthorized, a mistake, or security-relevant, like the open ingress rule from the earlier example. This can cause downtime if the manual change was actually needed, so check with whoever made it first.
• Adopt: update your Terraform code to match the new reality, then verify with terraform plan. Use this when the manual change was a legitimate, needed fix that your code simply has not been updated to reflect yet.
• Import: run terraform import to bring an unmanaged resource under management. Use this when a resource exists that should have been created by Terraform in the first place, like something a teammate created directly instead of through a PR.
• Drop: run terraform state rm to stop managing a resource that belongs elsewhere. Use this when a resource is legitimately owned by another team, tool, or automation, and Terraform tracking it was the actual mistake.

Step 4: Remediate and verify

Make the change, then confirm it worked: terraform plan should come back clean, no changes, for anything you reverted, adopted, or imported. Do not consider drift closed until the plan is clean. A "probably fixed" resolution is exactly how the same drift reappears in next month's scan.

Step 5: Prevent recurrence

Every remediation should end with one question: what stops this from happening again? If the answer is "nothing," you have fixed the symptom and left the cause in place.

Preventing future drift

Technical controls

• Restrict console access using IAM policies, most engineers shouldn't have write access outside of break-glass roles
• Enforce tags that identify IaC-managed resources, so drift tooling and cost reports can tell managed from unmanaged at a glance
• Use Service Control Policies (SCPs) or Azure Policy deny effects to block specific risky actions outright, not just detect them after the fact
• Add policy-as-code checks (Open Policy Agent, Sentinel, or a custom Conftest rule) to CI so a risky change gets caught in the pull request, before it ever reaches drift detection

A landing zone baseline is where most of this technical control work actually lives in practice: guardrails, deny policies, and access boundaries defined once at the subscription or account level instead of per-resource. Our Azure landing zone security baseline walks through building that baseline for an Azure environment.

Process controls

• Document break-glass procedures, so emergency console access has an actual process instead of "whoever has admin rights"
• Require PR review for all changes, no exceptions for "quick" ones
• Conduct regular drift audits, on a calendar, not "whenever someone remembers"
• Train the team on why IaC matters, most console-driven drift is not malicious, it's someone who doesn't understand what breaks downstream
• Assign an owner for the drift detection output itself, not just the pipeline that produces it

Common failure modes

Drift detection running but nobody reads the output

The single most common failure mode is not a missing tool, it is a tool nobody is accountable for. A scheduled terraform plan posting to a Slack channel with forty other automated messages a day gets scrolled past. Within weeks, drift alerts join the noise the team has learned to ignore, and by the time someone looks, the queue has months of unreviewed changes. If a drift alert has no named owner and no response SLA, assume it is not actually being monitored, whatever the pipeline dashboard shows.

Break-glass changes that never get reconciled

Break-glass access exists because production sometimes needs a fix faster than a PR and review cycle allows. The failure is not granting that access, it is not requiring a follow-up. Without a mandatory ticket to codify the change in IaC (24 to 48 hours is a common deadline), the emergency fix becomes permanent, undocumented drift, indistinguishable from the security-relevant kind described earlier. Break-glass access should auto-create a tracked follow-up the moment it is used, not rely on someone remembering to file one.

The tradeoff: strict IaC enforcement vs emergency access

Strict IaC-only enforcement

Blocking console write access entirely, via SCPs, Azure Policy deny effects, or IAM permission boundaries, eliminates most drift at the source. Every change goes through a PR, gets reviewed, and lands in version control. The cost is response time during incidents: if production is down and the fix needs a change no existing Terraform variable supports, the team waits on a full IaC change cycle while the outage continues.

Allowing emergency console access

Granting a small number of engineers time-boxed break-glass roles, ideally via a PAM tool with just-in-time elevation rather than standing admin access, keeps incident response fast. The cost: every use of that access is, by definition, drift, and it only stays safe if the reconciliation process above is actually enforced, not just documented.

In practice this is not one policy for the whole account. High-blast-radius environments, production payment infrastructure, anything holding regulated data, are worth the strict version even with slower incident response, backed by a well-rehearsed break-glass process for true emergencies. Lower-stakes environments, internal tooling, dev and test, can reasonably favor speed. The mistake is not choosing consciously, ending up strict where it barely matters while production stays wide open by default.

Key takeaways

• Drift is inevitable; detecting it quickly is what matters, and for security-relevant resources, quickly means hours, not weeks
• Automated scanning should run at least daily, more often for anything internet-facing or handling regulated data
• Prevention through access control is better than detection, but only if break-glass exceptions come with a mandatory reconciliation step
• Severity triage matters as much as detection: an open security group and a stray tag are not the same finding
• Document everything: who changed it, when, and why, both for compliance and for your own future self debugging the next incident

Zero drift is unrealistic. Quick detection, honest severity triage, and consistent remediation? That's achievable.

Frequently asked questions

What is infrastructure drift in Terraform?

Infrastructure drift occurs when the actual state of cloud resources diverges from the desired state defined in your Terraform code. It happens when engineers make manual changes in the console, when automated processes modify resources, or when Terraform applies partial changes due to errors. It's detected by running terraform plan and seeing changes it would make even though you haven't touched your code.

How do I detect Terraform infrastructure drift automatically?

Run terraform plan -detailed-exitcode on a schedule in CI/CD. Exit code 2 means drift was detected. Alert the team (Slack, Teams, email) when it fires. driftctl, Firefly, Spacelift, and env0 add reporting and remediation workflows on top; AWS Config or Azure Policy add continuous, tool-independent compliance evaluation.

How do I fix infrastructure drift in Terraform?

Four options: terraform apply to revert to match your code (may cause downtime if the change was intentional), update your code to match reality and verify with a clean plan, terraform import to adopt an unmanaged resource, or terraform state rm to drop a resource that belongs elsewhere. Triage severity first: an open security group or public bucket gets reverted immediately, not queued.

How do I prevent infrastructure drift in the first place?

Restrict console access with IAM policies and Service Control Policies so engineers cannot change resources outside Terraform. Require PR review for all changes. Use break-glass procedures for true emergencies, with a mandatory follow-up to codify the change within 24 to 48 hours. Regular automated drift detection catches whatever slips through.

What causes infrastructure drift in AWS and Azure environments?

Emergency production fixes applied directly in the console or CLI, console convenience over updating IaC, auto-scaling modifying resource counts, managed services creating resources on your behalf (ENIs, security groups, IAM roles), and incomplete Terraform runs that partially apply a change before failing.

Is infrastructure drift a security risk?

Yes. A security group rule opened during an incident, a bucket's public access toggled off during troubleshooting, or a firewall rule loosened for a test all bypass the review and policy checks a normal IaC change goes through. None of that requires a sophisticated attacker, just someone or something scanning for the exposure drift creates. Treat drift detection as a security control, not an operations hygiene task.