Best Practices and Troubleshooting

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production; see the section bibliography for the reference material consulted while preparing these pages.

Most of this section explains one feature at a time. This page is the opposite: it gathers the habits that experienced teams converge on, shows how to carve up an address space without arithmetic mistakes, explains how to turn on Terraform’s own logs when something goes wrong, and lists the errors that people meet most often together with what usually causes them. Each topic points to the page that covers it in depth.

The baseline is Terraform 1.17 and OpenTofu 1.13, as elsewhere in the section. The running example is Bookshelf (catalog-api, reviews-api, web, PostgreSQL, Redis, a covers bucket); every name, CIDR and identifier is a placeholder. Error messages are quoted only where an official page quotes them; elsewhere they are abbreviated and marked as illustrative, because wording changes between releases.

Consolidated best practices

The sources are the HashiCorp style guide, the HCP Terraform recommended practices, and the vendor guides for AWS, Google Cloud and Azure. They agree on far more than they differ, so the list below is organised by concern rather than by source.

Code layout and style

  • Run terraform fmt and terraform validate before every commit, and add a linter such as TFLint. The style guide recommends exactly this trio; the tools are covered in Code Quality and Security Scanning.

  • Use the conventional file names: main.tf, variables.tf, outputs.tf, providers.tf, terraform.tf, locals.tf and backend.tf, and split large configurations by theme (network.tf, storage.tf, compute.tf).

  • Indent by two spaces, align the = signs of consecutive single-line arguments, put meta-arguments first, then arguments, then nested blocks, then lifecycle and depends_on.

  • Name resources with nouns and underscores, and do not repeat the resource type in the name: aws_s3_bucket.covers, not aws_s3_bucket.covers_bucket.

  • Give every variable and output a type and a description, mark secrets sensitive, use validation sparingly, and keep locals for values that are used more than once or that name an idea.

  • Use count for near-identical resources or an on/off switch and for_each when each instance needs distinct values; prefer for_each with stable keys, because a list index shifts when an element is removed.

  • Comment with #, and only to explain what the code cannot say.

The layout conventions are covered in HCL Syntax and Configuration Files and Workspaces, Environments and Project Structure.

Versions and dependencies

  • Pin providers in required_providers with a constrained range, and set required_version for the CLI. Commit .terraform.lock.hcl so every run installs the same provider builds.

  • Pin registry modules to an exact or tightly bounded version. The style guide notes that the version argument does not apply to local modules.

  • Upgrade deliberately: change the constraint, run terraform init -upgrade, read the plan, and let Dependabot or Renovate open the pull request.

See Providers and Modules.

State

  • Use a remote backend with locking and encryption; never keep production state on a laptop, and keep state files out of version control. The style guide’s .gitignore guidance is to exclude state, saved plans, the .terraform directory and sensitive .tfvars files, and to commit the code, the lock file and a README.

  • Split state by blast radius and by rate of change (network, data stores, applications), not by team convenience.

  • Do not share whole state files; expose what other configurations need through outputs, terraform_remote_state or data sources.

  • Treat state surgery (state mv, state rm, import) as a reviewed change, and prefer the declarative moved, removed and import blocks.

See State, Backends and Remote State and Refactoring with moved and removed.

Security

  • Authenticate with short-lived credentials (OIDC or dynamic provider credentials, instance or workload identity), not long-lived keys in files or variables.

  • Give the pipeline role the least privilege it needs, and use separate roles for plan and apply where the platform allows it.

  • Remember that sensitive values still land in state in plain text. Use ephemeral values and write-only arguments where the provider supports them, and a secrets manager for the rest.

  • Scan the code before apply (Checkov, Trivy) and enforce guardrails with policy as code.

See Sensitive Data, Ephemeral Values and Write-only Arguments and Policy as Code.

Workflow

  • Make version control the source of truth, keep branches short-lived, run a speculative plan on every pull request and apply only from the main branch through automation. The recommended practices describe the progression from manual changes, through semi-automation and infrastructure as code, to collaborative infrastructure as code with HCP Terraform managing the boundaries between teams and tiers.

  • Test modules (terraform test, Terratest) and run the tests in CI.

  • Never change managed infrastructure by hand. When drift happens, either reconcile it with terraform plan -refresh-only or let the next apply correct it.

See CI/CD and Automation, Testing and HCP Terraform and Terraform Enterprise.

Source: Terraform Docs — Style Guide and Terraform Docs — HCP Terraform recommended practices

Cloud vendor guidance

Cloud Guide Emphasis

AWS

AWS Prescriptive Guidance — Best practices for using the Terraform AWS Provider

Five sections: security, backends, code base structure and organization, AWS provider version management and community modules

Google Cloud

Best practices for using Terraform

Separate pages for general style and structure, root modules, reusable modules, security and testing

Azure

Microsoft Learn — Terraform on Azure

Choosing between the AzureRM and AzAPI providers, state in Azure Storage with Entra ID authentication, Azure Export for Terraform

Each vendor guide is applied in its own page: Terraform on AWS, Terraform on Google Cloud and Terraform on Azure. The Microsoft pages are tutorials and overviews rather than a single numbered best-practice document, so the Azure row summarises what that documentation set covers.

Network design with cidrsubnet

A network plan is the one piece of configuration that is expensive to change after the fact: subnets are referenced by route tables, security groups, databases and peering connections. Computing CIDR blocks by hand invites overlaps, so let Terraform do the arithmetic from a single parent block.

cidrsubnet(prefix, newbits, netnum) extends the prefix by newbits and fills the new bits with netnum; cidrsubnets(prefix, newbits...) allocates consecutive ranges of possibly different sizes. The functions are described in Functions. The rule the documentation gives for cidrsubnets is the important operational one: once addresses from the result are assigned to real infrastructure, do not change the existing newbits arguments, because that would move every later range. Append new entries instead.

The Bookshelf plan below uses a /16 for the environment and reserves a /20 block per tier, so each tier can later be split further without touching its neighbours.

variable "vpc_cidr" {
  type        = string
  description = "Parent CIDR block of the Bookshelf network."
  default     = "10.20.0.0/16"

  validation {
    condition     = can(cidrnetmask(var.vpc_cidr))
    error_message = "vpc_cidr must be a valid IPv4 CIDR block."
  }
}

locals {
  azs = ["a", "b", "c"]

  # Three /20 tier blocks (4096 addresses each): public, private and data.
  # Append new tiers at the end; never reorder or edit existing entries once deployed.
  tier_blocks = cidrsubnets(var.vpc_cidr, 4, 4, 4)
  tiers = {
    public  = local.tier_blocks[0]
    private = local.tier_blocks[1]
    data    = local.tier_blocks[2]
  }

  # One /24 per tier and availability zone, carved out of each tier block.
  subnets = merge([
    for tier, block in local.tiers : {
      for i, az in local.azs : "${tier}-${az}" => cidrsubnet(block, 4, i)
    }
  ]...)
}

output "subnet_cidrs" {
  value = local.subnets
}

With these inputs the tier blocks are 10.20.0.0/20, 10.20.16.0/20 and 10.20.32.0/20, and the first public subnet is 10.20.0.0/24. Verify any plan with terraform console before applying it.

Source: Terraform Docs — cidrsubnets function and Terraform Docs — cidrsubnet function.

Design guidelines that follow from this:

  • Choose the parent block so it does not overlap any network you will ever peer or connect with (other environments, on-premises ranges, other clouds). Overlaps cannot be fixed by routing.

  • Leave unallocated space. A /16 split into three /20 blocks uses less than a fifth of the space on purpose.

  • Keep tiers in separate blocks so security rules can match a whole tier with one prefix.

  • Use for_each over a map of names to CIDRs, so removing one subnet does not renumber the others; the pitfalls of count are in Meta-arguments.

  • Provider specifics (subnets per zone on AWS, regional subnets on Google Cloud, delegated subnets on Azure) are in the three cloud pages.

Debugging with logs

Terraform has two sources of diagnostic information: the error messages it prints, and a detailed log that is off by default. The log is controlled by environment variables documented on the Debugging Terraform page.

Variable Meaning

TF_LOG

Setting it to any value turns on detailed logs on stderr. The levels, from most to least verbose, are TRACE, DEBUG, INFO, WARN and ERROR. The value JSON logs at TRACE level in a parseable JSON format, which the page says is not a stable interface

TF_LOG_CORE

Enables logging for Terraform core only, with the same levels

TF_LOG_PROVIDER

Enables logging for provider plugins only, with the same levels

TF_LOG_PATH

Appends the log to a file. TF_LOG must still be set for any logging to happen

A typical session isolates the layer that misbehaves:

# Everything, to a file (placeholder path)
TF_LOG=TRACE TF_LOG_PATH=./terraform-trace.log terraform plan

# Only the provider (HTTP calls and API errors), at debug level
TF_LOG_PROVIDER=DEBUG TF_LOG_PATH=./provider-debug.log terraform apply

# Only core (graph walk, state handling)
TF_LOG_CORE=DEBUG TF_LOG_PATH=./core-debug.log terraform plan

Source: Terraform Docs — Debugging Terraform

Practical advice:

  • Start at DEBUG; TRACE is large and rarely needed. Use TF_LOG_PROVIDER for authentication, throttling and API errors, because the provider logs the requests it makes, and TF_LOG_CORE for graph, state and evaluation problems.

  • Treat log files as secrets. Provider logs can contain request and response bodies; the CI page warns against publishing TRACE logs where many people can read them. Upload the file as an artifact only on failure, with restricted access (see CI/CD and Automation). Unset the variables afterwards: unset TF_LOG TF_LOG_CORE TF_LOG_PROVIDER TF_LOG_PATH.

  • When filing a bug, the Debugging page asks you to include the detailed log, for example shared through a gist service, after removing anything sensitive.

  • OpenTofu reads the same TF_LOG variables; the binary is tofu.

  • Provider authors can add their own subsystem variables; see Provider Development.

Crash logs

If Terraform core panics, the CLI writes a crash log, normally named crash.log, in the working directory and tells you to report the problem. A crash log contains a stack trace and can include configuration or state fragments, so review it before sharing. The file name is the reason that the .gitignore templates in this section list crash.log and crash.*.log (see Getting Started). A panic is always a bug, never a configuration mistake; the fix is to upgrade, and to report it with the crash log and a minimal configuration that reproduces it. The Debugging page does not describe crash logs, so check the file name and the conditions that produce one on the version you run.

Narrowing a problem

  1. Run terraform validate first; it needs no credentials.

  2. Run terraform plan -refresh=false to separate configuration problems from provider read problems.

  3. Use terraform console to evaluate an expression in the context of the current state.

  4. Use terraform state list and terraform state show ADDRESS to see what Terraform believes exists.

  5. Use terraform graph to understand ordering and cycles (see Dependency Graph and Plan).

  6. Reproduce with the smallest configuration possible; most bugs disappear or become obvious when everything else is removed.

The commands are in CLI Essentials.

Common errors and fixes

The messages below are abbreviated and illustrative unless stated; search for the key phrase, not the whole text.

Symptom Usual cause Fix

An "Error acquiring the state lock" message

Another run holds the lock, or a crashed runner or killed process left it behind

Wait for the other run. If you are certain nothing is running, terraform force-unlock LOCK_ID with the ID printed in the message. See Backends and Remote State

The lock file does not match the installed providers ("Inconsistent dependency lock file")

.terraform.lock.hcl lists different versions or platforms than the constraints or the machine need

Run terraform init -upgrade after changing constraints; commit the updated lock file; run terraform providers lock for every platform the team and CI use

Provider not found or version constraint not satisfied during init

Wrong source address, constraint that excludes every release, or a registry or mirror that cannot be reached

Check required_providers source and version, network access and any provider_installation configuration

Authentication or permission errors (HTTP 401 or 403)

Missing or expired credentials, wrong role, wrong subscription or project

Reproduce with the cloud CLI, then check the identity Terraform actually uses with TF_LOG_PROVIDER=DEBUG. See the three cloud pages

"Invalid for_each argument" or "Invalid count argument", mentioning values that cannot be determined until apply

The collection depends on an attribute only known after apply

Build keys from values known at plan time (names, variable inputs), not from generated IDs; see Meta-arguments

A "Cycle" error

Two objects reference each other, directly or through a chain

Break the loop, usually by removing the unnecessary reference or moving one attribute to a separate resource; terraform graph -draw-cycles helps. See Dependency Graph and Plan

"Instance cannot be destroyed"

prevent_destroy is set on the resource

Remove the argument deliberately, or change the plan; see Lifecycle, Validation and Checks

A plan wants to replace a resource you did not touch

An argument that forces replacement changed, a rename without a moved block, or a provider upgrade

Read which attribute is marked as forcing replacement; add a moved block for renames; use ignore_changes only for values managed elsewhere

A resource exists in the cloud but Terraform wants to create it

It was created outside Terraform, or state was lost

Bring it under management with an import block; see Import and Query

Plan shows changes every run

Drift, an attribute normalised by the API, or a non-deterministic value such as timestamp()

Run terraform plan -refresh-only to inspect drift; use plantimestamp() or ignore_changes where appropriate

A "Provider produced inconsistent result after apply" message

A provider bug

Upgrade the provider, retry, and report it with a debug log; it is not a mistake in your configuration

Backend initialisation errors after changing the backend block

The backend configuration changed

terraform init -reconfigure, or -migrate-state to move existing state

Variables not allowed in a backend block

Backend configuration is read before variables are evaluated

Use partial configuration with -backend-config; see Workspaces, Environments and Project Structure

Sensitive value appears in output or logs

Output not marked sensitive, or logs enabled in CI

Mark the output sensitive, disable TF_LOG, and rotate the exposed secret

Source: Terraform Docs — References to values and the Terraform CLI command pages cited in the pages linked from the table, for example terraform force-unlock and terraform init.

Reading a plan before applying

Most expensive mistakes are visible in the plan, and unnoticed. Check the summary line for unexpected destroy or replace counts, look for the "forces replacement" annotation on the attributes, look at what is (known after apply) and ask whether it should be, and read changes to IAM, network rules and anything with data. Save the plan (-out) and apply that file, so what was reviewed is what runs.

Review checklist

  • terraform fmt -check, validate, TFLint and a security scanner pass in CI.

  • Providers, modules and the CLI are pinned; the lock file is committed.

  • State is remote, locked, encrypted and split by blast radius.

  • No credentials, account IDs or state files in the repository.

  • Every variable and output has a type and a description; secrets are sensitive.

  • Network ranges come from one parent block and leave room to grow.

  • A saved, reviewed plan is what gets applied, and only by the pipeline.

  • Log variables are unset in routine runs.

References