Docker, Kubernetes and Helm Providers

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production; see the section bibliography for the reference material consulted while preparing these pages.

Terraform is good at creating the things Kubernetes runs on: networks, clusters, node pools, identities, registries, databases. It can also drive Docker and Kubernetes themselves, because both have an API and a provider. This page covers the three providers for that job: kreuzwerker/docker (a Docker daemon, which is how the Bookshelf runs on a laptop in this section), hashicorp/kubernetes (Kubernetes API objects) and hashicorp/helm (Helm releases). It also covers the awkward part that all of them share, the two-phase apply problem: a provider that must connect to a cluster that does not exist yet. The page then says when to stop using Terraform for workloads and hand them to GitOps or Helm, and ends with creating an EKS, AKS or GKE cluster.

The baseline is Terraform 1.17 (released 2026-10-21) and OpenTofu 1.13. The provider majors observed on the registries on 2026-10-23 are kreuzwerker/docker 4.6.0, hashicorp/kubernetes 3.4.0 and hashicorp/helm 3.3.0; the cloud providers used for the cluster examples are hashicorp/aws 6.x, hashicorp/azurerm 5.x and hashicorp/google 8.x. The page follows Providers, which explains required_providers, aliases and the lock file, and Resources and Data Sources. It leans on The Dependency Graph and the Plan for the order in which Terraform evaluates things, and on State for what ends up on disk. The cloud pages that build the rest of the infrastructure are Terraform on AWS, Terraform on Azure and Terraform on Google Cloud. For the section reading path see the section landing page.

Three layers, three tools

A Kubernetes-based Bookshelf has three layers that change at different speeds, and each layer has a tool that fits it best.

Layer Examples Natural tool

Infrastructure

Network, cluster, node pools, load balancer, DNS zone, registry, database, IAM, buckets

Terraform or OpenTofu. This is the part with a lifecycle (create, resize, replace, destroy), cost and cloud APIs.

Cluster add-ons

Ingress or Gateway controller, cert-manager, metrics stack, the GitOps agent itself

Terraform with helm_release, once, to bootstrap the cluster. Often the last thing Terraform installs.

Applications

catalog-api, reviews-api, web, their ConfigMaps, Services and routes

Helm charts, Kustomize overlays or plain manifests delivered by a GitOps agent or a CI job, as described in Helm and GitOps and CI/CD.

The Kubernetes and Helm providers live in the middle of that table, and the rule of thumb on this page is to use them for the bootstrap and for objects that are really part of the infrastructure (a namespace with a quota, a service account bound to a cloud identity, a secret created from a cloud secret store), not as a replacement for the delivery pipeline of the application. A later section turns that rule of thumb into a decision table.

The Docker provider: the Bookshelf on a laptop

The kreuzwerker/docker provider talks to a Docker daemon through its API, so it works with a single local engine, with a remote engine over SSH or TLS, and with Swarm. It is the provider the earlier pages of this section use to run the Bookshelf without a cloud account. Version 4.0.0 (released 2026-04-03) moved part of the provider to the Terraform Plugin Framework and made BuildKit’s buildx the default builder; the resource names and the block syntax of the long-standing resources are unchanged.

Provider configuration

terraform {
  required_version = ">= 1.9.0, < 2.0.0"

  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 4.6"
    }
  }
}

# Local engine. The socket path is the Linux default; Docker Desktop and
# Colima use a different one, which the host argument or a context selects.
provider "docker" {
  host = "unix:///var/run/docker.sock"
}

The arguments worth knowing are:

Argument Meaning

host

The daemon address: unix:///path/to/docker.sock, tcp://host:2376/ or ssh://user@host:22.

context

The name of a Docker context. It can also come from DOCKER_CONTEXT and overrides host when set.

ssh_opts

Extra ssh flags used with an ssh:// host.

cert_path, ca_material, cert_material, key_material

TLS client material for a tcp:// host, either as a directory or as PEM content.

registry_auth

A repeatable block, one per registry, with address and either username and password, a config_file, or config_file_content. The username and password also default to DOCKER_REGISTRY_USER and DOCKER_REGISTRY_PASS. config_file takes precedence over the other fields.

disable_docker_daemon_check

Skips the connection check. The docker_registry_image data source and resource do not need a daemon, so this lets them run in an environment without one, at the price of breaking every other resource.

The config_file is read from the machine that runs Terraform, even when host points to another machine. Prefer a credential helper or config_file over literal passwords in registry_auth, and feed secrets from TF_VAR_ variables or the environment as described in Sensitive Data, Ephemeral Values and Write-Only Arguments.

A complete local Bookshelf

The following configuration starts PostgreSQL and Redis on a private network, with a named volume for the database and a health check that Terraform waits for, and then starts catalog-api against them. The catalog-api image name is a placeholder for the image built in the Docker section of this site; every other image is a public one.

terraform {
  required_version = ">= 1.9.0, < 2.0.0"

  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 4.6"
    }
  }
}

variable "db_password" {
  type      = string
  sensitive = true
  # Supply it as TF_VAR_db_password; never commit a value.
}

variable "catalog_image" {
  type    = string
  default = "ghcr.io/example/bookshelf-catalog-api:1.0.0" # placeholder registry
}

provider "docker" {}

resource "docker_network" "bookshelf" {
  name = "bookshelf"
}

resource "docker_volume" "pgdata" {
  name = "bookshelf-pgdata"
}

resource "docker_image" "postgres" {
  name         = "postgres:17-alpine"
  keep_locally = true
}

resource "docker_image" "redis" {
  name         = "redis:7-alpine"
  keep_locally = true
}

resource "docker_image" "catalog_api" {
  name = var.catalog_image
}

resource "docker_container" "db" {
  name    = "bookshelf-db"
  image   = docker_image.postgres.image_id
  restart = "unless-stopped"

  env = [
    "POSTGRES_USER=bookshelf",
    "POSTGRES_PASSWORD=${var.db_password}",
    "POSTGRES_DB=bookshelf",
  ]

  networks_advanced {
    name    = docker_network.bookshelf.name
    aliases = ["db"]
  }

  volumes {
    volume_name    = docker_volume.pgdata.name
    container_path = "/var/lib/postgresql/data"
  }

  healthcheck {
    test     = ["CMD-SHELL", "pg_isready -U bookshelf"]
    interval = "5s"
    timeout  = "3s"
    retries  = 10
  }

  wait         = true
  wait_timeout = 60
}

resource "docker_container" "cache" {
  name    = "bookshelf-cache"
  image   = docker_image.redis.image_id
  restart = "unless-stopped"

  networks_advanced {
    name    = docker_network.bookshelf.name
    aliases = ["redis"]
  }
}

resource "docker_container" "catalog_api" {
  name    = "bookshelf-catalog-api"
  image   = docker_image.catalog_api.image_id
  restart = "unless-stopped"

  env = [
    "DATABASE_URL=postgres://bookshelf:${var.db_password}@db:5432/bookshelf",
    "REDIS_URL=redis://redis:6379",
  ]

  ports {
    internal = 8080
    external = 8080
  }

  networks_advanced {
    name = docker_network.bookshelf.name
  }

  # The database container is already healthy when this one is created,
  # because docker_container.db waits for its healthcheck (wait = true).
  depends_on = [docker_container.db, docker_container.cache]
}

output "catalog_api_url" {
  value = "http://localhost:${docker_container.catalog_api.ports[0].external}"
}

A few details of the Docker resources explain why the example is written this way:

  • The container’s image is an image ID, not a name. The documentation recommends passing docker_image.<name>.image_id, so that a new image ID forces the container to be replaced. A tag such as latest alone would hide the change.

  • Health and ordering. depends_on only waits for the container to be created. With wait = true, the provider also waits (up to wait_timeout seconds, default 60) for the container to be healthy, and it fails if the image defines no health check. That is why the database has a healthcheck block of its own.

  • Pulling and keeping images. docker_image pulls on create and deletes the image on destroy unless keep_locally = true. To re-pull when a mutable tag moves, combine the docker_registry_image data source with pull_triggers, shown next.

  • Ports. external may be omitted, in which case Docker picks a free port of 32768 or higher, and the real value is available as ports[0].external.

  • Builds. docker_image also has a build block (with a context path) for building from a Dockerfile; the documentation notes that it requires the containerd image store option of Docker Desktop to be off. Provisioners, Actions and Post-Apply Operations explains why image builds are usually better left to CI, and Docker CI/CD with GitHub Actions shows that pipeline.

data "docker_registry_image" "postgres" {
  name = "postgres:17-alpine"
}

resource "docker_image" "postgres" {
  name          = data.docker_registry_image.postgres.name
  pull_triggers = [data.docker_registry_image.postgres.sha256_digest]
  keep_locally  = true
}

Secrets and state with the Docker provider

The password in the example reaches the container as an environment variable, and env is an ordinary argument, so its value is stored in the state file in clear text even though Terraform hides it in plan output because it derives from a sensitive variable. For a laptop that is acceptable if the state stays local and is ignored by Git. For anything shared, use a remote backend with encryption (Backends and Remote State) or move the secret out of the container definition, for example into a Docker secret or a file mounted at runtime.

Remote hosts and daemon defaults

Pointing host at ssh://user@remote-host:22 manages a remote engine. The documentation warns that the remote daemon may apply defaults to new containers (log options are the example given), which then appear as a diff on the next terraform plan. The recommended fix is lifecycle { ignore_changes = [...] } for the attribute concerned; see Lifecycle, Validation and Checks.

When not to use the Docker provider

The Docker provider is a development and small-host tool. It does not schedule across machines, and it has no notion of rolling updates beyond replacing a container. Compose already describes a multi-container laptop environment well (Compose for Local Development); Terraform earns its place here when the same configuration language, modules and tests should work for the laptop and for the cloud, which is how this section uses it. For a real cluster use the Kubernetes and Helm providers below, or a GitOps agent.

Connecting to a cluster: the Kubernetes provider

The hashicorp/kubernetes provider manages Kubernetes API objects. It requires Terraform 1.0.0 or later, and the current major is 3.x (3.0.0 was released on 2025-12-03).

terraform {
  required_version = ">= 1.9.0, < 2.0.0"

  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 3.4"
    }
  }
}

provider "kubernetes" {
  config_path    = "~/.kube/config"
  config_context = "bookshelf-dev"
}

The provider can get its configuration in two ways, explicitly through arguments of the provider block, or implicitly from the environment. The provider documentation lists four mechanisms.

Mechanism How it is configured

kubeconfig file

config_path (or KUBE_CONFIG_PATH), or several files with config_paths (or KUBE_CONFIG_PATHS); optionally config_context, config_context_auth_info and config_context_cluster. If no context is set the provider uses the default one.

Static credentials

host, plus a bearer token, or username and password, or client_certificate, client_key and cluster_ca_certificate (PEM-encoded). insecure and tls_server_name adjust TLS verification.

Exec plugin

An exec block with api_version, command, optional args and env, which runs a credential plugin before each Terraform operation, so tokens do not expire mid-run.

In-cluster

With no arguments, the provider detects KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT and uses the pod’s service account. This is how Terraform running inside the target cluster (a Job, an in-cluster runner) connects.

Two details trip people up. First, the provider does not read the KUBECONFIG environment variable by default. The variables that map to provider arguments all start with KUBE_: KUBE_CONFIG_PATH, KUBE_HOST, KUBE_TOKEN, KUBE_CTX and so on, so a pipeline that exports only KUBECONFIG quietly connects to nothing. Second, do not mix an exec block with other credential arguments such as token or client_certificate; the documentation says which credential wins is undefined.

The provider also has two arguments for external controllers that edit metadata: ignore_annotations and ignore_labels take lists of regular expressions and make Terraform ignore matching keys on every resource it manages (an autoscaler or a service mesh injecting labels, a load-balancer controller adding annotations). They apply to the root metadata only; for annotations nested in a Pod template use lifecycle { ignore_changes }.

provider "kubernetes" {
  config_path = "~/.kube/config"

  ignore_annotations = [
    "^service\\.beta\\.kubernetes\\.io\\/aws-load-balancer.*",
  ]
}

Configuring providers from a cluster resource

The usual reason to use these providers is that the cluster is itself created by Terraform. The cloud providers expose the endpoint and credentials of a cluster as resource or data source attributes, and those attributes are passed into the provider block. Each cloud has its own shape.

EKS

The aws_eks_cluster data source returns the API endpoint and the base64-encoded certificate authority. Authentication uses the AWS CLI as an exec plugin, so a fresh token is requested for every operation (EKS tokens are short-lived).

variable "cluster_name" {
  type    = string
  default = "bookshelf-dev"
}

data "aws_eks_cluster" "this" {
  name = var.cluster_name
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.this.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)

  exec {
    api_version = "client.authentication.k8s.io/v1"
    command     = "aws"
    args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
  }
}

The AWS provider also has an aws_eks_cluster_auth data source that returns a token directly; the Kubernetes provider’s own documentation says EKS tokens expire in 15 minutes, which is why the exec plugin is the sturdier choice for long applies. The aws command and its credentials (for example an OIDC role in CI, see CI/CD and Automation) must therefore be available wherever Terraform runs.

AKS

An AKS cluster exposes a kube_config block with the host and client certificate material, each base64-encoded.

data "azurerm_kubernetes_cluster" "this" {
  name                = "bookshelf-dev"
  resource_group_name = "bookshelf-dev-rg"
}

provider "kubernetes" {
  host                   = data.azurerm_kubernetes_cluster.this.kube_config[0].host
  client_certificate     = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].client_certificate)
  client_key             = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].client_key)
  cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].cluster_ca_certificate)
}

The AzureRM documentation notes that all arguments of the cluster, including these credentials, are stored in the state in plain text. Clusters that disable local accounts need a different authentication path (Microsoft Entra ID with an exec plugin); see AKS Clusters for the identity options.

GKE

GKE uses an OAuth access token of the identity that runs Terraform, plus the cluster endpoint and CA.

data "google_client_config" "default" {}

data "google_container_cluster" "this" {
  name     = "bookshelf-dev"
  location = "europe-west1"
}

provider "kubernetes" {
  host  = "https://${data.google_container_cluster.this.endpoint}"
  token = data.google_client_config.default.access_token

  cluster_ca_certificate = base64decode(
    data.google_container_cluster.this.master_auth[0].cluster_ca_certificate,
  )
}

The Google guide points out that the token in google_client_config is short-lived and stored in state, and offers the gke-gcloud-auth-plugin exec plugin (with the kubectl auth plugin installed) as the way to fetch a fresh token for each operation. Remember not to combine exec with token: if you choose the plugin, drop the token line.

The Helm provider needs the same data

The Helm provider takes the same information, but since version 3 as a nested object attribute instead of a block, which changes the punctuation:

terraform {
  required_providers {
    helm = {
      source  = "hashicorp/helm"
      version = "~> 3.3"
    }
  }
}

provider "helm" {
  kubernetes = {
    host                   = data.aws_eks_cluster.this.endpoint
    cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)

    exec = {
      api_version = "client.authentication.k8s.io/v1"
      command     = "aws"
      args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
    }
  }
}

The two providers are configured with different syntax: in the Kubernetes provider exec is a block, while the Helm provider was migrated to the Plugin Framework in 3.0.0 and takes exec inside an object attribute. Copying a provider block from one to the other is the most common source of "Unsupported block type" and "Unsupported argument" errors.

The two-phase apply problem

A provider block is configured before the resources that use it are planned. When the provider arguments come from a resource that does not exist yet, such as the endpoint of a cluster created in the same configuration, they are unknown at plan time. The Kubernetes provider documentation therefore carries an explicit warning: credentials passed from other resources to the Kubernetes provider should not be created in the same Terraform module where Kubernetes provider resources are used, because this leads to intermittent and hard-to-diagnose errors, and the most reliable setup is to manage the cluster and the Kubernetes resources with separate apply operations, conveying values between the two through data sources. The Terraform documentation of the provider block states the underlying rule in general terms: provider arguments should be values Terraform knows before applying, and computed resource attributes are not.

The failure shows up in several ways, all rooted in the same cause:

  • a first plan that cannot reach the cluster, because the provider falls back to localhost or to an empty endpoint, and then fails with a connection error;

  • a kubernetes_manifest resource that cannot be planned at all, because it needs the API server to look up the schema of the object kind (see below);

  • a cluster replacement (a change forcing a new cluster) that makes the endpoint unknown again, so the plan for every Kubernetes object in the same configuration is computed against the old, about-to-be-destroyed cluster;

  • a destroy that fails because the cluster has already gone when Terraform tries to delete the objects inside it.

flowchart LR subgraph S1["Phase 1: infrastructure (own state)"] A["VPC, subnets, IAM"] --> B["Managed cluster and node pools"] end subgraph S2["Phase 2: cluster contents (own state)"] C["data source: cluster endpoint and CA"] --> D["kubernetes and helm providers"] D --> E["Namespace, secrets, quotas"] D --> F["helm_release: ingress, cert-manager, GitOps agent"] end B -->|"apply finishes, cluster exists"| C

The practical solution is the one the documentation recommends, expressed as repository layout: two root modules (or two Stacks components, see below) with their own state and their own apply.

bookshelf-infra/
├── cluster/            # phase 1: network, EKS/AKS/GKE, node pools, IAM
│   ├── main.tf
│   └── outputs.tf      # cluster_name, region
└── cluster-contents/   # phase 2: kubernetes + helm providers
    ├── providers.tf    # data source for the cluster, provider blocks
    └── main.tf         # namespaces, secrets, helm_release bootstrap

The second module needs only the name of the cluster (and the region or resource group); everything else comes from a data source, as in the EKS, AKS and GKE snippets above. The name can be passed as a variable, read from the first module’s outputs with the terraform_remote_state data source, or looked up by tag. The options and their trade-offs are in Backends and Remote State. In CI, run the two configurations as two jobs with the second depending on the first, as in CI/CD and Automation.

Other approaches exist, each with a cost:

  • -target. terraform apply -target=aws_eks_cluster.this first, then a full apply, is a way to get a single configuration to converge on the first run. The documentation presents targeting as a tool for exceptional circumstances, and it leaves the rest of the configuration unplanned (The Dependency Graph and the Plan). Do not make it part of a pipeline.

  • OpenTofu. OpenTofu’s provider configuration also has to be known before apply, so it needs the same split. Its dynamic features (provider for_each, early evaluation) address other problems; see Providers.

  • Terraform Stacks. A Stack lays out several components in one configuration and orders them, and the Stacks overview describes deferring a plan for changes that cannot be planned until something else is applied, such as a Kubernetes custom resource that needs its definition. The Stacks documentation notes that deferred changes need the Kubernetes provider 2.32.0 or later. Stacks run on HCP Terraform; see Terraform Stacks.

  • Terraform running in the cluster. With in-cluster configuration, the provider needs no arguments, and the cluster must exist before the run starts, which is the same two-phase split in another form.

Managing workloads with the Kubernetes provider

Once the provider connects, the resources map to Kubernetes kinds. The provider’s resources fall into two generations.

The versioned (_v1) resources

From version 2.7.0, resources and data sources that cover a standard Kubernetes API carry the API version in their name: kubernetes_namespace_v1, kubernetes_deployment_v1, kubernetes_service_v1, kubernetes_ingress_v1 and so on. Version 3.0.0 (2025-12-03) marked every resource and data source without a version suffix as deprecated and listed the replacement for each. The guide states that unversioned resources remain available in 3.0.0 with no removal version scheduled, and that any removal would be announced in advance and happen in a major release.

Deprecated name Use instead

kubernetes_namespace

kubernetes_namespace_v1

kubernetes_service

kubernetes_service_v1

kubernetes_deployment

kubernetes_deployment_v1

kubernetes_config_map, kubernetes_secret

kubernetes_config_map_v1, kubernetes_secret_v1

kubernetes_stateful_set, kubernetes_daemonset

kubernetes_stateful_set_v1, kubernetes_daemon_set_v1 (note the spelling)

kubernetes_job, kubernetes_cron_job

kubernetes_job_v1, kubernetes_cron_job_v1

kubernetes_ingress, kubernetes_ingress_class

kubernetes_ingress_v1, kubernetes_ingress_class_v1

kubernetes_horizontal_pod_autoscaler

kubernetes_horizontal_pod_autoscaler_v1 or kubernetes_horizontal_pod_autoscaler_v2

kubernetes_role, kubernetes_role_binding, kubernetes_cluster_role, kubernetes_cluster_role_binding

the same names with a _v1 suffix

kubernetes_network_policy, kubernetes_pod_disruption_budget

kubernetes_network_policy_v1, kubernetes_pod_disruption_budget_v1

kubernetes_pod_security_policy

removed upstream; use Pod Security Admission (Pod Security)

The point of the suffix is stability. A _v1 resource follows the v1 Kubernetes API, and the provider promises not to change its schema incompatibly; new convenience attributes may still be added with deprecation warnings and state upgraders. Unversioned resources, in contrast, tried to cover several API versions in one schema (the horizontal pod autoscaler is the example the provider gives) and became hard to maintain.

Migrating an existing configuration is a rename plus a state change, because the resource type is part of the address. Do it with a moved block (Refactoring: moved, removed) where the provider supports it; the 3.3.0 and 3.4.0 changelog entries state that moved blocks work from the deprecated kubernetes_namespace to kubernetes_namespace_v1 and (with Terraform 1.8 or later) from kubernetes_role to kubernetes_role_v1. For other resources check the release notes, and otherwise use terraform state mv or remove-and-import (Import and Query) and read the first plan carefully before applying.

The Bookshelf in Kubernetes objects

The following configuration creates a namespace, a Deployment for catalog-api with a readiness probe and a secret reference, and a Service. Kubernetes provider blocks are nested Terraform blocks that mirror the YAML, with metadata, spec and container as repeated blocks rather than lists.

resource "kubernetes_namespace_v1" "bookshelf" {
  metadata {
    name = "bookshelf"
    labels = {
      "pod-security.kubernetes.io/warn" = "restricted"
    }
  }
}

resource "kubernetes_deployment_v1" "catalog_api" {
  metadata {
    name      = "catalog-api"
    namespace = kubernetes_namespace_v1.bookshelf.metadata[0].name
    labels    = { app = "catalog-api" }
  }

  spec {
    replicas = 2

    selector {
      match_labels = { app = "catalog-api" }
    }

    template {
      metadata {
        labels = { app = "catalog-api" }
      }

      spec {
        container {
          name  = "catalog-api"
          image = "ghcr.io/example/bookshelf-catalog-api:1.0.0" # placeholder

          port {
            container_port = 8080
          }

          env {
            name = "DATABASE_PASSWORD"
            value_from {
              secret_key_ref {
                name = "catalog-db"
                key  = "password"
              }
            }
          }

          resources {
            requests = { cpu = "100m", memory = "128Mi" }
            limits   = { memory = "256Mi" }
          }

          readiness_probe {
            http_get {
              path = "/healthz"
              port = 8080
            }
            initial_delay_seconds = 5
          }
        }
      }
    }
  }

  wait_for_rollout = true
}

resource "kubernetes_service_v1" "catalog_api" {
  metadata {
    name      = "catalog-api"
    namespace = kubernetes_namespace_v1.bookshelf.metadata[0].name
  }

  spec {
    selector = { app = "catalog-api" }

    port {
      port        = 80
      target_port = "8080"
    }

    type = "ClusterIP"
  }
}

Points to notice:

  • wait_for_rollout (default true) makes apply wait for the rollout, so a Deployment that never becomes ready fails the run instead of reporting success. The Pod semantics behind it are in ReplicaSets and Deployments and Health Probes and Lifecycle.

  • target_port is a string in this provider (a number or a named port).

  • Two blocks, metadata and spec, are mandatory and each appears exactly once, so references need an index: kubernetes_namespace_v1.bookshelf.metadata[0].name.

  • The probe path /healthz is a Bookshelf convention, not something the provider adds.

  • Every attribute is applied in place through the Kubernetes API, so changes made by other tools (an HPA scaling replicas, kubectl rollout restart adding an annotation) show up as drift. Use ignore_changes for the attribute concerned, for example the kubectl.kubernetes.io/restartedAt annotation example in the provider documentation.

Secrets and the state file

kubernetes_secret_v1 stores its data in the state in plain text; the provider’s own documentation says so. The resource also has write-only arguments, data_wo and binary_data_wo, each with a matching revision argument (data_wo_revision, binary_data_wo_revision) that you increment to push a new value; write-only values are not stored in the state or the plan. Feed them from an ephemeral value, as described in Sensitive Data, Ephemeral Values and Write-Only Arguments. For the Kubernetes side of secrets (encryption at rest, External Secrets, Sealed Secrets) see Secrets.

variable "catalog_db_password" {
  type      = string
  ephemeral = true
  # Supply it as TF_VAR_catalog_db_password from the CI secret store.
}

resource "kubernetes_secret_v1" "catalog_db" {
  metadata {
    name      = "catalog-db"
    namespace = kubernetes_namespace_v1.bookshelf.metadata[0].name
  }

  data_wo          = { password = var.catalog_db_password }
  data_wo_revision = 1
}

Write-only arguments need Terraform 1.11 or OpenTofu 1.11 (ephemeral variables need Terraform 1.10). Because the value is never recorded, Terraform cannot tell whether it changed: incrementing data_wo_revision is the signal to write it again. The page on secrets lists the details and the other sources of ephemeral values (a secret manager read as an ephemeral resource). Many teams instead keep generated credentials in a cloud secret store and sync them into the cluster with an operator, so that the application and the database share one source of truth.

kubernetes_manifest: custom resources and anything else

The typed resources cover the standard APIs. For a custom resource, or a kind the provider has no resource for, the kubernetes_manifest resource takes the object as an HCL value in its manifest attribute:

resource "kubernetes_manifest" "bookshelf_route" {
  manifest = {
    apiVersion = "gateway.networking.k8s.io/v1"
    kind       = "HTTPRoute"

    metadata = {
      name      = "bookshelf"
      namespace = kubernetes_namespace_v1.bookshelf.metadata[0].name
    }

    spec = {
      parentRefs = [{ name = "bookshelf-gateway" }]
      hostnames  = ["bookshelf.example.com"]
      rules = [{
        matches     = [{ path = { type = "PathPrefix", value = "/catalog" } }]
        backendRefs = [{ name = "catalog-api", port = 80 }]
      }]
    }
  }
}

The resource has a few properties that the typed ones do not, all taken from its documentation:

  • It needs the API at plan time. The documentation says the cluster must be reachable while planning, so the cluster cannot be created in the same apply, and it recommends the resource only for custom resources or kinds not yet fully supported by the provider. The CRD itself must also exist before the plan that uses it: install it in an earlier phase (a helm_release or a separate kubernetes_manifest applied first), or the plan fails. This is the same two-phase problem again, one level down.

  • It uses server-side apply. Operations go through Server-side Apply, which needs Kubernetes 1.16 at least and 1.17 or later is strongly recommended. The field_manager block sets the manager name and force_conflicts forces ownership of fields another manager owns.

  • computed_fields lists the manifest fields the API server may change during apply. It defaults to metadata.annotations and metadata.labels; add the fields that controllers fill in (defaulted values) to avoid "inconsistent result after apply" errors.

  • object holds the state as returned by the API, including defaults, and wait blocks (rollout, condition, fields) make an apply wait for readiness; the older wait_for argument is deprecated.

  • YAML conversion. The manifest is HCL, so existing YAML is converted with yamldecode(file("route.yaml")) or with the tfk8s tool the documentation mentions. A multi-document YAML file needs one resource per document, because yamldecode reads a single document.

For installing a whole operator with its CRDs, a helm_release of the operator chart is usually less fragile than many kubernetes_manifest resources; the operator pattern is explained in Custom Resources and Operators. The Gateway API objects used above are described in Gateway API.

The Helm provider

The hashicorp/helm provider installs charts as Terraform resources, using the Helm library rather than the helm binary. The helm_release resource is the main one, and the helm_template data source renders a chart without installing it. The provider’s documentation states that it does not use KUBECONFIG by default, and the same KUBE_* variables as the Kubernetes provider apply.

Version 3 syntax

Version 3.0.0 (2025-06-18) moved the provider to the Plugin Framework. It is a breaking change in syntax, not in behavior: blocks became attributes.

Item Helm provider 2.x Helm provider 3.x

Cluster connection

kubernetes { ... } block

kubernetes = { ... } object attribute

Registries

repeatable registry { ... } block

registries = [ { ... } ] list attribute

Experiments

experiments { manifest = true } block

experiments = { manifest = true } object attribute

Values set from HCL

repeatable set, set_list, set_sensitive blocks

set, set_list and set_sensitive as lists of objects, set = [ { name = "...", value = "..." } ]

Version 3.x also requires Terraform 1.0 or later (the provider now speaks protocol version 6); users of older Terraform can pin the 2.x provider. A mixed .tf file that uses block syntax against the 3.x provider fails at validation, so the upgrade guide is the place to start when terraform init -upgrade brings in a new major.

Installing the add-ons

A typical bootstrap installs an ingress or Gateway controller, then the GitOps agent. The first example uses a chart from an OCI registry, the way Helm distributes charts today (Helm).

resource "helm_release" "bookshelf" {
  name       = "bookshelf"
  namespace  = kubernetes_namespace_v1.bookshelf.metadata[0].name
  repository = "oci://ghcr.io/example/charts" # placeholder registry
  chart      = "bookshelf"
  version    = "1.0.0"

  atomic  = true
  timeout = 300 # seconds

  values = [
    yamlencode({
      catalogApi = { replicas = 2 }
      ingress    = { host = "bookshelf.example.com" }
    })
  ]

  set = [
    {
      name  = "catalogApi.image.tag"
      value = "1.0.0"
      type  = "string"
    }
  ]

  set_sensitive = [
    {
      name  = "redis.password"
      value = var.redis_password
    }
  ]
}

The arguments that matter most, all from the resource schema:

Argument Meaning

name, chart

Required: the release name (at most 53 characters) and the chart; a chart can be a name (with repository), a path or an absolute URL to a .tgz.

repository, version

Where to find the chart and the exact chart version. If version is omitted the latest is installed, which makes plans unstable; pin it.

namespace, create_namespace

Where to install (default default) and whether to create the namespace.

values

A list of raw YAML documents, merged in order. yamlencode of an HCL object is the common way to build one.

set, set_list, set_sensitive

Individual overrides. set_sensitive is marked sensitive in output. A type of string stops Helm from converting values such as true or 1 to other types. Dots, commas, braces and brackets in names or values need double escaping.

set_wo, set_wo_revision

Write-only counterpart of set and its revision number: the value is not stored in the state.

wait, wait_for_jobs, timeout

Wait for all resources to be ready (default true), for Jobs to complete (default false), and the per-operation timeout in seconds (default 300).

atomic, cleanup_on_fail

Roll back (purge a failed first install) if the operation fails, and delete resources created in a failed upgrade. atomic turns on wait.

upgrade_install

Behave like helm upgrade --install: adopt a release that exists but is not in the state. The documentation warns it may not suit production, because the values then come from the resource and not from the original install.

take_ownership

Let Helm adopt existing resources not marked as managed by the release.

max_history, reuse_values, reset_values

Revision history limit and the Helm flags of the same names.

skip_crds, disable_crd_hooks, disable_webhooks

Control CRD installation and hooks.

lint, verify, dependency_update

Run helm lint while planning, verify the chart’s provenance, and run helm dependency update before installing.

postrender

A post-renderer binary run on the manifests (binary_path, args).

status, manifest, metadata

Read-only: the release status, the rendered manifest (with experiments.manifest), and the deployed release metadata.

The helm_template data source takes the same chart and values arguments and returns the rendered manifests, which is useful to feed them into kubernetes_manifest or into a policy check. Release history is kept in the cluster by Helm’s storage driver, which defaults to Secrets and can be set with the provider argument helm_driver.

Behavior to expect from helm_release

  • What Terraform tracks. The state holds the release name, namespace, chart, version and the values you supplied. Terraform does not compare the live objects with the chart; if someone edits a Deployment by hand, the next plan shows no change because the release did not change. Drift detection of in-cluster objects is what GitOps agents do, which is one of the reasons to hand off (see below).

  • Values end up in the state. values and set are stored in the state in clear text, and so are the sensitive values unless set_wo is used. Treat the state as a secret, as described in State.

  • Failed installs. With atomic = true a failed installation is purged (and wait is switched on), which keeps the next apply from starting on top of a half-installed release. Pair it with a timeout that fits the chart.

  • Chart upgrades are version changes. Changing version produces an upgrade; CRDs shipped in the chart’s crds/ directory are installed on first install but not upgraded by Helm, so upgrade CRDs separately.

  • Helm 4. The provider embeds the Helm library. As of provider 3.3.0 the changelog states that the embedded Helm moved from 3.18.5 to 3.20.2 (the Helm 3 line). The Kubernetes section covers what changed in Helm 4 for the CLI.

When to hand off to GitOps or Helm

Terraform manages infrastructure with a lifecycle and a state file. Kubernetes already has controllers that reconcile objects continuously, and a Git repository can describe the desired state of every application. Running the same objects through Terraform too is possible, but it mixes two reconcilers and two sources of truth.

Situation Use Why

Create or resize the cluster, node pools, networks, IAM, registries and databases

Terraform

Cloud APIs with a lifecycle, cost and dependencies that Terraform models well.

One-off bootstrap: the GitOps agent, an ingress or Gateway controller, cert-manager, CRDs

Terraform with helm_release

They must exist before anything else, and are rarely changed; the module that installs them can be versioned and tested.

Namespaces, quotas, service accounts bound to a cloud identity (IRSA, Workload Identity), cluster-level RBAC

Terraform (Kubernetes provider)

They are tied to the cloud identity and the account, not to a release of the application.

Application releases of catalog-api, reviews-api and web, many times a day, by many teams

Helm or Kustomize delivered by Argo CD or Flux

Continuous reconciliation and drift correction, pull requests as the approval step, no cloud credentials in the pipeline. See GitOps and CI/CD.

A package to install on several clusters with different values

Helm (chart) plus an Argo CD ApplicationSet or a Flux HelmRelease

Values per environment live in Git next to the chart, not in the Terraform state.

Environment-specific patches of plain manifests

Kustomize

See Kustomize.

Values that come from Terraform (a database endpoint, a bucket name, a role ARN)

Terraform writes them to a ConfigMap or a helm_release value, or an operator syncs them

A small, explicit hand-off point between infrastructure and application. The GitOps repository then reads the ConfigMap or the Helm values file.

A common pattern is to let Terraform install Argo CD (or Flux) with helm_release and create the root Application or Kustomization object once, after which the GitOps agent owns every workload. After the first apply the Terraform configuration only changes when the cluster itself changes. This keeps the state file small, avoids drift fights between two reconcilers, and keeps application credentials out of the Terraform run. If a team prefers Terraform to deploy applications anyway, give helm_release a pinned version, keep the values in one place and run it from the same pipeline that built the image, accepting that rollbacks are then Terraform operations.

Managed clusters add details of their own: the choice between AKS, EKS and GKE, node management and upgrade channels are described in Managed Kubernetes and Distributions, and the Kubernetes administration pages cover what is left for you to operate (Cluster Setup and Administration).

Creating EKS, AKS and GKE clusters

The cloud providers have a cluster resource each. The snippets below are the minimum needed to get a working managed cluster; the complete baselines (network design, node pools, add-ons, private endpoints) are on the cloud pages Terraform on AWS, Terraform on Azure and Terraform on Google Cloud. Each snippet is also the phase 1 of the two-phase pattern above; the outputs are what phase 2 reads. Cluster and node pool concepts are explained on EKS Clusters, AKS Clusters and GKE Clusters. Every snippet creates billable resources: destroy it when you are done (terraform destroy), and read the plan first.

EKS with Auto Mode

EKS Auto Mode has AWS manage the compute, block storage and load balancing for the cluster. The provider documentation requires the three Auto Mode capabilities (compute_config, storage_config.block_storage and kubernetes_network_config.elastic_load_balancing) to be enabled or disabled together, and bootstrap_self_managed_addons to be false.

terraform {
  required_version = ">= 1.9.0, < 2.0.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

variable "subnet_ids" {
  type        = list(string)
  description = "Private subnets in at least two availability zones."
}

data "aws_iam_policy_document" "cluster_trust" {
  statement {
    actions = ["sts:AssumeRole", "sts:TagSession"]
    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }
  }
}

data "aws_iam_policy_document" "node_trust" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "cluster" {
  name               = "bookshelf-eks-cluster"
  assume_role_policy = data.aws_iam_policy_document.cluster_trust.json
}

resource "aws_iam_role_policy_attachment" "cluster" {
  for_each = toset([
    "AmazonEKSClusterPolicy",
    "AmazonEKSComputePolicy",
    "AmazonEKSBlockStoragePolicy",
    "AmazonEKSLoadBalancingPolicy",
    "AmazonEKSNetworkingPolicy",
  ])

  role       = aws_iam_role.cluster.name
  policy_arn = "arn:aws:iam::aws:policy/${each.key}"
}

resource "aws_iam_role" "node" {
  name               = "bookshelf-eks-auto-node"
  assume_role_policy = data.aws_iam_policy_document.node_trust.json
}

resource "aws_iam_role_policy_attachment" "node" {
  for_each = toset([
    "AmazonEKSWorkerNodeMinimalPolicy",
    "AmazonEC2ContainerRegistryPullOnly",
  ])

  role       = aws_iam_role.node.name
  policy_arn = "arn:aws:iam::aws:policy/${each.key}"
}

resource "aws_eks_cluster" "this" {
  name     = "bookshelf-dev"
  role_arn = aws_iam_role.cluster.arn

  access_config {
    authentication_mode = "API"
  }

  bootstrap_self_managed_addons = false

  compute_config {
    enabled       = true
    node_pools    = ["general-purpose"]
    node_role_arn = aws_iam_role.node.arn
  }

  kubernetes_network_config {
    elastic_load_balancing {
      enabled = true
    }
  }

  storage_config {
    block_storage {
      enabled = true
    }
  }

  vpc_config {
    subnet_ids              = var.subnet_ids
    endpoint_private_access = true
    endpoint_public_access  = true # restrict or disable for production
  }

  depends_on = [
    aws_iam_role_policy_attachment.cluster,
    aws_iam_role_policy_attachment.node,
  ]
}

output "cluster_name" {
  value = aws_eks_cluster.this.name
}

The depends_on is there on purpose: the provider documentation says the IAM role permissions must be created before, and deleted after, the cluster, otherwise EKS cannot delete the managed EC2 infrastructure it created (such as security groups) when the cluster is destroyed. An authentication_mode of API selects EKS access entries over the legacy aws-auth ConfigMap. The version argument is optional; omitted, AWS picks its default, so pin it when upgrades must be controlled. Outside Auto Mode the same resource takes managed node groups (aws_eks_node_group) or Fargate profiles.

AKS

The azurerm_kubernetes_cluster resource needs a resource group, a default_node_pool and an identity.

terraform {
  required_version = ">= 1.9.0, < 2.0.0"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 5.0"
    }
  }
}

provider "azurerm" {
  features {}
  # The subscription comes from ARM_SUBSCRIPTION_ID or the az CLI.
}

resource "azurerm_resource_group" "bookshelf" {
  name     = "bookshelf-dev-rg"
  location = "westeurope"
}

resource "azurerm_kubernetes_cluster" "this" {
  name                = "bookshelf-dev"
  location            = azurerm_resource_group.bookshelf.location
  resource_group_name = azurerm_resource_group.bookshelf.name
  dns_prefix          = "bookshelf-dev"
  sku_tier            = "Standard"

  oidc_issuer_enabled       = true
  workload_identity_enabled = true

  default_node_pool {
    name       = "system"
    node_count = 2
    vm_size    = "Standard_D2s_v5"
  }

  node_provisioning_profile {
    mode = "Manual"
  }

  identity {
    type = "SystemAssigned"
  }
}

output "cluster_name" {
  value = azurerm_kubernetes_cluster.this.name
}

output "resource_group_name" {
  value = azurerm_resource_group.bookshelf.name
}

Notes from the resource documentation: you must define either dns_prefix or dns_prefix_private_cluster; sku_tier is Free by default and Standard includes the uptime SLA; workload_identity_enabled needs oidc_issuer_enabled; and the provider recommends running the latest AzureRM release for AKS, because the service moves quickly. The block node_provisioning_profile is listed as required in the current argument reference, with mode defaulting to Manual; its example omits it, so it is written out above. Autoscaling node pools usually combine auto_scaling_enabled with lifecycle { ignore_changes = [default_node_pool[0].node_count] }, a pattern the documentation suggests for node_count. The features {} block is required by the provider and may be empty. See also AKS Clusters for the Azure-side explanation.

GKE

A GKE Autopilot cluster has Google manage the nodes. The provider documentation states that, from provider 5.0.0, you must explicitly set deletion_protection = false and apply it to write the field to the state before Terraform can destroy the cluster, so set it explicitly for a throwaway environment.

terraform {
  required_version = ">= 1.9.0, < 2.0.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 8.0"
    }
  }
}

variable "project_id" {
  type = string
}

provider "google" {
  project = var.project_id
  region  = "europe-west1"
}

resource "google_container_cluster" "this" {
  name     = "bookshelf-dev"
  location = "europe-west1"

  enable_autopilot    = true
  deletion_protection = false # only for a throwaway environment
}

output "cluster_name" {
  value = google_container_cluster.this.name
}

output "cluster_location" {
  value = google_container_cluster.this.location
}

For a Standard cluster, the documentation recommends a separately managed node pool: create the cluster with remove_default_node_pool = true and initial_node_count = 1, and add google_container_node_pool resources with a least-privilege service account. Regional locations (a region, not a zone) give a highly available control plane and are preferred. The Google guide also warns about master_authorized_networks_config: restrict it to the networks that administer the cluster rather than allowing every address.

From cluster to workloads

With the cluster created in one configuration, the second configuration reads it with the data sources of the previous section, and the pipeline runs cluster/ first and cluster-contents/ second. Credentials for kubectl on a laptop are fetched separately (aws eks update-kubeconfig, az aks get-credentials, gcloud container clusters get-credentials); the cloud pages in the Kubernetes section show them. Application delivery then continues as described under GitOps and CI/CD, and the full Bookshelf on Kubernetes is in the Kubernetes end-to-end page.

Terraform and OpenTofu differences

The three providers are ordinary providers, so they work unchanged with OpenTofu. They are distributed through the OpenTofu registry as well: at the time of writing it lists kreuzwerker/docker 4.6.0, hashicorp/kubernetes 3.4.0 and hashicorp/helm 3.3.0, the same versions as the Terraform Registry. The differences are in the surrounding language.

Topic Terraform 1.17 OpenTofu 1.13

Provider arguments from a cluster resource

Must be known before apply for the provider to configure; split into two applies, or use Stacks and deferred changes on HCP Terraform

Same rule; provider arguments can use only values known before apply. Split into two applies

Several clusters in one configuration

One aliased provider block per cluster (static), or separate roots

Aliased provider blocks accept for_each (OpenTofu 1.9 and later), one instance per element

Write-only and ephemeral values for secrets

data_wo, set_wo and ephemeral values (ephemeral values from Terraform 1.10, write-only arguments from 1.11)

Ephemerality and write-only arguments from OpenTofu 1.11

Command name

terraform

tofu

What changed recently

  • Docker provider 4.0.0 (2026-04-03): part of the provider was ported to the Plugin Framework, and the buildx builder became the default. Later 4.x releases added actions (image save and load, container export, docker_exec, system prune) and the docker_containers data source.

  • Kubernetes provider 3.0.0 (2025-12-03): every unversioned resource and data source is deprecated in favor of its _v1 (or _v2) form; kubernetes_pod_security_policy has no replacement; the provider needs Terraform 1.0 or later; kubernetes_manifest now resolves credentials like the other resources (the KUBE_* environment variables no longer override values present in the configuration).

  • Kubernetes provider 3.3.0 and 3.4.0 (2026-10-01 and 2026-10-16): kubernetes_namespace_v1 and kubernetes_role_v1 moved to the Plugin Framework with stricter validation, and null map values in their metadata are now rejected. moved blocks from the old names are supported.

  • Helm provider 3.0.0 (2025-06-18): Plugin Framework, attribute syntax for kubernetes, registries, experiments, set, set_list and set_sensitive; set_wo added. Later 3.x releases added qps, take_ownership, timeouts and the resources attribute of the experimental manifest storage.

  • Terraform 1.10 and 1.11 (OpenTofu 1.11): ephemeral values and write-only arguments make it possible to push secrets to the cluster without storing them in state.

Where the books are outdated or wrong

Books on Terraform and Kubernetes were mostly written against the Kubernetes provider 1.x and 2.x. Their listings use the unversioned resources (kubernetes_service, kubernetes_ingress, kubernetes_deployment), which are deprecated since 3.0.0; the kubernetes.io/ingress.class annotation they rely on is superseded by ingressClassName, so use kubernetes_ingress_v1 (with its ingress_class_name argument, the Kubernetes ingressClassName field) or the Gateway API. One book’s Role examples bind a namespaced kubernetes_role through a ClusterRoleBinding, which does not match scope, and another opens GKE’s master authorized networks to 0.0.0.0/0; use a matching Role and RoleBinding, and restrict the authorized networks. Examples that put a GKE access token or an AKS client key straight into the provider block without an exec plugin or a data source, or that create the cluster and the objects in it in a single configuration, ignore the two-phase problem described above. Listings of helm_release that use set { ... } blocks are Helm provider 2.x syntax and fail on 3.x.

Pitfalls

  • Cluster and contents in one root module. It looks convenient and fails on the first plan, on cluster replacement and on destroy. Split them, or use Stacks.

  • KUBECONFIG instead of KUBE_CONFIG_PATH. Neither provider reads KUBECONFIG, so a CI job may configure nothing and then fall back to a local or in-cluster default.

  • Mixing exec with token or client certificates. The behavior is undefined.

  • Block syntax against the Helm provider 3.x. Use kubernetes = { ... }, set = [ ... ], registries = [ ... ].

  • Unpinned chart versions. helm_release without version installs the latest chart at each create and pulls surprises into a plan.

  • A CRD and its custom resources in the same apply. kubernetes_manifest needs the CRD at plan time; install the CRD first.

  • Secrets in values, set and data. They live in the state in clear text; use set_sensitive plus a protected state, or the write-only arguments, or an operator that syncs from a secret store.

  • Two reconcilers for one object. If Argo CD or Flux owns a Deployment, Terraform must not also manage it, or each apply and each sync undoes the other.

  • Destroying the cluster first. A destroy of the cluster contents after the cluster is gone fails; destroy the contents configuration before the infrastructure one.

  • Docker container replaced on every plan. This usually means the image argument uses a name that resolves to a new ID each time; reference image_id and use pull_triggers deliberately.

  • Cloud tokens with a lifetime shorter than the apply. Use exec plugins so tokens are refreshed for each operation.

  • Forgetting the cost. Managed clusters bill while they exist. A destroy after a demo is part of the exercise.