Deployment Strategies

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production; see the section bibliography for the reference material consulted while preparing these pages.

ReplicaSets and Deployments covers the Deployment object’s own built-in RollingUpdate and Recreate strategies, which replace one Deployment’s Pods with another version of the same Deployment. This page covers the strategies that sit one level above that: shifting client traffic between two independent Deployments (or two independent Services), so a new version can be validated against a slice of real traffic, or switched over instantly, before the old version is removed. All patterns below reuse the Bookshelf catalog-api Deployment and its app.kubernetes.io/* labels (Objects and the Kubernetes API) and are current for Kubernetes 1.37 and Gateway API 1.6.

Blue/green: switching a Service selector

A blue/green deployment runs the old ("blue") and new ("green") versions as two complete, independent Deployments at the same time, each with its own full replica count, and moves all traffic from one to the other in a single step by editing a Service’s spec.selector — exactly the "loose coupling" idea from Objects and the Kubernetes API: a Service does not know which Pods it targets until its selector is evaluated, so retargeting it is just an object update, not a Pod-level change.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog-api-blue
  labels:
    app.kubernetes.io/name: catalog-api
    app.kubernetes.io/version: "1.0.0"
    track: blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: catalog-api
      track: blue
  template:
    metadata:
      labels:
        app.kubernetes.io/name: catalog-api
        track: blue
    spec:
      containers:
        - name: catalog-api
          image: ghcr.io/example/bookshelf-catalog-api:1.0.0
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog-api-green
  labels:
    app.kubernetes.io/name: catalog-api
    app.kubernetes.io/version: "2.0.0"
    track: green
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: catalog-api
      track: green
  template:
    metadata:
      labels:
        app.kubernetes.io/name: catalog-api
        track: green
    spec:
      containers:
        - name: catalog-api
          image: ghcr.io/example/bookshelf-catalog-api:2.0.0
---
apiVersion: v1
kind: Service
metadata:
  name: catalog-api
spec:
  selector:
    app.kubernetes.io/name: catalog-api
    track: blue          # flip to "green" to cut over
  ports:
    - port: 80
      targetPort: 8080

Source: Kubernetes Docs — Service.

# validate green out-of-band first (a second, temporary Service, or a debug Pod)
kubectl patch service catalog-api -p '{"spec":{"selector":{"app.kubernetes.io/name":"catalog-api","track":"green"}}}'
kubectl rollout status deployment/catalog-api-green   # green already up before the cut-over

The cut-over itself is atomic from the client’s point of view — kube-proxy reprograms its rules from the EndpointSlice that follows the new selector (Services) — but any in-flight request against a blue Pod that is deleted immediately afterwards is not: keep catalog-api-blue running, scaled down but not deleted, until in-flight requests have drained, then remove it. Rolling back is the same patch in reverse, which is the main appeal of blue/green over a rolling update: instant, complete reversal with no partially-updated state in between. The cost is running double the replica count for the overlap window, and this pattern gives no partial traffic exposure — a Pod either serves 100% of requests (as part of the active colour) or 0% (idle), which is what the canary strategies below are for.

flowchart LR subgraph Blue["catalog-api-blue (track=blue)"] B1[Pod v1.0.0] B2[Pod v1.0.0] B3[Pod v1.0.0] end subgraph Green["catalog-api-green (track=green)"] G1[Pod v2.0.0] G2[Pod v2.0.0] G3[Pod v2.0.0] end SVC["Service catalog-api\nselector: track=blue"] -->|100% traffic| Blue SVC -.->|selector flipped to track=green| Green

Canary: two Deployments behind one Service

A canary release exposes a small, real slice of production traffic to the new version without a hard cut-over, following the same shape as the official canary deployment tutorial: two Deployments share a common label that the Service selects on, but differ in a second label (conventionally track) that the Service selector omits, so both Deployments' Pods end up behind the same Service and share its traffic in proportion to their replica counts.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog-api-stable
  labels:
    app.kubernetes.io/name: catalog-api
    track: stable
spec:
  replicas: 9              # 90% of Pods
  selector:
    matchLabels:
      app.kubernetes.io/name: catalog-api
      track: stable
  template:
    metadata:
      labels:
        app.kubernetes.io/name: catalog-api
        track: stable
    spec:
      containers:
        - name: catalog-api
          image: ghcr.io/example/bookshelf-catalog-api:1.0.0
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog-api-canary
  labels:
    app.kubernetes.io/name: catalog-api
    track: canary
spec:
  replicas: 1               # 10% of Pods
  selector:
    matchLabels:
      app.kubernetes.io/name: catalog-api
      track: canary
  template:
    metadata:
      labels:
        app.kubernetes.io/name: catalog-api
        track: canary
    spec:
      containers:
        - name: catalog-api
          image: ghcr.io/example/bookshelf-catalog-api:2.0.0
---
apiVersion: v1
kind: Service
metadata:
  name: catalog-api
spec:
  selector:
    app.kubernetes.io/name: catalog-api   # matches both tracks; "track" itself is left out
  ports:
    - port: 80
      targetPort: 8080

Source: Kubernetes Docs — Canary Deployments.

The traffic split this produces is coarse and Pod-count-driven — 9 stable replicas and 1 canary replica gives approximately 90/10, because a Service has no concept of weight, only a selector and whatever Pods currently match it (Services). kubectl scale deployment/catalog-api-canary --replicas=3 and scaling catalog-api-stable down by the same amount is the whole mechanism for widening the canary; deleting catalog-api-canary (or scaling it to zero) is the whole rollback. This is precise enough for a small number of coarse steps, but it cannot express "1%" without an impractical total replica count, which is exactly the gap the weighted canary below closes.

flowchart LR subgraph Stable["catalog-api-stable (track=stable, 9 Pods)"] S1[Pod v1.0.0] S2[Pod v1.0.0] Sn["... 9 Pods"] end subgraph Canary["catalog-api-canary (track=canary, 1 Pod)"] C1[Pod v2.0.0] end SVC["Service catalog-api\nselector: app.kubernetes.io/name=catalog-api"] -->|"~90%"| Stable SVC -->|"~10%"| Canary

Weighted canary: Gateway API HTTPRoute backendRefs[].weight

The Gateway API page covers HTTPRoute end to end; this section only covers the one field that turns it into an exact-percentage canary, without repeating that page’s content. HTTPRoute.spec.rules[].backendRefs can list several backend Services, each carrying an integer weight; the implementation (here, Envoy Gateway) splits traffic across them in proportion to weight / sum-of-weights, entirely independently of how many Pods sit behind each Service:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: catalog-api
spec:
  parentRefs:
    - name: bookshelf-gateway
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /catalog
      backendRefs:
        - name: catalog-api-stable
          port: 80
          weight: 90
        - name: catalog-api-canary
          port: 80
          weight: 10

Source: Gateway API Docs — HTTP traffic splitting.

The two backendRefs are Services of their own, one per track, selecting the track label of the stable and canary Deployments shown earlier:

apiVersion: v1
kind: Service
metadata:
  name: catalog-api-stable
spec:
  selector:
    app.kubernetes.io/name: catalog-api
    track: stable
  ports:
    - port: 80
      targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: catalog-api-canary
spec:
  selector:
    app.kubernetes.io/name: catalog-api
    track: canary
  ports:
    - port: 80
      targetPort: 8080

Source: Kubernetes Docs — Service.

Unlike the plain-Service canary above, each backend here is its own Service (each selecting only its own track’s Pods), so the replica count on either side is free to be whatever capacity requires — one canary Pod can validate a change at exactly 1% of traffic by setting weight: 1 against weight: 99, with no replica-count arithmetic at all. Moving the rollout forward is a one-line kubectl patch (or a GitOps commit) of the two weight fields, raising the canary’s share step by step (for example 10, 25, 50). Once the canary is trusted, promote it: set the new image on catalog-api-stable, wait for that rollout to finish, move the weights back to 100/0, then remove the canary backendRef and delete catalog-api-canary. Rolling back at any step is a patch that sets the canary’s weight to 0. HTTPRoute also supports routing a specific cohort into the canary deterministically with a header or cookie match rule ahead of the weighted rule — useful for internal dogfooding before any weighted exposure — which the Gateway API page’s filters section covers.

flowchart LR Client([Client request]) --> GW["Gateway: bookshelf-gateway"] GW --> Route["HTTPRoute: catalog-api\npath /catalog"] Route -->|"weight: 90"| SvcStable["Service catalog-api-stable"] --> Stable["Pods (track=stable)"] Route -->|"weight: 10"| SvcCanary["Service catalog-api-canary"] --> Canary["Pods (track=canary)"]

A/B testing by labels

A/B testing looks like a canary at the manifest level — two Deployments, one Service or HTTPRoute splitting traffic between them — but the goal is different: a canary asks "is the new version safe", and traffic is shifted by risk tolerance until it reaches 100%; an A/B test asks "which version performs better against a business metric" for two versions that both stay live, split by a cohort rule rather than a global percentage. Book 1 ch. 3 describes this as choosing a request’s target by a label on the client’s identity (a user segment, a request header, a cookie) rather than by a fixed round-robin ratio — the mechanism is the same label-selector plumbing already used above, just driven by a routing rule keyed on the request instead of on replica counts:

  • At the plain-Service level, this means an intermediate layer (an authenticated ingress controller annotation, or an application-level router) that inspects the request and rewrites it to a track=variant-a or track=variant-b selector-matching Service — Kubernetes' own Service load-balancing has no concept of "this cohort of users", only "this label".

  • At the HTTPRoute level, this is the header/cookie matches rule mentioned above used instead of (or ahead of) a weight split: a rule matching X-Bookshelf-Cohort: variant-b routes deterministically to the catalog-api-variant-b Service, so the same user is consistently served the same variant across requests, which a purely weighted split cannot guarantee.

Because A/B testing is a product-analytics decision rather than a release-safety one, it usually runs for much longer than a canary window and is measured against conversion or engagement metrics rather than error rates or latency.

Argo Rollouts and Flagger

Everything above is expressed with plain Deployments, Services and `HTTPRoute`s, driven by hand or by a script. In practice, most teams doing this continuously delegate the weight/replica arithmetic, the pause-and-analyze steps and the automatic rollback to a dedicated progressive delivery controller:

  • Argo Rollouts replaces the Deployment object with its own Rollout custom resource, which drives blue/green and canary strategies declaratively — a canary Rollout lists an ordered sequence of setWeight / pause steps, and can gate progression on live Prometheus (or other provider) metrics via an AnalysisTemplate, automatically rolling back if a metric fails its threshold. It manages the traffic split itself against a plain Service, or, when installed, against Gateway API HTTPRoute weights directly, making the manual kubectl patch steps above unnecessary.

  • Flagger takes the opposite shape: it stays a controller watching an existing Deployment (no new workload CRD to adopt) and drives the canary/A-B/blue-green rollout by editing the weights of a service mesh or ingress/gateway resource on the object’s behalf, again gated on metrics it queries itself (Prometheus, Datadog, CloudWatch), promoting or rolling back automatically.

Both tools turn the strategies above into a closed, automated loop (shift traffic → observe metrics → advance or roll back) instead of a manually-run script, at the cost of an extra controller and, for Argo Rollouts, replacing the Deployment kind itself for any workload it manages.

Feature flags vs deployment strategies

A deployment strategy controls which running version of a container image a request reaches; a feature flag is an application-level if branch, evaluated inside a single running version, that turns a code path on or off per request without deploying anything. The two solve different problems and are commonly combined rather than substituted for one another: a canary or weighted rollout de-risks the deployment of new code (does the new binary crash, leak memory, or regress latency), while a feature flag de-risks turning on a finished feature independently of when it was deployed (a flag can ship dark in every replica of a canary and a stable Deployment alike, then be enabled instantly for one cohort with no rollout at all, and rolled back with no Pod restart). A team that only has feature flags still needs a deployment strategy for the binary itself; a team that only has canary deployments still cannot cheaply disable one feature for one cohort without redeploying. Kubernetes has no built-in feature-flag mechanism — a flag’s state is ordinary application configuration, most often read from the ConfigMap or a dedicated flag service, not from anything covered on this page.

References