Autoscaling

An HPA is its own object, kind: HorizontalPodAutoscaler in the autoscaling/v2 API group (GA since Kubernetes 1.23; the once-parallel autoscaling/v1, resource-metrics-only shape is still served but v2 is the only version worth writing new manifests against). It targets one scalable resource (spec.scaleTargetRef, almost always the Bookshelf catalog-api Deployment) through its scale subresource, and a dedicated horizontal pod autoscaler controller inside kube-controller-manager polls the metrics APIs described below on a fixed sync period (15 seconds by default), computes a desired replica count from spec.metrics, and writes it to the target’s spec.replicas — the HPA never touches Pods directly, only the Deployment’s replica count, which is why it composes cleanly with the rolling-update mechanics already covered on ReplicaSets and Deployments.

spec.metrics accepts four metric types, and an HPA may list several of them at once, in which case the controller computes a desired replica count from each and scales to the largest:

Source: Kubernetes Docs — Horizontal Pod Autoscaling.

spec.metrics[].type: Resource entries are served by the Metrics API (metrics.k8s.io), a narrow, current-usage-only API distinct from the full custom.metrics.k8s.io / external.metrics.k8s.io APIs that back Pods/Object/External metrics. metrics-server is the in-cluster component that implements it: it scrapes every kubelet’s Summary API on an interval, keeps only the latest sample in memory (no history, no long-term storage — kubectl top and the HPA controller are its only real clients), and registers itself as an APIService aggregating metrics.k8s.io into the main API server. Its Metrics API became stable in Kubernetes 1.37; earlier releases already shipped metrics-server broadly, but the API surface itself only reached that maturity level at 1.37.

Source: Kubernetes Docs — Resource metrics pipeline and metrics-server — Configuration.

If kubectl top or kubectl describe hpa shows unknown targets, metrics-server is either not installed, not reachable (a common kind/minikube pitfall, addressed above), or has not completed its first scrape yet — the HPA controller simply skips a metric it cannot read on that sync cycle rather than failing outright, so a transient gap self-heals once metrics-server catches up.

Where an HPA changes replica count, a VPA changes a Pod’s own resource requests (and, depending on mode, limits) over time, based on historical usage it observes — the fit for a workload whose per-replica CPU/memory need drifts as its data or traffic shape changes, rather than one whose number of replicas should track load. A VPA is not part of core Kubernetes; it ships as a separate project (kubernetes/autoscaler, subdirectory vertical-pod-autoscaler) with three components: a recommender (watches usage via the Metrics API and computes a recommendation), an updater (resizes or evicts Pods whose current requests drift too far from the recommendation, in the Recreate, InPlaceOrRecreate and InPlace modes), and an admission controller (rewrites a Pod’s requests at creation time to the current recommendation).

spec.updatePolicy.updateMode selects how aggressively a VPA acts:

Source: Kubernetes Docs — Autoscaling Workloads and VPA Docs — Quick start.

A VPA and an HPA must never scale the same resource dimension of the same workload at once — a VPA raising a Pod’s CPU requests while an HPA is scaling replica count on CPU utilisation changes the denominator the HPA’s percentage is computed against, and the two fight each other. `notifier’s replica count is already driven by KEDA (below) on queue depth, not CPU, so pointing a VPA at its CPU/memory requests is safe; a VPA must never be pointed at `catalog-api’s CPU while its HPA above is also scaling on CPU utilisation.

Resizing a running container’s CPU/memory requests/limits without restarting its Pod reached Stable (GA) in Kubernetes 1.35. Before this feature, any change to spec.containers[].resources was immutable after Pod creation and forced a full Pod replacement (delete and recreate), which is what VPA’s InPlaceOrRecreate mode above falls back to when a resize cannot be applied in place.

Each container’s resizePolicy states, per resource, whether a change can be absorbed by the running process (NotRequired, the default) or needs the container itself restarted (RestartContainer — typical for memory limits, since most runtimes cannot shrink a process’s live memory ceiling):

Source: Kubernetes Docs — Resize CPU and Memory Resources assigned to Containers.

A resize is requested against the Pod’s dedicated resize subresource, not by editing the Pod’s own spec.containers[].resources field directly:

Source: Kubernetes Docs — Resize CPU and Memory Resources assigned to Containers.

The kubelet reports progress through two Pod conditions (the old status.resize field is gone): PodResizeInProgress (the resize was accepted and is being applied) and PodResizePending, whose reason is Deferred (the node currently lacks room; retried later) or Infeasible (the request exceeds the node’s capacity outright, or cannot be applied on it). Per container, status.containerStatuses[].resources/allocatedResources show what is actually applied. This subresource is exactly what VPA’s InPlaceOrRecreate mode drives internally instead of evicting the Pod, and it is also usable by hand for a one-off adjustment with no VPA installed at all.

Every autoscaler above only changes how many Pods, or how large each Pod’s requests are; none of them can add capacity if every node is already full. Node-level autoscaling watches for Pods that are Pending because nothing schedulable exists for them (or, in the other direction, nodes that are underused for long enough), and adds or removes nodes accordingly — exactly the "external LB changes replica counts" idea Book 1 sketches, reframed one layer down: the book’s scaling signal came from a load balancer noticing demand and asking for more replicas; here the signal is the scheduler itself noticing it has no room for the replicas an HPA already asked for, and asking the cloud provider for more nodes.

Both projects are driven purely by the combination of Pod resource requests and scheduling constraints (Scheduling and Disruptions covers affinity, taints and topology spread in depth) — neither one reads an HPA or VPA object directly; they react to the Pending Pods (or idle nodes) those controllers' own scaling decisions produce. On a local kind or minikube cluster there is no cloud node group to scale at all, so Bookshelf's node autoscaling is prose-only here and exercised for real only on a managed cluster (see AKS clusters for the equivalent AKS-managed node pool autoscaler).

References: Kubernetes Docs — Node Autoscaling, Cluster Autoscaler (GitHub), Karpenter Documentation.

notifier has no CPU/memory signal worth scaling on, and no HTTP traffic at all — its only load signal is how many messages are waiting on its queue, which is exactly the External metric type an HPA already supports in principle (shown above), but only after someone stands up a custom/external metrics adapter translating that queue’s API into the Kubernetes metrics APIs. KEDA (Kubernetes Event-driven Autoscaling) packages that adapter together with a ScaledObject CRD that is far more approachable than authoring a raw External HPA by hand, and its scalers cover dozens of event sources (message queues, cloud metrics, Prometheus, cron schedules) out of the box. Internally, a ScaledObject creates and owns a plain autoscaling/v2 HPA on the operator’s behalf — which is why a ScaledObject and a hand-written HPA must never target the same workload at once, the same rule already stated for VPA and HPA above.

Source: KEDA Docs — Redis Lists Scaler.

Source: KEDA Docs — Deploy KEDA.

minReplicaCount: 0 gives notifier genuine scale-to-zero on an ordinary queue — the same beta HPA HPAScaleToZero capability from earlier on this page, but reached through KEDA’s own ScaledObject rather than by hand-authoring an External-metric HPA and its adapter. On a managed cluster where the message broker is a cloud service instead of the in-cluster Redis shown above, KEDA is commonly installed as the platform’s own managed add-on instead of a self-managed Helm release — AKS microservices walks through exactly that (the --enable-keda AKS add-on, a ScaledObject against an Azure Service Bus queue, and the workload-identity wiring the managed add-on needs) and is not repeated here.