Autoscaling & Resource Management
Horizontal Pod Autoscaler (HPA) — Scaling on Metrics
Scaling pod replica counts automatically based on CPU/memory/custom metrics.
Introduction
Static workload scaling is inefficient for applications with fluctuating traffic. The Horizontal Pod Autoscaler (HPA) automatically adjusts the number of replica pods in a Deployment, StatefulSet, or ReplicaSet based on observed CPU utilization, memory consumption, or custom metrics.
HPA implements horizontal scaling, meaning it adds or removes Pod replicas (scaling out/in), rather than changing the size of existing pods (scaling up/down).
Why It Matters
Without dynamic scaling, you are forced to over-provision resources for peak traffic (wasting money during off-peak hours) or risk downtime due to resource starvation when a sudden traffic spike crashes your fixed-replica setup.
Real-World Analogy
Think of a supermarket checkout area. When there are only a few shoppers, one or two cashier lanes are open. As customer queues grow (high workload), the manager opens more lanes (horizontal scaling) to distribute the traffic. When the crowd leaves, empty lanes are closed to reduce operating overhead.
How It Works
The HPA is implemented as a control loop inside the Kubernetes Controller Manager. By default, every 15 seconds, the controller queries the resource metrics API (like the Metrics Server) to check container metrics for the pods listed in the scale target.
The desired number of replicas is computed using the following formula:
desiredReplicas = ceil[ currentReplicas * ( currentMetricValue / targetMetricValue ) ]
If the ratio is close to 1.0 (within a default 10% tolerance), the controller skips scaling to prevent unnecessary replica fluctuations.
Internal Architecture
HPA works by communicating with three distinct APIs depending on the configuration:
- Resource Metrics API (
metrics.k8s.io): Typically provided by the Metrics Server, which aggregates CPU/Memory usage metrics directly from the Kubelet's cAdvisor agent on each node. - Custom Metrics API (
custom.metrics.k8s.io): Allows scaling on internal application metrics (like HTTP request rate or database connection counts) pulled from external monitors (e.g., Prometheus Adapter). - External Metrics API (
external.metrics.k8s.io): Scales based on resources outside the cluster, such as cloud message queue lengths (e.g., AWS SQS queue depth).
Visual Explanation
Practical Example
Below is an HPA manifest scaling a deployment named web-api between 2 and 10 replicas, maintaining a target average CPU utilization of 60%:
Common Mistakes
- Missing Resource Requests: HPA calculates utilization as a percentage of the pod's configured resource request. If requests are not defined in the Pod specification, HPA cannot calculate the percentage and will fail to scale.
- Flapping (Thrashing): Occurs when load shifts rapidly, causing HPA to scale up, scale down, and scale up again in short intervals. This causes container restart overhead and scheduling lag.
- Co-configuring HPA and VPA on Resource Metrics: Using both HPA and VPA to scale on CPU/Memory usage. This causes conflicts where VPA increases pod sizes while HPA scales replica counts, leading to resource loops.
Quick Quiz
Q1: If an HPA has a target CPU utilization of 50%, the current replicas is 2, and the current average CPU utilization is 75%, how many replicas will the HPA desire?
A) 2 replicas
B) 3 replicas
C) 4 replicas
D) 5 replicas
Answer: B — Using the formula: desiredReplicas = ceil[ 2 * ( 75 / 50 ) ] = ceil[ 2 * 1.5 ] = ceil[ 3 ] = 3 replicas.
Q2: Which configuration parameter is used to explicitly prevent rapid scale-down flapping in HPA?
A) minReplicas
B) scaleTargetRef
C) scaleDown stabilizationWindowSeconds (in behavior config)
D) horizontalPodAutoscalerSyncPeriod
Answer: C — The stabilization window allows specifying a duration (e.g., 300 seconds) that must pass before HPA will scale down, filtering out short, transient load drops.
Scenario-Based Challenge
Production Scenario:
During marketing campaigns, your API receives burst traffic that doubles in under 30 seconds. By default, HPA scales up gradually, but the application experiences high latency and drops requests because new pods take 45 seconds to compile and start up. How do you configure HPA behavior to scale up aggressively during bursts?
View Solution
To handle sudden traffic spikes, you must customize the behavior block in the autoscaling/v2 HPA spec to increase the scale-up velocity:
Add a behavior configuration to allow scaling up to the maximum replicas or adding a large percentage immediately, bypassing default rate limits:
Setting
stabilizationWindowSeconds: 0 for scale-up ensures immediate action, while selectPolicy: Max picks the policy that adds the highest number of pods.
Debugging Exercise
Broken Configuration:
You applied the HPA manifest, but running kubectl get hpa shows <unknown>/60% under TARGETS. The HPA refuses to scale:
Debugging Steps:
1. Run kubectl describe hpa web-api-hpa. You see an error: missing request for cpu on container web-app.
2. Check the deployment template: kubectl get deploy web-api -o yaml. Verify if resource requests are present under the container specifications.
The Fix: Add CPU requests to the container spec in the Deployment manifest:
Interview Questions
1. Conceptual: How does HPA compute the target number of replicas? Explain the calculation formula.
HPA calculates the target replicas based on the ratio of current metrics to target metrics: desiredReplicas = ceil[ currentReplicas * ( currentMetricValue / targetMetricValue ) ]. The controller takes the average metric value across all pods in the scaling target group. If custom behaviors or multi-metrics are defined, HPA calculates desired replicas for each metric separately and selects the highest desired replica count to ensure safety.
2. Technical: How would you configure HPA to scale based on a custom metric like RabbitMQ queue length or HTTP request rate?
To scale on custom metrics:
1. Deploy a metric collector (like Prometheus) and a metrics API adapter (like Prometheus Adapter) to map custom metrics into the Kubernetes API server.
2. Register the metrics under the custom.metrics.k8s.io API.
3. Define the metric source type as Object or Pods in the HPA manifest:
metrics:
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: 100
Production Considerations
Always set realistic minReplicas to sustain background load and maxReplicas to prevent run-away autoscaling from consuming the cluster budget. Test scale-down stabilization windows (default 5 minutes) to avoid thrashing during intermittent network bursts. Ensure the Metrics Server is configured with high availability (HA) in large clusters.