Configure Kubernetes Horizontal Pod Autoscaler with custom metrics, scaling policies, and behavior tuning for predictable production auto-scaling.
Beyond Basic CPU Autoscaling
The Horizontal Pod Autoscaler (HPA) is one of Kubernetes' most powerful features, yet most deployments use only the simplest configuration - scale when CPU exceeds 50 percent. In production, CPU-based scaling often reacts too slowly or too aggressively, leading to either request queuing or resource waste.
This guide covers advanced HPA configuration including custom metrics, scaling behaviors, and stabilization windows that we implement as part of our Kubernetes management practice.
How HPA Works Internally
The HPA controller runs a control loop every 15 seconds (configurable via --horizontal-pod-autoscaler-sync-period). It queries the metrics API, calculates the desired replica count using the formula:
desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue))
The controller then applies scaling policies, stabilization windows, and tolerance thresholds before issuing a scale command.
The Tolerance Band
By default, HPA has a 10 percent tolerance (--horizontal-pod-autoscaler-tolerance=0.1). If the ratio of current to desired metric value falls within 0.9 to 1.1, no scaling action occurs. This prevents oscillation but can delay response to genuine load changes.
Multi-Metric HPA Configuration
Production services should scale based on multiple signals:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: web-api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: web-api
minReplicas: 3
maxReplicas: 50
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 65
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "100"
behavior:
scaleUp:
stabilizationWindowSeconds: 30
policies:
- type: Percent
value: 50
periodSeconds: 60
- type: Pods
value: 5
periodSeconds: 60
selectPolicy: Max
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 10
periodSeconds: 120
selectPolicy: Min
Understanding the Behavior Section
Scale-up uses a 30-second stabilization window and two policies: add 50 percent of current replicas or 5 pods, whichever is greater (selectPolicy: Max). This allows aggressive scale-up during traffic spikes.
Scale-down uses a 300-second (5-minute) stabilization window and a conservative policy: remove at most 10 percent of replicas every 2 minutes (selectPolicy: Min). This prevents the cluster from scaling down too quickly after a traffic spike, which would cause problems if traffic returns.
Custom Metrics with Prometheus
The built-in CPU and memory metrics are limited. Real application health is better measured by request latency, queue depth, or error rates.
Setting Up the Metrics Pipeline
Install the Prometheus Adapter to expose custom metrics to the HPA:
# prometheus-adapter-config.yaml
rules:
- seriesQuery: 'http_request_duration_seconds_count{namespace!="",pod!=""}'
resources:
overrides:
namespace: {resource: "namespace"}
pod: {resource: "pod"}
name:
matches: "^(.*)_count$"
as: "http_requests_per_second"
metricsQuery: 'rate(<<.Series>>{<<.LabelMatchers>>}[2m])'
- seriesQuery: 'rabbitmq_queue_messages{namespace!=""}'
resources:
overrides:
namespace: {resource: "namespace"}
name:
as: "queue_depth"
metricsQuery: 'avg(<<.Series>>{<<.LabelMatchers>>})'
Verifying Custom Metrics
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/production/pods/*/http_requests_per_second" | jq .
Scaling Based on Queue Depth
For worker deployments that process jobs from a message queue, scale based on queue depth rather than CPU:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: worker-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: queue-worker
minReplicas: 2
maxReplicas: 30
metrics:
- type: Object
object:
describedObject:
apiVersion: v1
kind: Service
name: rabbitmq
metric:
name: queue_depth
target:
type: Value
value: "50"
This configuration maintains roughly 50 messages per worker pod. As the queue grows, new workers spawn. As it drains, workers scale down.
Scaling Pitfalls to Avoid
Cold Start Penalty
If your application takes 30 seconds to start, newly scaled pods cannot serve traffic during that window. Combine HPA with proper readiness probes and consider overprovisioning by setting minReplicas high enough to handle average load without scaling.
Resource Requests and Limits
HPA CPU percentage is calculated against resources.requests, not limits. If requests are set too low, the HPA sees artificially high utilization and over-scales:
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: "1"
memory: 1Gi
Set requests to match your P50 usage and limits to match P99 usage.
Cluster Autoscaler Interaction
HPA adds pods, but those pods need nodes to run on. Ensure the Cluster Autoscaler is configured with matching instance types and can provision nodes faster than your scale-up rate. A node that takes 5 minutes to provision defeats the purpose of a 30-second scaling reaction.
Monitoring HPA Performance
Track these metrics to verify HPA effectiveness:
kube_hpa_status_current_replicasvskube_hpa_spec_max_replicas- if current regularly hits max, increase the ceilingkube_hpa_status_condition{condition="ScalingLimited"}- indicates HPA wants to scale but cannot- Request latency P99 during scale-up events - reveals whether scaling is fast enough
Use these metrics to iteratively tune stabilization windows and scaling policies for your specific workload patterns. For hands-on help configuring auto-scaling, explore our server optimization services.
Frequently asked questions
How often does the Kubernetes HPA recalculate the replica count?
The HPA controller runs its control loop every 15 seconds, configurable with --horizontal-pod-autoscaler-sync-period. Each pass computes desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue)), then applies scaling policies, stabilization windows and the tolerance band before issuing a scale command.
Why doesn't the Kubernetes HPA react to small changes in load?
Small deviations fall inside the HPA's default 10 percent tolerance, set with --horizontal-pod-autoscaler-tolerance=0.1. When the ratio of the current to the desired metric value sits between 0.9 and 1.1, no scaling happens. The band prevents oscillation, though it can delay the response to genuine load changes.
How do I make the Kubernetes HPA scale up fast and scale down slowly?
Tune the behavior section of an autoscaling/v2 HPA. For scale-up, a 30-second stabilization window with two policies, 50 percent of current replicas or 5 pods per 60 seconds, and selectPolicy Max adds whichever is greater. For scale-down, a 300-second window with at most 10 percent of replicas removed every 120 seconds and selectPolicy Min keeps the cluster from shrinking too fast after a traffic spike.
How do I autoscale Kubernetes pods on requests per second with Prometheus?
Install the Prometheus Adapter and add a rule that turns http_request_duration_seconds_count into a per-pod http_requests_per_second metric using rate over 2m. Reference it in the HPA as a Pods metric with an AverageValue target, for example 100. Confirm the metric is exposed with kubectl get --raw on /apis/custom.metrics.k8s.io/v1beta1/namespaces/production/pods/*/http_requests_per_second piped to jq.
How do I scale Kubernetes queue workers on RabbitMQ queue depth?
Expose rabbitmq_queue_messages through the Prometheus Adapter as a queue_depth metric, then point the worker HPA at it with an Object metric whose describedObject is the rabbitmq Service. A target value of 50 keeps roughly 50 messages per worker pod, with the worker Deployment ranging from 2 to 30 replicas. Workers spawn as the queue grows and scale down as it drains, which fits job processors better than CPU-based scaling.
Does the Kubernetes HPA measure CPU utilization against requests or limits?
Against requests. HPA CPU percentage is calculated from resources.requests, so requests set too low make utilization look artificially high and the HPA over-scales. Set requests to match P50 usage and limits to match P99, for example requests of 250m CPU and 512Mi memory with limits of 1 CPU and 1Gi.
Why are new pods slow to take traffic when the Kubernetes HPA scales up?
Two delays sit outside the HPA. An application that takes 30 seconds to start cannot serve traffic during that window, so pair the HPA with proper readiness probes and set minReplicas high enough to handle average load without scaling. New pods also need nodes, and a node that takes 5 minutes to provision defeats a 30-second scaling reaction, so configure the Cluster Autoscaler with matching instance types.
How can I tell if the Kubernetes HPA is hitting its limits?
Compare kube_hpa_status_current_replicas with kube_hpa_spec_max_replicas, and raise the ceiling if current replicas regularly reach the maximum. The kube_hpa_status_condition metric with condition ScalingLimited shows when the HPA wants to scale and cannot. P99 request latency during scale-up events reveals whether scaling is fast enough.
Read how we handle it in Cloud & Server Management.
Related Articles
How to Troubleshoot and Fix the "Error Establishing a Database Connection" in WordPress
An in-depth troubleshooting guide for the WordPress database connection error, covering wp-config.php verification, password resets, database repair, MySQL service management, and preventive measures.
Server & DevOpsUpgrading Magento 2 to the Latest Version in 10 Steps
A structured 10-step process for upgrading Magento 2 from v2.4.3 to v2.4.7-p3 on Ubuntu, covering system preparation, backups, staging setup, Composer updates, custom module handling, and post-upgrade optimization.
Server & DevOpsHow to Upgrade PostgreSQL Major Versions With Almost No Downtime
An in-place major upgrade takes your database down for as long as the upgrade runs, and once it has started there is no way back. Logical replication turns that into a cutover you can measure in seconds, with the old server still consistent and still able to take traffic if the first minute goes badly. The method works because the new server is built and caught up while the old one keeps serving. The risk is entirely in what logical replication declines to carry across, so this guide spends most of its time on sequences, DDL, large objects and tables without a replica identity.