Kubernetes autoscaling becomes much more useful when an application can scale based on the metric that actually represents demand.

CPU and memory are useful for many workloads, but they are not always the best signals.

A message-processing service might need to scale based on queue depth.

An API might need to scale based on requests per second.

A machine learning service might need to scale based on GPU cache utilization.

A database-backed application might need to react to request latency or throughput.

Google Kubernetes Engine now provides built-in support for using PromQL queries with Horizontal Pod Autoscaling (HPA). The integration allows HPA to use Prometheus metrics available through Google Cloud Managed Service for Prometheus without requiring a third-party Prometheus adapter.

The architectural change is straightforward:

Code
Application
    |
    v
Prometheus Metrics
    |
    v
Managed Service for Prometheus
    |
    v
PromQL Query
    |
    v
GKE HPA
    |
    v
Pod Count

This gives developers considerably more control over what "demand" means for their workloads.

Why CPU Is Often the Wrong Autoscaling Signal

Consider a web API.

Its traffic might look like:

Code
Requests/second: 1000
CPU:              35%

The service may be under significant load while CPU remains relatively low.

Now consider a different workload:

Code
Requests/second: 100
CPU:              85%

The second service might need more replicas even though it receives fewer requests.

CPU utilization tells you how busy the containers are.

It does not necessarily tell you how much work the application has waiting.

That is why application-specific metrics can produce better scaling decisions.

What PromQL Adds

PromQL is the query language used by Prometheus.

It allows developers to transform raw metrics into signals that are more useful for autoscaling.

For example:

SQL
Raw metric
    |
    v
PromQL
    |
    +--> Average
    +--> Rate
    +--> Sum
    +--> Percentile
    +--> Filter
    +--> Aggregate
    |
    v
Autoscaling signal

Instead of simply saying:

Code
CPU > 70%

you can define a workload-specific condition such as:

Code
Requests per second per Pod > target

or:

Code
Queue depth per worker > target

This makes the autoscaler much closer to the application's actual workload characteristics.

The New Architecture

Previously, teams using Prometheus metrics with Kubernetes HPA often needed an adapter between Prometheus and the Kubernetes autoscaling APIs.

A simplified architecture looked like:

Code
Application
    |
    v
Prometheus
    |
    v
Prometheus Adapter
    |
    v
Kubernetes HPA

With GKE's built-in support:

Code
Application
    |
    v
Managed Service for Prometheus
    |
    v
PromQL
    |
    v
GKE Autoscaling Integration
    |
    v
HPA

The important benefit is not merely fewer components.

Removing a separately managed adapter also removes some configuration, authentication, deployment, upgrade, and troubleshooting work.

Google specifically describes the new capability as built-in PromQL processing for HPA, using the existing AutoscalingMetric integration.

Why Application Metrics Are Better

Imagine a background worker processing messages.

CPU might be:

Code
CPU = 20%

But the queue could contain:

Code
Pending messages = 50,000

CPU alone might not cause enough scaling.

Queue depth is a much better representation of demand.

The desired architecture is:

Code
Queue
  |
  v
Prometheus / Cloud Monitoring
  |
  v
PromQL
  |
  v
HPA
  |
  v
Workers

As the queue grows, more workers are created.

As the queue drains, the workload can scale back down.

PromQL Makes the Metric More Expressive

Suppose an application exposes:

Code
http_requests_total

A raw counter is not directly useful as an autoscaling signal.

PromQL can transform it into a rate.

Conceptually:

Code
rate(http_requests_total[2m])

Now the autoscaler can reason about requests per second rather than the total number of requests since the application started.

Similarly, multiple series can be aggregated:

Code
sum(rate(http_requests_total[2m]))

The exact query should reflect the workload's architecture and metric labels.

Percentiles Can Be More Useful Than Averages

Latency is another example.

Suppose average latency is:

Code
Average = 120 ms

but the 99th percentile is:

Code
P99 = 2.5 seconds

The average may look healthy while a meaningful portion of users are experiencing slow responses.

PromQL allows more expressive calculations, including percentile-oriented queries when the underlying metric type supports them.

That means autoscaling can be based on a signal closer to the actual user experience.

However, developers should be careful.

Scaling directly on latency can create feedback loops if poorly designed.

Autoscaling Should Follow Demand

A good autoscaling metric answers:

How much work does the application currently have relative to its available capacity?

Examples include:

Code
Requests per second
Queue depth
Concurrent requests
Messages per worker
Active sessions
GPU cache utilization
Pending jobs

Poor signals include metrics that are correlated with load but do not directly represent capacity.

For example:

Code
Heap size
Number of log messages
Garbage collections

These may be useful operational signals, but they do not necessarily tell the autoscaler how many replicas are required.

A Simple PromQL-Based Design

Suppose a .NET API exposes:

Code
http_server_requests_total

A conceptual PromQL query could calculate request rate:

Code
sum(
  rate(http_server_requests_total[2m])
)

The autoscaler can then use that metric to determine when additional replicas are required.

The architecture becomes:

Code
ASP.NET Core
     |
     v
Prometheus Metrics
     |
     v
PromQL
     |
     v
HPA
     |
     +--> 1 Pod
     +--> 2 Pods
     +--> 5 Pods

The exact metric name and labels depend on how the application exposes telemetry.

A GKE AutoscalingMetric

GKE uses an AutoscalingMetric resource to define PromQL-backed metrics for autoscaling.

A simplified example is:

Code
apiVersion: autoscaling.gke.io/v1beta1
kind: AutoscalingMetric
metadata:
  name: api-request-rate
  namespace: production
spec:
  metrics:
    - promql:
        name: requests-per-second
        query: |
          sum(
            rate(http_server_requests_total[2m])
          )
        type: External

The metric can then be referenced by an HPA.

The exact query and metric type should match the metric's semantics and the workload being scaled.

GKE supports PromQL-backed custom Pod metrics and external metrics through the managed autoscaling infrastructure.

Connecting the Metric to HPA

A simplified HPA can look like:

VB.NET
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: External
      external:
        metric:
          name: autoscaling.gke.io|api-request-rate|requests-per-second
        target:
          type: AverageValue
          averageValue: "100"

This example expresses a simple policy:

Code
Target:
approximately 100 requests/second per replica

The actual target should be determined through load testing.

Do not choose a value simply because it looks reasonable.

Per-Pod Versus External Metrics

This distinction matters.

A Pod metric describes a metric associated with individual pods.

An external metric represents something outside the pod, such as:

Code
Queue depth
Cloud service activity
Pub/Sub messages
External workload demand

The choice affects how the HPA interprets the value.

For example:

Code
Pod metric:
"How busy is each pod?"

External metric:
"How much work exists outside the pods?"

Queue depth is usually an external signal.

CPU utilization is a resource signal.

Request rate can be modeled according to how the metric is exposed and how the desired scaling policy is defined.

Multiple Metrics Can Be Combined

Real applications often have more than one bottleneck.

For example:

Code
CPU utilization
+
Request rate
+
Queue depth

An HPA can use multiple metrics.

This can protect against cases where one signal is misleading.

Consider:

Code
CPU = 30%
Queue = 10,000

The queue signal indicates significant work even though CPU is low.

Or:

Code
CPU = 90%
Queue = 0

The application may be CPU-bound even without a backlog.

Multiple metrics can therefore provide a more complete picture.

However, every additional metric increases autoscaling complexity.

PromQL Filters Are Critical

Prometheus metrics frequently contain labels such as:

Code
namespace
pod
cluster
service
region
instance

An autoscaling query must avoid accidentally aggregating unrelated workloads.

For example, this can be dangerous:

Code
sum(rate(http_requests_total[2m]))

if multiple applications expose the same metric name.

A safer query might constrain the relevant workload:

Code
sum(
  rate(
    http_requests_total{
      namespace="production",
      service="orders"
    }[2m]
  )
)

Google's guidance specifically recommends using filters such as namespace or cluster when PromQL queries could otherwise combine identically named metrics from different applications.

Metric Freshness Matters

Autoscaling decisions are time-sensitive.

If a metric arrives late, the HPA may make decisions using stale information.

Google notes that metrics stored through Managed Service for Prometheus are typically queryable a few seconds after ingestion, although freshness is not guaranteed and network latency can add additional delay.

That means autoscaling logic should not be excessively aggressive.

For example, a query that reacts instantly to every tiny change may cause:

Code
Scale up
   |
   v
Scale down
   |
   v
Scale up
   |
   v
Scale down

This is scaling thrash.

Stabilization Windows

Autoscaling systems need hysteresis.

You generally want:

Code
Demand rises
   |
   v
Scale up relatively quickly

Demand falls
   |
   v
Wait for sustained lower demand
   |
   v
Scale down

GKE's HPA supports stabilization windows and other scaling behaviors that help avoid unnecessary oscillation.

For bursty applications, scale-up should usually be more responsive than scale-down.

Scaling to Zero

The PromQL integration also becomes interesting when combined with GKE's scale-to-zero capabilities.

GKE supports HPA scaling to zero on supported versions when using an external or object metric.

A workload can therefore follow:

Code
No work
   |
   v
0 Pods
   |
   v
PromQL metric > 0
   |
   v
1+ Pods
   |
   v
Process workload

Resource metrics such as CPU and memory cannot provide the activation signal when there are zero pods.

An external or object metric is required.

This makes PromQL particularly useful for event-driven workloads.

Queue-Based Autoscaling

Consider a worker processing jobs from a queue.

The metric might represent:

Code
pending_jobs

A PromQL expression could aggregate queue metrics:

Code
sum(
  pending_jobs{
    namespace="production",
    queue="image-processing"
  }
)

The HPA can then use the result to determine worker capacity.

The architecture is:

Code
Job Producer
     |
     v
Queue
     |
     +----> Prometheus Metric
     |             |
     |             v
     |          PromQL
     |             |
     |             v
     |            HPA
     |             |
     |             v
     +--------> Workers

This is often more meaningful than scaling workers based on CPU.

GPU Workloads

AI workloads are another strong candidate.

Suppose a model server exposes:

Code
GPU cache utilization

or another application-specific metric.

The autoscaler can use the metric to determine when more replicas are needed.

Google's GKE documentation gives vLLM GPU cache utilization as an example of a metric that can be used for autoscaling.

The architecture can look like:

Code
Inference Requests
       |
       v
Model Server
       |
       v
GPU Utilization Metric
       |
       v
PromQL
       |
       v
HPA
       |
       v
More Model Replicas

This is more meaningful than blindly scaling on CPU for GPU-heavy inference.

.NET Applications Fit Naturally

ASP.NET Core applications can expose Prometheus-compatible metrics through standard .NET observability libraries.

A service might expose:

Code
http_server_requests_total
http_server_request_duration
active_requests

The metrics pipeline then becomes:

Code
ASP.NET Core
     |
     v
OpenTelemetry / Prometheus Metrics
     |
     v
Managed Service for Prometheus
     |
     v
PromQL
     |
     v
GKE HPA

The important design decision is choosing metrics that represent actual capacity.

Do Not Scale Directly on Every Business Metric

Prometheus can expose thousands of metrics.

That does not mean every metric should drive autoscaling.

For example:

Code
Orders created
Users logged in
Emails sent
Cache misses
Exceptions

These can be useful for monitoring.

But the autoscaling signal should answer:

Does this metric indicate that the current number of replicas is insufficient?

For example:

Code
Orders per second

might be useful.

But:

Code
Total orders ever created

is not.

Avoid Autoscaling on Noisy Metrics

Suppose a metric fluctuates:

Code
98
102
97
105
99
103

If the target is 100 and the HPA reacts aggressively to every change, replicas may oscillate.

Instead, use:

Code
Rate windows
Averages
Percentiles
Smoothing
Stabilization

PromQL can transform noisy raw signals into more stable indicators.

The goal is not the fastest possible reaction.

The goal is the correct reaction.

Autoscaling Is a Feedback Control System

It is useful to think of HPA as a control loop:

Code
Demand
   |
   v
Metric
   |
   v
PromQL
   |
   v
HPA Decision
   |
   v
Replica Count
   |
   v
Application Capacity
   |
   +------> Demand Served

The output changes the system that produces the metric.

That means poorly designed scaling policies can create feedback instability.

For example:

Code
Scale up
   |
   v
Capacity increases
   |
   v
Metric falls
   |
   v
Scale down too quickly
   |
   v
Metric rises
   |
   v
Scale up again

Good autoscaling requires appropriate time windows and stabilization.

Performance Considerations

PromQL-based autoscaling introduces additional metric-query work.

Consider:

Code
Number of HPAs
      x
Number of metrics
      x
Query complexity
      x
Metric cardinality

The cost can increase quickly.

GKE's Performance HPA profile is designed to improve reaction time and support larger numbers of HPA objects. Google documents support for up to thousands of HPA objects under qualifying configurations.

Still, developers should keep queries efficient.

Avoid unnecessarily expensive PromQL expressions for an autoscaler.

High Cardinality Can Become a Problem

Prometheus metrics can have many label combinations.

For example:

Code
user_id
request_id
session_id
transaction_id

These labels can produce extremely high cardinality.

They may be useful for traces or application diagnostics, but they are usually poor choices for a scaling metric.

Prefer relatively stable dimensions:

Code
service
namespace
cluster
region
queue
endpoint

The autoscaling query should operate on a manageable metric set.

Common Mistakes

Scaling on CPU when the workload is queue-bound

Use the metric that represents actual demand.

Writing an unfiltered PromQL query

You may accidentally aggregate metrics from unrelated workloads.

Using a noisy metric

Autoscaling can oscillate if the signal changes too frequently.

Ignoring metric freshness

Delayed metrics can produce delayed scaling decisions.

Using CPU for scale-from-zero

There are no pods from which to collect CPU utilization when the workload is at zero.

Using high-cardinality labels

They can make metric processing unnecessarily expensive.

Choosing arbitrary thresholds

Use load testing and capacity measurements to establish realistic targets.

Adding too many metrics

Every metric adds complexity to the control loop.

Ignoring downstream bottlenecks

Scaling API replicas does not help if the database is already saturated.

Advantages and Disadvantages

Advantages

More expressive autoscaling: PromQL allows developers to transform metrics into application-specific scaling signals.

Fewer infrastructure components: Built-in GKE support can eliminate the need for a separately managed Prometheus adapter.

Better workload awareness: Applications can scale based on queue depth, request rate, latency, or other meaningful metrics.

Good fit for AI workloads: GPU and model-serving metrics can become autoscaling inputs.

Works with managed Prometheus: Existing Prometheus telemetry can become part of the scaling architecture.

Supports sophisticated queries: Aggregation, filtering, rates, and other PromQL capabilities provide more control than simple resource metrics.

Disadvantages

More configuration: PromQL-based scaling is more complex than CPU-based HPA.

Query design matters: A bad query can cause incorrect scaling.

Metric latency exists: Monitoring pipelines are not instantaneous.

Higher operational complexity: Developers need to understand Prometheus labels, query semantics, and HPA behavior.

Scaling can still oscillate: Poorly tuned thresholds and stabilization settings can create instability.

A Practical Adoption Strategy

Teams can introduce PromQL-based autoscaling incrementally:

  1. Identify a workload where CPU or memory does not represent demand accurately.

  2. Define the business or technical signal that represents capacity.

  3. Expose that metric using Prometheus-compatible instrumentation.

  4. Store or expose the metric through the supported GKE metrics pipeline.

  5. Write a PromQL query that filters the correct workload.

  6. Validate the query independently before connecting it to HPA.

  7. Establish a target using load testing.

  8. Configure the AutoscalingMetric.

  9. Connect the metric to an autoscaling/v2 HPA.

  10. Configure minimum and maximum replicas.

  11. Tune scale-up and scale-down behavior.

  12. Test burst traffic and metric delays.

  13. Monitor scaling decisions in production.

This approach is safer than immediately connecting an untested metric to production autoscaling.

What Developers Should Take Away

The most important change is conceptual.

Kubernetes autoscaling no longer has to think only in terms of:

Code
CPU
Memory

It can reason about metrics that describe the application's actual workload:

Code
Requests
Queues
Latency
Concurrent work
GPU utilization
Application capacity

PromQL makes those signals programmable.

A strong architecture looks like:

Code
Application
    |
    v
Meaningful Metrics
    |
    v
PromQL
    |
    v
HPA
    |
    v
Replica Count

The quality of the scaling behavior depends heavily on the quality of the metric.

If the metric does not represent demand, a sophisticated PromQL query will not fix the problem.

Summary

GKE's built-in PromQL support for Horizontal Pod Autoscaling gives Kubernetes workloads a more expressive way to decide when to add or remove replicas.

The major benefit is the ability to move beyond generic infrastructure metrics and scale according to signals that actually describe workload demand.

For example:

Code
Queue depth
Requests per second
Latency
GPU cache utilization
Concurrent jobs

can often be more useful than CPU utilization alone.

The architecture becomes:

Code
Application
      |
      v
Prometheus Metrics
      |
      v
PromQL
      |
      v
GKE Autoscaling
      |
      v
Horizontal Pod Autoscaler
      |
      v
Kubernetes Replicas

For .NET developers, this opens an especially useful pattern: instrument ASP.NET Core or background workers with meaningful metrics, expose them through the managed Prometheus pipeline, and let GKE use those metrics directly for scaling decisions.

The important engineering principle is scale on demand, not merely on resource consumption.

When the metric accurately represents the work the application needs to perform, HPA becomes a much more effective control loop for performance, reliability, and infrastructure efficiency.