Managing Batch and GPU Job Queues in Kubernetes With Kueue

Kubernetes can run many types of workloads, including web applications, scheduled jobs, machine learning training, and large-scale data processing. However, managing batch workloads becomes difficult when multiple teams compete for limited CPU, memory, and GPU resources.

Consider a Kubernetes cluster with eight GPUs. One team submits several model-training jobs, while another team needs GPU capacity for an experiment. If every job is scheduled as soon as Kubernetes can place its Pods, one workload may consume most of the available resources and delay other teams.

This is where Kueue helps.

Kueue is a Kubernetes-native job queueing system that manages when workloads are admitted to run. It works with Kubernetes scheduling and supported workload types, allowing administrators to define resource quotas, queue policies, and admission rules.

Instead of allowing every job to compete for cluster resources immediately, Kueue can hold workloads in a queue until the required resources are available.

This article explains how Kueue works, how to configure queues, and how to use it to manage GPU-intensive batch workloads.

What Is Kueue?

Kueue is a job queueing system designed for Kubernetes. It manages workload admission based on resource availability, quotas, and configured policies.

It does not replace the Kubernetes scheduler. Instead, Kueue determines when a workload is allowed to begin consuming cluster resources. Once admitted, the Kubernetes scheduler places the workload's Pods on suitable nodes.

A typical workflow looks like this:

1. A user submits a batch job.
2. Kueue recognizes the workload and evaluates its resource requirements.
3. The workload waits in a queue if the required quota is unavailable.
4. Kueue admits the workload when the applicable resource and policy requirements are satisfied.
5. Kubernetes schedules the workload's Pods.
6. When the workload finishes and releases its resource reservation, capacity can become available for other queued workloads.

This separation is useful because Kubernetes scheduling and workload admission solve different problems.

The Kubernetes scheduler decides where a Pod should run. Kueue helps decide whether a workload should be allowed to start consuming resources in the first place.

Important Kueue components

These components work together to provide centralized resource management without requiring every team to manage the cluster independently.

Why GPU Workloads Need Queue Management

GPU workloads often have stricter resource requirements than ordinary application Pods.

A model-training job might require four GPUs on a compatible node, while a smaller inference experiment may need only one GPU. If the cluster has limited GPU capacity, starting a large number of jobs simultaneously can leave some workloads pending for long periods.

Queue management helps administrators define how that capacity is allocated.

Common use cases include:

• Machine learning model training
• Distributed GPU workloads
• Batch inference
• Scientific computing
• Large-scale data processing
• Scheduled experimentation across multiple teams

For example, a platform team might reserve a quota for the research team and another quota for the production analytics team. Kueue can use those quotas to control workload admission.

However, a queue does not create additional GPUs. It controls access to the capacity that the cluster and its configured resource-sharing policies make available.

Installing Kueue and Preparing the Cluster

Before configuring queues, install a Kueue release compatible with your Kubernetes version and workload requirements.

Installation methods and manifests can differ between releases, so use the instructions for the version selected by your platform team.

After installation, verify that the Kueue controller is running in its namespace.

kubectl get pods -n kueue-system

The namespace shown here is the common installation namespace; verify the actual namespace used by your installation.

You can also inspect the installed custom resource definitions:

kubectl get crd | grep kueue.x-k8s.io

A cluster intended for GPU workloads must also have suitable GPU nodes, the required device drivers, and the Kubernetes device plugin or supported resource integration that advertises GPU capacity.

For example, a node might advertise NVIDIA GPU capacity through the extended resource nvidia.com/gpu.

Inspect node resources before creating queue quotas:

kubectl describe nodes

Check the node's capacity and allocatable resources, along with labels and taints that may affect scheduling.

Kueue can admit a workload according to its queue configuration, but the Kubernetes scheduler still needs a suitable node on which to run it.

Defining GPU Resource Flavors

A ResourceFlavor allows Kueue to distinguish between resource categories. This is useful when a cluster contains different GPU models, machine types, or node groups.

For example, a platform might label nodes that have a particular GPU configuration:

kubectl label nodes gpu-node-01 accelerator=nvidia-a100

A corresponding ResourceFlavor can identify that node category.

apiVersion: kueue.x-k8s.io/v1beta1
kind: ResourceFlavor
metadata:
  name: a100-gpu
spec:
  nodeLabels:
    accelerator: nvidia-a100

The example uses the v1beta1 API version for illustration. Confirm that the installed Kueue release supports the API version and fields used in your manifests.

The label must match the actual node configuration. If no suitable nodes have the label, workloads requiring this flavor may remain pending even when the queue has quota available.

Resource flavors are particularly useful when the cluster contains hardware that is not interchangeable. A workload requiring one GPU model should not be admitted against a quota intended for a different model unless the configuration explicitly permits that resource selection.

Creating a ClusterQueue

A ClusterQueue defines resource quotas and workload admission policies.

The following example configures a queue with a quota for GPUs and CPU resources.

apiVersion: kueue.x-k8s.io/v1beta1
kind: ClusterQueue
metadata:
  name: research-gpu-queue
spec:
  namespaceSelector: {}
  resourceGroups:
    - coveredResources:
        - cpu
        - memory
        - nvidia.com/gpu
      flavors:
        - name: a100-gpu
          resources:
            - name: cpu
              nominalQuota: "32"
            - name: memory
              nominalQuota: 128Gi
            - name: nvidia.com/gpu
              nominalQuota: "8"

This configuration illustrates a quota of 32 CPU cores, 128 GiB of memory, and eight GPUs associated with the selected flavor.

The quota values are examples, not a recommendation for every cluster. They should reflect the resources available to Kueue and the platform's allocation policies.

The namespaceSelector determines which namespaces may use the ClusterQueue through the supported queue configuration. An empty selector generally matches all namespaces, so production environments should apply namespace restrictions when workloads need to be isolated by team or environment.

The quota also does not mean that the cluster contains eight GPUs. Administrators must ensure that the cluster's real capacity and any other resource consumers are accounted for correctly.

Connecting a LocalQueue to the ClusterQueue

A LocalQueue provides a namespace-scoped queue for users. It references a ClusterQueue that controls resource allocation.

First, create a namespace for the research team:

apiVersion: v1
kind: Namespace
metadata:
  name: research

Then create the LocalQueue:

apiVersion: kueue.x-k8s.io/v1beta1
kind: LocalQueue
metadata:
  name: gpu-jobs
  namespace: research
spec:
  clusterQueue: research-gpu-queue

Apply the manifests:

kubectl apply -f research-namespace.yaml
kubectl apply -f gpu-resource-flavor.yaml
kubectl apply -f research-cluster-queue.yaml
kubectl apply -f research-local-queue.yaml

Use the actual filenames created for your manifests.

Once the queue is configured and active, workloads can reference it through the supported Kueue integration for their workload type.

Submitting a GPU Batch Job

A Kubernetes Job is a common way to run a finite batch workload. Kueue supports workload integrations that can manage supported jobs, but the exact configuration depends on the installed Kueue version and whether the relevant integration is enabled.

The following example illustrates a GPU training job using the Kueue queue label commonly used with the built-in Job integration.

apiVersion: batch/v1
kind: Job
metadata:
  name: model-training
  namespace: research
  labels:
    kueue.x-k8s.io/queue-name: gpu-jobs
spec:
  parallelism: 1
  completions: 1
  template:
    metadata:
      labels:
        app: model-training
    spec:
      restartPolicy: Never
      containers:
        - name: trainer
          image: example.com/ml/trainer:1.0
          command:
            - python
            - train.py
          resources:
            requests:
              cpu: "4"
              memory: 16Gi
              nvidia.com/gpu: "1"
            limits:
              cpu: "4"
              memory: 16Gi
              nvidia.com/gpu: "1"

Replace the example image with a valid image containing the training application and its dependencies.

The job requests one GPU, four CPU cores, and 16 GiB of memory. Its workload can be admitted only when the relevant queue policy and resource requirements permit it.

The queue label alone is not proof that Kueue is managing the Job. Confirm that the Job integration is enabled and that the workload is recognized by Kueue.

Apply the job:

kubectl apply -f model-training-job.yaml

Then inspect its status:

kubectl get jobs -n research
kubectl get pods -n research

You can also inspect the corresponding Kueue resources:

kubectl get localqueues -n research
kubectl get clusterqueues
kubectl get workloads -n research

These commands help distinguish a workload waiting for admission from a Pod that has already been created but cannot be scheduled.

Managing Multiple Teams and Priorities

A shared cluster often serves teams with different requirements. A research team may submit large experiments, while a production analytics team needs predictable access to compute capacity.

Kueue supports policies for managing how workloads are admitted. Depending on the configuration, administrators can use separate queues, quotas, priorities, and cohort-based resource sharing.

For example, separate ClusterQueues can define nominal quotas for research and analytics workloads. LocalQueues then give each team a namespace-scoped entry point.

This provides a clearer allocation model than allowing all teams to submit jobs without centralized admission control.

Prioritizing workloads

Priority classes can help determine which eligible workloads should be admitted first. The exact result depends on the configured queue policy, available quota, and any other admission requirements.

Priority does not automatically guarantee that a job will run immediately. A high-priority workload can still wait if its required resources are unavailable or if it cannot be scheduled onto compatible nodes.

Administrators should document what priority means for each team and avoid giving every workload the highest priority. Otherwise, the intended distinction between urgent and routine work becomes ineffective.

Sharing unused resources

Cohorts can allow ClusterQueues to share eligible unused capacity under configured rules.

This can improve utilization when one team is temporarily idle and another team has additional work to run.

However, resource sharing must be designed carefully. Quota sharing does not eliminate hardware limitations, node constraints, or the need to schedule Pods successfully.

Common Problems and How to Troubleshoot Them

Start troubleshooting with resource status and events rather than deleting workloads immediately.

For example:

kubectl describe clusterqueue research-gpu-queue
kubectl describe localqueue gpu-jobs -n research
kubectl get events -n research --sort-by=.lastTimestamp

The exact status fields and event messages vary by Kueue version and workload type. Use them to determine whether the problem is related to admission, quota, configuration, or actual Pod scheduling.

Best Practices for Production Environments

Use the following practices when operating Kueue in a shared Kubernetes cluster:

1. Define quotas from actual cluster capacity and account for other workloads that consume the same resources.
2. Separate teams or workload classes through clear namespace and queue policies.
3. Request realistic CPU, memory, and GPU resources so admission decisions reflect actual requirements.
4. Verify that ResourceFlavor labels match the nodes that provide the required hardware.
5. Monitor queued workload age, admission delays, resource utilization, and failed jobs.
6. Test quota changes and priority policies in a non-production environment before applying them to shared clusters.
7. Configure retry and failure behavior for batch jobs so transient application errors do not create repeated resource pressure.
8. Document the process for users whose workloads remain queued, including which status information administrators need for troubleshooting.

Kueue can improve resource allocation, but it cannot correct an application that requests the wrong GPU type, requires more memory than the cluster provides, or fails during execution.

Advantages and Limitations of Kueue

Kueue provides centralized workload admission, resource quotas, queueing, and policy-based allocation. These capabilities are useful when many teams share limited compute capacity or submit large batches of jobs.

It can reduce uncontrolled competition for resources and make workload admission more predictable.

However, Kueue introduces additional configuration and operational responsibilities. Administrators must maintain queues, quotas, flavors, integrations, and monitoring. Workloads can still remain pending because of Kubernetes scheduling constraints, and queueing cannot increase the physical capacity of the cluster.

For a small cluster running only a few simple jobs, the additional management layer may not be necessary. It becomes more valuable as the number of teams, workloads, and resource constraints grows.

Summary

Kueue helps Kubernetes administrators manage when batch workloads can start, rather than allowing every job to compete for resources immediately. Its queues, quotas, ResourceFlavors, and admission policies are particularly useful for shared GPU clusters and large data-processing workloads.

A successful setup requires more than installing Kueue. Administrators must configure realistic resource quotas, connect LocalQueues to ClusterQueues, enable the appropriate workload integrations, and ensure that Kubernetes has compatible nodes available.

By combining admission control with monitoring and clear resource policies, teams can manage shared compute capacity more consistently while keeping batch workloads under control.