Manage Kueue on Kueue

Setup RBAC

Mon, 14 Feb 2022 00:00:00 +0000

This page shows you how to setup role-based access control (RBAC) in your cluster to control the types of users that can view and create Kueue objects.

The page is intended for a batch administrator.

Before you begin

Make sure the following conditions are met:

Administer Cluster Quotas

Mon, 14 Mar 2022 00:00:00 +0000

This page shows you how to manage your cluster resource quotas and to establish Fair Sharing rules among the tenants.

The intended audience for this page are batch administrators.

Before you begin

Make sure the following conditions are met:

A Kubernetes cluster is running.
The kubectl command-line tool has communication with your cluster.
Kueue is installed.

Single ClusterQueue and single ResourceFlavor setup

In the following steps, you will create a queuing system with a single ClusterQueue and a single ResourceFlavor to govern the quota of your cluster.

Share Quotas Across Resource Flavors

Wed, 28 Jan 2026 00:00:00 +0000

This page demonstrates how to use resource transformations to enforce a unified quota that applies across multiple resource flavors.

A common use case is when you have multiple hardware types (for example, different CPU architectures or GPU models) but want to limit the total usage per team regardless of which flavor they consume. By defining a virtual “credits” resource, you can assign a cost to each flavor and enforce a combined quota.

Run job with WorkloadPriority

Mon, 02 Oct 2023 00:00:00 +0000

Usually, in Kueue, workload’s priority is calculated using for pod’s priority for queuing and preemption.
By using a WorkloadPriorityClass, you can independently manage the priority of workloads for queuing and preemption, separate from pod’s priority.

This page contains instructions on how to run a job with workload priority.

Before you begin

Make sure the following conditions are met:

Setup All-or-nothing with ready Pods

Mon, 14 Mar 2022 00:00:00 +0000

Some jobs need all pods to be running at the same time to operate; for example, synchronized distributed training or MPI-based jobs which require pod-to-pod communication. On a default Kueue configuration, a pair of such jobs may deadlock if the physical availability of resources do not match the configured quotas in Kueue. The same pair of jobs could run to completion if their pods were scheduled sequentially.

To address this requirement, in version 0.3.0 we introduced an opt-in mechanism configured via the flag waitForPodsReady that provides a simple implementation of the all-or-nothing scheduling. When enabled, the workload is monitored by Kueue until all of its Pods are ready (meaning scheduled, running, and passing the optional readiness probe). If not all pods of the workload are ready within the configured timeout, then the workload is evicted and requeued.

Set Up Dynamic Resource Allocation

Sun, 22 Mar 2026 00:00:00 +0000

This page shows you how to configure Kueue to account for DRA devices in quota management.

The intended audience for this page are batch administrators.

For conceptual details, see Dynamic Resource Allocation concepts. For instructions on submitting workloads with DRA devices, see Run Workloads With DRA Devices.

Before you begin

Make sure the following conditions are met:

Setup a MultiKueue environment

Mon, 26 Feb 2024 00:00:00 +0000

This tutorial explains how you can configure a management cluster and one worker cluster to run JobSets and batch/Jobs in a MultiKueue environment.

Check the concepts section for a MultiKueue overview.

Let’s assume that your manager cluster is named manager-cluster and your worker cluster is named worker1-cluster. To follow this tutorial, ensure that the credentials for all these clusters are present in the kubeconfig in your local machine. Check the kubectl documentation to learn more about how to Configure Access to Multiple Clusters.

Setup garbage-collection of workload

Fri, 16 May 2025 00:00:00 +0000

This guide shows you how to enable and configure optional object retention policies in Kueue to automatically delete finished or deactivated Workloads after a specified time. By default, Kueue leaves all Workload objects in cluster indefinitely; with object retention you can free up etcd storage and reduce Kueue’s memory footprint.

Prerequisites

A running Kueue installation at v0.12 or newer.

Set up a retention policy

Follow the instructions described here to install a release version by extending the configuration with the following fields:

Setup failure recovery

Thu, 04 Dec 2025 00:00:00 +0000

This describes how to configure the failure recovery mechanism that automatically transitions pods into the Failed phase when they are assigned to unreachable nodes and stuck terminating. This is especially relevant for Job-based workloads that specify podReplacementPolicy: Failed, preventing the Job from making progress if such scenario occurrs.

Without the failure recovery controller, a node failure (e.g. network partition or a malfunction of the kubelet) results in the pods running on that node becoming stuck. The failure recovery mechanism unblocks workloads relying on pods being in a terminated state by forcefully moving them to the Failed phase after a grace period of 60 seconds since they became stuck.

Enable KueueViz

Fri, 18 Jul 2025 00:00:00 +0000

Note

For a streamlined installation and simplified upgrades, we recommend deploying Kueue using Helm.
You can customize the deployment with a local values.yaml file to fit your environment.
See the Helm chart installation guide for full instructions.

KueueViz is a web-based visualization tool that provides real-time monitoring of Kueue workloads, queues, and resource allocation. It offers an intuitive dashboard for observing job queue status, resource utilization, and workload progression.

This page shows how to install and configure KueueViz in your cluster.