Developer Tools on Kueue

Enabling pprof endpoints

Fri, 21 Jul 2023 00:00:00 +0000

This page shows you how to enable pprof endpoints for Kueue controller manager.

The intended audience for this page are batch administrators.

Before you begin

Make sure the following conditions are met:

A Kubernetes cluster is running.
The kubectl command-line tool has communication with your cluster.
Kueue is installed.

Enabling pprof endpoints

Feature state stable since Kueue v0.5

To enable pprof endpoints, you need to set a pprofBindAddress is set in the manager’s configuration.

Setup Dev Monitoring

Wed, 04 Feb 2026 00:00:00 +0000

This page shows how to set up Prometheus for development, debugging, and testing Kueue metrics.

The page is intended for a platform developer.

Quick start with e2e infrastructure

The fastest way to get a Prometheus-enabled cluster is to use the e2e test infrastructure:

E2E_MODE=dev GINKGO_ARGS="--label-filter=feature:prometheus" make kind-image-build test-e2e

This provisions a Kind cluster with the Prometheus Operator, a Prometheus instance, Kueue, and all ServiceMonitors pre-configured.

Setup MultiKueue Development Environment

Tue, 13 Jan 2026 00:00:00 +0000

This tutorial explains how you can configure a manager cluster and worker clusters to run jobs with Topology-Aware Scheduling (TAS) in a MultiKueue environment. We also outline the automated steps using Kind for local testing.

Check the concepts section for a MultiKueue overview and Topology-Aware Scheduling overview.

Setup MultiKueue with E2E Test Cluster

The e2e test development mode can be used to maintain a MultiKueue cluster setup and run end-to-end tests against it without recreating and tearing it down each time.

External Frameworks

Fri, 17 Jan 2025 00:00:00 +0000

See external frameworks for examples of using existing integrations to integrate external frameworks.

Integrate a custom Job with Kueue

Tue, 25 Jul 2023 00:00:00 +0000

Kueue has built-in integrations for several Job types, including Kubernetes batch Job, MPIJob, RayJob and JobSet.

There are three options for using Kueue to manage Job-like CRDs that lack built-in integrations.

Leverage the built-in AppWrapper integration by wrapping instances of the custom Job in an AppWrapper. See Running a Wrapped Custom Workload for details.
Build a new integration as part of the Kueue repository.
Build a new integration as an external controller.

This guide is for platform developers and describes how to build a new integration. Integrations should be built using the APIs provided by Kueue’s jobframework package. This will both simplify development and ensure that your controller will be properly structured to become a core built-in integration if your Job type is widely used by the community.

Develop an AdmissionCheck Controller

Tue, 03 Sep 2024 00:00:00 +0000

An Admission Check Controller, referred to as ACC in the rest of the doc, is a component that manages the AdmissionChecks associated with it (given by the corresponding value of spec.controllerName) and the workloads queued against ClusterQueues configured to use those AdmissionChecks.

Read Admission Check to learn more about the mechanism from a user perspective.

Subcomponents

The ACC can be built-in into Kueue or run in a different kubernetes controller manager and should implement reconcilers for AdmissionChecks and Workloads.