[KernelGYM & Dr. Kernel] 一个分布式 GPU 环境和一系列 RL 训练方法,用于支持 Kernel 世代的 RL [ICML 2026]
[KernelGYM & Dr. Kernel] 一个分布式 GPU 环境和一系列 RL 训练方法,用于支持 Kernel 世代的 RL [ICML 2026]
KernelGYM is a GPU-distributed environment designed for evaluating and training AI models on GPU kernel generation tasks. This repository also includes our validated RL training methods from our paper:
Dr.Kernel: Reinforcement Learning Done Right for Triton Kernel Generations
See drkernel/ for training implementation details and recipe.
Training AI models to generate optimized GPU kernels presents unique challenges that standard code generation environments cannot address. KernelGYM provides:
GPU Resource Management: Unlike CPU-bound code execution, kernel evaluation requires careful GPU memory management and device isolation. A single CUDA error can crash an entire evaluation pipeline.
Performance Measurement Complexity: Accurate kernel timing requires proper warmup, multiple trials, and CUDA event synchronization - not just wall-clock timing.
Correctness Verification: Kernels must produce numerically correct results within floating-point tolerances, requiring reference implementations for comparison.
Scalability: Training requires evaluating thousands of kernel samples per batch, necessitating distributed evaluation across multiple GPUs and nodes.
Subprocess Worker Pool: Each GPU worker maintains a pool of subprocess workers. CUDA errors are isolated to subprocesses and automatically recovered, preventing cascading failures.
Precise Timing Infrastructure: Built-in CUDA event-based timing with configurable warmup and trial counts ensures accurate performance measurements.
Flexible Correctness Checking: Support for custom tolerance levels (rtol/atol), multiple test cases, and decoy kernel detection.
Horizontal Scalability: Redis-based task queue enables seamless scaling from single-GPU to multi-node deployments.
High-level architecture of KernelGYM: API server + task manager + distributed GPU workers with subprocess isolation.
…| Component | Description |
|---|---|
| Backend | Handles kernel compilation, loading, and execution. Abstracts different kernel frameworks (CUDA, Triton). |
| Toolkit | Implements evaluation logic - correctness checking, performance measurement, profiling. |
| Workflow Controller | Orchestrates multi-step evaluation workflows (e.g., reference timing + kernel evaluation). |
| Task Manager | Redis-based queue management with priority scheduling and worker assignment. |
| GPU Worker | Manages task execution on a specific GPU with subprocess isolation for error recovery. |
# Clone the repository
git clone https://github.com/hkust-nlp/KernelGYM.git
cd kernelgym
# Run the setup script
bash setup.shThe setup script will:
requirements.txtYou have two options:
.env on first startup..env yourself with explicit values.Manual .env example:
# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD= # Optional
# API Server Configuration
API_HOST=0.0.0.0
API_PORT=10907
# GPU Configuration
GPU_DEVICES=[0,1,2,3,4,5,6,7] # List of GPU indices to use
# Optional: Node identification for multi-node setup
NODE_ID=node-1If your goal is to get a local KernelGYM service up quickly:
# 1) Install dependencies
bash setup.sh
# 2) Auto-configure .env (or skip; start script can do this automatically)
bash scripts/auto_configure.sh
# 3) Start API + workers
./start_all_with_monitor.sh
# 4) Verify service
curl http://localhost:10907/health
curl http://localhost:10907/workers/statusscripts/auto_configure.sh includes a practical example strategy for fast setup:
Treat this as a reference implementation, not a fixed policy. If your machine or cluster has different networking constraints, edit the port-selection logic in scripts/auto_configure.sh (for example, custom port ranges, reserved ports, or service-specific pinning) to match your environment.
For single-node deployments with all components on one machine:
# Start the API server and workers
./start_all_with_monitor.shThis script starts:
Notes:
.env does not exist, start_all_with_monitor.sh triggers scripts/auto_configure.sh.localhost/127.0.0.1) and not running, the startup script launches Redis automatically.For distributed setups across multiple machines:
On the main node (API server + Redis):
# Ensure Redis is accessible from worker nodes
redis-server --bind 0.0.0.0
# Start API server
python -m kernelgym.server.api.serverOn worker nodes:
.env file pointing to the main node:API_HOST=
API_PORT=10907
REDIS_HOST=
REDIS_PORT=6379
NODE_ID=worker-node-1 # Unique identifier for this node
GPU_DEVICES=[0,1,2,3] # GPUs available on this node./start_worker_multinode.shThe worker will:
For more details on deployment configuration and options, refer to these scripts:
| Script | Description |
|---|---|
| scripts/auto_configure.sh | Auto-generates .env configuration with detected IP, available ports, and GPU devices |
| start_all_with_monitor.sh | Single-node deployment: starts Redis, API server, worker monitor, and GPU workers |
| start_worker_multinode.sh | Multi-node worker startup: connects worker nodes to remote API/Redis server |
Useful auto_configure.sh options:
bash scripts/auto_configure.sh --force to overwrite existing .envbash scripts/auto_configure.sh --use-indexed-ports to use PORT0, PORT1, ... as candidatesThe kernel_simple workflow evaluates a standalone kernel with provided test cases:
…The kernelbench workflow compares a generated kernel against a reference implementation:
…| Backend | Description | Use Case |
|---|---|---|
| kernelbench | Full KernelBench evaluation backend | Reference-based kernel evaluation with CUDA/Triton support |
The KernelBench backend supports two sub-backends:
KernelBench evaluation also supports two execution modes:
torch.compile() for optimized execution and fair comparison with compiled referencesTo select execution mode, use the reference_backend parameter:
# Eager mode (default) - compare against eager PyTorch reference
payload = {
"backend": "triton",
"reference_backend": "pytorch", # or omit for default
# ...
}
# Compile mode - compare against torch.compile() optimized reference
payload = {
"backend": "triton",
"reference_backend": "compile",
# ...
}| Toolkit | Description | Workflow |
|---|---|---|
| kernelbench | Full evaluation with reference comparison | kernelbench |
| kernel_simple | Standalone kernel evaluation with test cases | kernel_simple |
# For Triton kernels
payload = {
"backend": "triton",
# ...
}
# For CUDA kernels
payload = {
"backend": "cuda",
# ...
}KernelGYM is designed to be extensible. You can add new GPU tasks by implementing the core abstractions.
To add support for a new type of GPU task, implement these components:
…Register your backend:
from kernelgym.backend import register_backend
register_backend("my_backend", MyBackend)…Register your toolkit:
from kernelgym.toolkit import register_toolkit
register_toolkit("my_toolkit", MyToolkit)…The kernel_simple module demonstrates how to extend KernelGYM for custom evaluation logic. Here's how it's structured:
1. Schema Definition (kernelgym/schema/simple_task.py):
@dataclass
class KernelSimpleTask:
task_id: str
kernel_code: str
entry_point: str = "ModelNew"
device: str = "cuda:0"
backend: str = "triton"
cases: Optional[List[Dict]] = None # Test cases暂无开放 Issues,或尚未同步最近议题。