An interface library for RL post training with environments.
An interface library for RL post training with environments.
An e2e framework for creating, deploying and using isolated execution environments for agentic RL training, built using Gymnasium style simple APIs.
Featured Example: Train LLMs to play BlackJack using torchforge (PyTorch's agentic RL framework): examples/grpo_blackjack/
Zero to Hero Tutorial: End to end tutorial from our GPU Mode lecture and other hackathons.
Install the OpenEnv package:
pip install openenvInstall an environment client (e.g., Echo):
pip install git+https://huggingface.co/spaces/openenv/echo_envThen use the environment:
…Synchronous usage is also supported via the .sync() wrapper:
from echo_env import CallToolAction, EchoEnv
# Use .sync() for synchronous context manager
with EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as client:
result = client.reset()
result = client.step(
CallToolAction(
tool_name="echo_message",
arguments={"message": "Hello, World!"},
)
)
print(result.observation.result)For a detailed quick start, check out the docs page.
OpenEnv provides a standard for interacting with agentic execution environments via simple Gymnasium style APIs - step(), reset(), state(). Users of agentic execution environments can interact with the environment during RL training loops using these simple APIs.
In addition to making it easier for researchers and RL framework writers, we also provide tools for environment creators making it easier for them to create richer environments and make them available over familiar protocols like HTTP and packaged using canonical technologies like docker. Environment creators can use the OpenEnv framework to create environments that are isolated, secure, and easy to deploy and use.
The OpenEnv CLI (openenv) provides commands to initialize new environments and deploy them to Hugging Face Spaces.
⚠️ Early Development Warning OpenEnv is currently in an experimental stage. You should expect bugs, incomplete features, and APIs that may change in future versions. The project welcomes bugfixes, but significant changes should be discussed before implementation so the technical committee and community can coordinate scope, compatibility, and release timing. It's recommended that you signal your intention to contribute in the issue tracker, either by filing a new issue or by claiming an existing one.
Below is a list of active and historical RFCs for OpenEnv. RFCs are proposals for major changes or features. Please review and contribute!
…OpenEnv includes a built-in web interface for interactive environment exploration and debugging. The web interface provides:
The web interface is conditionally enabled based on environment variables:
ENABLE_WEB_INTERFACE=trueTo use the web interface:
from openenv.core.env_server import create_web_interface_app
from your_env.models import YourAction, YourObservation
from your_env.server.your_environment import YourEnvironment
env = YourEnvironment()
app = create_web_interface_app(env, YourAction, YourObservation)When enabled, open http://localhost:8000/web in your browser to interact with the environment.
Base class for implementing environment logic:
reset(): Initialize a new episode, returns initial Observationstep(action): Execute an Action, returns resulting Observationstate(): Access episode metadata (State with episode_id, step_count, etc.)Base class for environment communication:
async with and await for all operations.sync() to get a SyncEnvClient for synchronous usageManage container deployment:
LocalDockerProvider: Run containers on local Docker daemonDockerSwarmProvider: Deploy to Docker Swarm clustersUVProvider, DaytonaProvider, ACASandboxProvider: Additional runtime providersKubernetesProvider: Deploy to Kubernetes clusters (planned)Type-safe data structures:
Action: Base class for environment actionsObservation: Base class for environment observationsState: Episode state trackingStepResult: Combines observation, reward, done flagUse the CLI to quickly scaffold a new environment:
openenv init my_envThis creates the following structure:
…OpenEnv uses pyproject.toml as the primary dependency specification:
pyproject.toml: Each environment defines its own dependenciespyproject.toml: Contains shared core dependencies (fastapi, pydantic, uvicorn)requirements.txt: Can be auto-generated from pyproject.toml for Docker buildsDevelopment Workflow:
# Install environment in editable mode
cd my_env
pip install -e .
# Or using uv (faster)
uv pip install -e .
# Run server locally without Docker
uv run server --host 0.0.0.0 --port 8000See envs/README.md for a complete guide on building environments.
To use an environment:
pip install git+https://huggingface.co/spaces/openenv/echo_envfrom echo_env import CallToolAction, EchoEnvAsync (recommended):
async with EchoEnv(base_url="...") as client:
result = await client.reset()
result = await client.step(action)Sync (via .sync() wrapper):
with EchoEnv(base_url="...").sync() as client:
result = client.reset()
result = client.step(action)See example scripts in examples/ directory.
The OpenEnv CLI provides commands to manage environments:
openenv init <env_name> - Initialize a new environment from templateopenenv import <source> --name <env_name> --output-dir <dir> - Wrap a supported third-party source environment, including ORS/OpenReward and Verifiers, as OpenEnvopenenv push [--repo-id <repo>] [--private] - Deploy environment to Hugging Face Spacesopenenv serve - Serve an environment locally with optional auto-reloadopenenv build - Build the Docker image for an environmentopenenv fork <space-id> - Fork a Space from HF Hub to your accountopenenv validate - Validate an environment configuration# Create a new environment
openenv init my_game_env
# Or import an ORS/OpenReward or Verifiers source environment
openenv import path/to/source --name my_game_env --output-dir .
# Deploy to Hugging Face (will prompt for login if needed)
cd my_game_env
openenv pushFor detailed options run any command with --help.
# Clone the repository
git clone https://github.com/huggingface/OpenEnv.git
cd OpenEnv
# Install core package in editable mode
pip install -e .
# Or using uv (faster)
uv pip install -e .OpenEnv uses a modular dependency structure: the core package is minimal, and each environment has its own dependencies. This means some tests require environment-specific packages.
# Install pytest (required for running tests)
uv pip install pytest
# Run all tests (skips tests requiring uninstalled dependencies)
PYTHONPATH=src:envs uv run pytest tests/ -v --tb=short
# Run a specific test file
PYTHONPATH=src:envs uv run pytest tests/envs/test_echo_environment.py -vTo run environment-specific tests, install that environment's dependencies:
# Example: Install coding_env with dev dependencies (includes smolagents + pytest)
uv pip install -e "envs/coding_env[dev]"
# Then run coding_env tests
PYTHONPATH=src:envs uv run pytest tests/envs/test_python_codeact_rewards.py -vTests will be automatically skipped if their required dependencies aren't installed.
OpenEnv works with a growing ecosystem of RL frameworks and platforms. If your project supports OpenEnv, open a PR to add it here.
See the TRL example on how to integrate OpenEnv environments with GRPO training.
See GRPO BlackJack training example: examples/grpo_blackjack/
See the 2048 game example based on gpt-oss: Colab notebook
See the SkyRL example on how to train on OpenEnv environments with SkyRL.
See the ART example on how OpenEnv environments can be used to train models with ART.
See the Oumi example on how OpenEnv environments can be used to train models with Oumi.
| Environment | Description |
|---|---|
| Echo Environment | Echoes back messages with metadata. Ideal for testing HTTP server infrastructure, learning framework basics, and verifying container deployment. |
| Coding Environment | Sandboxed Python code execution via smolagents. Captures stdout/stderr/exit codes, supports persistent episode context, and provides detailed error handling. |
| Chess Environment | Chess RL environment with configurable opponents and full rules support. |
| Atari Environment | Cl |
No open issues yet, or sync has not completed.