#1364·Gymnasium

Allow custom rewards in FrozenLakeEnv

Author: IsaacWeber1Created Apr 26, 2025Updated Jun 26, 2026
Labelsenhancement

Proposal

Currently, the environment hardcodes the rewards as:

  • Goal (G) → +1
  • Hole (H) → 0
  • Frozen tile (F) → 0

Motivation

Currently, it is difficult to experiment with alternative reinforcement schedules in FrozenLake without modifying the environment code itself.

Pitch

Add three optional arguments to the FrozenLakeEnv constructor:

  • reward_goal (default 1.0)
  • reward_hole (default 0.0)
  • reward_step (default 0.0)

The default values are the existing values to avoid changing existing projects.

Alternatives

One alternative would be to subclass FrozenLakeEnv externally and override the step method. However, this approach is less clean, harder to maintain, and forces users to reimplement internal behavior that could easily be parameterized.

Additional context

This proposal is related to an extra credit assignment offered in a university Artificial Intelligence course.

The goal was to modify the FrozenLake environment to support customizable reinforcement structures without changing the external interface.

Checklist

  • I have checked that there is no similar issue in the repo

Source: Farama-Foundation/Gymnasium