[Question] why self.env.reset doesn't return an obs variable in FireResetEnv wrapper reset function, while it does in EpisodicLifeEnv and NoopResetEnv wrapper reset function

Author: RongpingZhouCreated May 15, 2026Updated May 15, 2026

Labelsquestion

❓ Question

In FireResetEnv wrapper in the file of stable-baselines3/stable_baselines3/common/atari_wrappers.py, I believe it is an issue that there is no obs return from self.env.reset(**kwargs) The code is as in the below:

def reset(self, **kwargs) -> AtariResetReturn:
        self.env.reset(**kwargs)
        obs, _, terminated, truncated, _ = self.env.step(1)
        if terminated or truncated:
            self.env.reset(**kwargs)
        obs, _, terminated, truncated, _ = self.env.step(2)
        if terminated or truncated:
            self.env.reset(**kwargs)
        return obs, {}

I believe it should be modified to

def reset(self, **kwargs) -> AtariResetReturn:
        self.env.reset(**kwargs)
        obs, _, terminated, truncated, _ = self.env.step(1)
        if terminated or truncated:
            self.env.reset(**kwargs)
        obs, _, terminated, truncated, _ = self.env.step(2)
        if terminated or truncated:
            obs, _ = self.env.reset(**kwargs)
        return obs, {}

I have encountered an issue with GravitarNoFrameskip-v4 environment, when I run the command

python train.py --algo dqn --env GravitarNoFrameskip-v4 --seed 37 --eval-freq -1 --n-timesteps 10000000 --tensorboard-log ./runs/

on my computer, when it was running to the step 23374, when FireResetEnv reset execute

obs, _, terminated, truncated, _ = self.env.step(2),

the environment terminated, and executed

        if terminated or truncated:
            self.env.reset(**kwargs)

the code will execute EpisodicLifeEnv reset() and NoopResetEnv reset(), however, the retruned obs variable from FireResetEnv reset is still obs before reset process,

obs, _, terminated, truncated, _ = self.env.step(2)

while I think it should be the obs after reset process.

        if terminated or truncated:
            obs, _ = self.env.reset(**kwargs)

Here is the datalog after I add some print lines:

EpisodicLifeEnv step(): lives: 4 -> 3, terminated: True, truncated: False
common/vec_env/dummy_vec_env.py: DummyVecEnv step_wait(): before env reset for env 0
FireResetEnv reset() called with kwargs: {'seed': None, 'options': None}
EpisodicLifeEnv reset() called with kwargs: {'seed': None, 'options': None}
EpisodicLifeEnv reset() end: lives: 3
EpisodicLifeEnv reset() end: obs shape: (210, 160, 3), obs.sum(): 3139748.00, obs.mean(): 31.148294
FireResetEnv step() called with action 1 after, terminated: False, truncated: False
FireResetEnv step() called with action 2 before, terminated: False, truncated: False
********** Monitor wrapper terminated: True, episode_num: 37, episode reward: 250.0, episode length: 2358, total steps: 96075 **********
EpisodicLifeEnv step(): lives: 3 -> 0, terminated: True, truncated: False
FireResetEnv step() called with action 2 after, terminated: True, truncated: False
EpisodicLifeEnv reset() called with kwargs: {'seed': None, 'options': None}
EpisodicLifeEnv performing full reset
NoopResetEnv reset() called with kwargs: {'seed': None, 'options': None}
********** Monitor reset() called
********** Monitor reset() with kwargs: {'seed': None, 'options': None} end
NoopResetEnv reset() end: performed 21 no-ops， obs shape: (210, 160, 3), obs.sum(): 211087
EpisodicLifeEnv reset() end: lives: 6
EpisodicLifeEnv reset() end: obs shape: (210, 160, 3), obs.sum(): 211087.00, obs.mean(): 2.094117
FireResetEnv reset() end
FireResetEnv reset() end: obs shape: (210, 160, 3), obs.sum(): 3122004.00, obs.mean(): 30.972262

Checklist

I have checked that there is no similar issue in the repo
I have read the documentation
If code there is, it is minimal and working
If code there is, it is formatted using the markdown code blocks for both code and stack traces.

Source: DLR-RM/stable-baselines3

View original on GitHub View discussion on GitHub