[Bug]: VecEnv sub-environment seeds (seed + i) overlap across runs with adjacent base seeds

Author: abaiseroCreated Jul 17, 2026Updated Jul 24, 2026
Labelsbug

Bug

VecEnv.seed(seed) (and make_vec_env(..., seed=seed)) seeds sub-environment i with seed + i, which causes issues with the standard workflow of sweeping seeds 0..N to get independent runs for evaluation.

As a result of the sub-environment seeding, two runs with adjacent base seeds share most of their sub-environment seeds: with n_envs=4, a run with seed=0 uses env seeds {0,1,2,3} and a run with seed=1 uses {1,2,3,4}, i.e., 3 of 4 identical. Since env seeding is deterministic, the shared sub-envs produce identical RNG streams across the two runs. The runs may still end up partially differing due to other sources of randomness (e.g., due to input actions that may differ across sub-environments despite their identical seed); however the problem remains valid, and the issue in particular remains the same if the same root seed is also used indirectly to determine the actions.

A simple fix is to derive sub-env seeds via np.random.SeedSequence(seed).spawn(n_envs) (or generate_state(n_envs)) instead of seed + i, so any two distinct base seeds yield disjoint, independent env streams.

To Reproduce

python
import numpy as np

from stable_baselines3.common.env_util import make_vec_env

venv_a = make_vec_env("Pendulum-v1", n_envs=4, seed=0)
venv_b = make_vec_env("Pendulum-v1", n_envs=4, seed=1)

obs_a = venv_a.reset()
obs_b = venv_b.reset()

print("run A (seed=0) sub-env seeds:", venv_a.seed(0))  # [0, 1, 2, 3]
print("run B (seed=1) sub-env seeds:", venv_b.seed(1))  # [1, 2, 3, 4]
print("A[1:] == B[:3]:", np.allclose(obs_a[1:], obs_b[:3]))  # True

Relevant log output / Error message

bash
run A (seed=0) sub-env seeds: [0, 1, 2, 3]
run B (seed=1) sub-env seeds: [1, 2, 3, 4]
A[1:] == B[:3]: True

System Info

  • OS: Linux-7.0.0-27-generic-x86_64-with-glibc2.43 # 27-Ubuntu SMP PREEMPT_DYNAMIC Thu Jun 18 19:13:49 UTC 2026
  • Python: 3.14.4
  • Stable-Baselines3: 2.8.0a4
  • PyTorch: 2.13.0+cu130
  • GPU Enabled: True
  • Numpy: 2.5.1
  • Cloudpickle: 3.1.2
  • Gymnasium: 1.2.3

Checklist

  • My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
  • I have checked that there is no similar issue in the repo
  • I have read the documentation
  • I have provided a minimal and working example to reproduce the bug
  • I've used the markdown code blocks for both code and stack traces.

Source: DLR-RM/stable-baselines3