Leakage of NUMBER option out of docstring scope
Author: MatthewFlammCreated May 7, 2022Updated Sep 16, 2026
Labelsplugin: doctests
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
This was originally noted in #9892, but there is a more targeted MRE here.
MRE
Running pytest --doctest-modules test_x.py with x=1, 2 or 3 results in the following pattern of failing and passing tests. The test inside test_wo_number incorrectly passes only on the cases (test_1.py and test_2.py) where a failing test that uses NUMBER (test_w_off_by_2_number) also exists in the same module. This leakage does not occur when there are tests that pass that utilize NUMBER (test_3.py)
# test_1.py
def test_w_number():
"""
Correctly Passes
Examples
--------
>>> 1. # doctest: +NUMBER
0.
"""
def test_w_off_by_2_number():
"""
Correctly Fails
Examples
--------
>>> 0. # doctest: +NUMBER
2.
"""
def test_wo_number():
"""
Incorrectly Passes
Examples
--------
>>> 1.
0.
"""# test_2.py
def test_w_off_by_2_number():
"""
Correctly Fails
Examples
--------
>>> 0. # doctest: +NUMBER
2.
"""
def test_wo_number():
"""
Incorrectly Passes
Examples
--------
>>> 1.
0.
"""# test_3.py
def test_w_number():
"""
Correctly Passes
Examples
--------
>>> 1. # doctest: +NUMBER
0.
"""
def test_wo_number():
"""
Correctly Fails
Examples
--------
>>> 1.
0.
"""pip list
Package Version
---------- -------
attrs 21.4.0
iniconfig 1.1.1
numpy 1.22.3
packaging 21.3
pip 22.0.4
pluggy 1.0.0
py 1.11.0
pyparsing 3.0.8
pytest 7.1.2
setuptools 41.2.0
tomli 2.0.1system version
CentOS 7.9
Source: pytest-dev/pytest