Test suite maintainence and new ideas
Author: harshil21Created Jun 26, 2024Updated Aug 20, 2026
Labels⚙️ tests
Our test suite has once again degraded in performance and flakiness has been reintroduced. We can experiment / make our test suite better in different ways, some of my current ideas are listed roughly below:
- Try to make everything as concurrent as possible (#4271)
- Profile all tests and see what clogs up and fix that.
- solve the TODO in
test_official.exceptions - #4514
-
dirty-equalssounds like it could be quite useful for us! - Think about renaming the
Test*Baseclasses to just*Basesince pytest collects the classes starting withTest*. #4453 - Change CI's pytest
--disttoworksteal#4491 . - Extend that naming scheme to
telegram.exttests - fix the class name https://github.com/python-telegram-bot/python-telegram-bot/blob/51ef571a0761bbfea136c455bbfa67d47b20585c/tests/test_menubutton.py#L94 #4453
Follow https://github.com/zupo/awesome-pytest-speedup, not all items but some are worth looking at:
- Introduce
pytest-socketand completely disable network access for*WithoutRequesttests (#4317, #4454) -
Introduce(no major benefit found - https://github.com/python-telegram-bot/python-telegram-bot/pull/4271#issuecomment-2127968133)pyfakefsto test suite - Check if using
uvloopin tests provides any measurable speed up. - Isolate tests from other tests
- Check if introducing
pytest-randomly(need to check if pytest-xdist effectively does the job of randomly)
-> Check if using pytest-asyncio-cooperative is worth it
- Nope it is not.
Source: python-telegram-bot/python-telegram-bot