Proposal: Test Suite Modernization, Flakiness Elimination, and Concurrency/Coverage Testing

Summary

The test suite currently has several areas of flakiness and legacy patterns that cause unrelated CI failures across PRs:

1. False-positive CI failures on unrelated PRs: Because tests rely on hardcoded sleep() timings, loaded CI runners often fail on tests unrelated to a PR's actual changes, giving the false impression that a PR broke the build.
2. Flaky & timing-dependent tests due to hardcoded sleep() calls.
3. Shared state / lack of test isolation in legacy test modules.
4. Concurrency / race condition testing (especially relevant for free-threading 3.13t/3.14).
5. Coverage gaps across different platform observers.

---

1. Test Reliability & Flakiness

• CI false positives & hardcoded sleeps: Tests like tests/test_observers_polling.py rely on sequential sleep(0.4) delays. Under loaded CI runners (Linux, macOS, Windows), thread scheduling differences cause intermittent failures due to missed or extra intermediate events. This frequently leads to CI failing on completely unrelated PRs, making it harder to review and trust CI status.
• Race conditions in emitter tests: As tracked in #1128 (test_renaming_top_level_directory flaky on Inotify/Windows), recursive moves and directory tracking tests have timing sensitivities.
• Test Isolation: Legacy test files use module-level TEMP_DIR = mkdtemp(). Tests like test_delete_watched_dir delete this directory, breaking order-independence and individual test runs.

Action items:

• Migrate legacy test modules from module-level mkdtemp() to pytest's tmp_path fixture. (Good first issue candidate)
• Replace brittle sleep() calls with deterministic event queue waiting / assertions.
• Break large monolithic test functions into discrete unit tests per operation.
• Fix known flaky tests (e.g. #1128, test_observers_polling.py).

---

2. Concurrency & Multithreaded Testing

• Watchdog relies heavily on daemon emitter threads, event queues, and snapshot diffing.
• Connects with #1174 (Consider Blanket for multithreaded testing) to enable deterministic multithreaded testing and eliminate thread race conditions across all supported Python versions (including free-threading 3.13t / 3.14).

Action items:

• Explore deterministic multithreaded testing approaches (#1174).
• Ensure emitter teardown and thread joins are clean and don't leak threads in test runs.

---

3. Test Coverage & Edge Cases

• Expand coverage across platform-specific observers (fsevents, inotify, kqueue, winapi, polling) and core utilities (delayed_queue, dirsnapshot, watchmedo).
• Connects with #19 (Finish writing all the tests).

Action items:

• Identify uncovered branches in observer backends and utility modules.
• Add tests for edge cases (symlinks, rapid renames, permission errors, deep directory trees).

---

Related Existing Issues

• Flakiness & Timing:
  • #1128: test_renaming_top_level_directory is flaky
  • #1098: expect_event(DirMovedEvent(p("dir1", "dir2"), p("dir2"))) -> raise Empty
  • #1206: test_move_nested_subdirectories_on_windows fails on native Windows 11 (passes on CI)
  • #977: tests.test_observers_winapi.test___init__() fails locally on Python 3.11.2 on Windows 11
• Resource Management:
  • #1095: New test_select_fd uses a lot of open files
• Concurrency & Free-Threading:
  • #1174: Consider Blanket for multithreaded testing
  • #1132: test_tricks_from_file[tricks-from] crashes Python on 3.14t on Windows
• Coverage:
  • #19: Finish writing all the tests.
  • #974 : CI Test job support for pypy

---