No CI runs the test suite — `release.yml` publishes to PyPI with no test gate

Author: rajarshidattapyCreated Aug 29, 2026Updated Aug 29, 2026

Labels: ci, infrastructure

Description

.github/workflows/ contains exactly one workflow:

.github/workflows/release.yml

It triggers on release: [published], builds distributions, and uploads them to PyPI. There is no test job, no lint job, and nothing that runs on push or pull_request.

The repo has a real test suite — 168 unit tests across tests/unit/{test_admin,test_daemon,test_helpers,test_ipc,test_macos,test_run,test_skill}.py plus tests/integration/ — and none of it runs automatically, ever.

Impact

  • Regressions land on main unnoticed. tests/unit/test_skill.py is currently failing on Windows checkouts (see the SKILL.md symlink issue) and nothing surfaced it.
  • release.yml will happily build and publish a broken wheel: the publish job's only steps are python -m build and pypa/gh-action-pypi-publish. A green release is not evidence that anything works.
  • CONTRIBUTING.md never tells a contributor how to run the tests, so there is no manual fallback either. It documents ./browser-harness and domain skills only.

Suggested fix

Add a test.yml that runs on pull_request and push to main:

yaml
strategy:
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]
    python-version: ["3.11", "3.12"]

running pytest tests/unit. The Windows leg matters most — the codebase carries a lot of platform-specific code (_ipc.IS_WINDOWS TCP loopback, spawn_kwargs(), _process_start_time() ctypes path, tasklist probes) that no one is currently exercising in CI.

Then make release.yml depend on that job, so a red suite blocks the PyPI upload.

Also worth adding to CONTRIBUTING.md: a one-line "run pytest tests/unit before opening a PR".

Source: browser-use/browser-harness