tests(engine): "no API keys" cloud tests clear 2 of the 8 key env vars CloudEngine reads, so the suite is red on any machine with real keys

Author: tim703223-glitchCreated Sep 7, 2026Updated Sep 7, 2026

Problem

The "no API keys" tests in tests/engine/test_cloud.py and tests/engine/test_cloud_extended.py clear only 2 of the 8 API-key environment variables that CloudEngine actually reads, so they assert "no keys are configured" while up to six keys are still visible.

CloudEngine reads:

ANTHROPIC_API_KEY   DEEPSEEK_API_KEY   GEMINI_API_KEY      GOOGLE_API_KEY
MINIMAX_API_KEY     OPENAI_API_KEY     OPENAI_CODEX_API_KEY OPENROUTER_API_KEY

The tests clear:

python
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)

So on any machine with, say, OPENROUTER_API_KEY exported, a client constructs, health() returns True, and the assertion fails.

Affected

tests/engine/test_cloud.py::TestCloudEngineHealth::test_health_no_keys
tests/engine/test_cloud.py::TestCloudEngineListModels::test_list_models_no_keys
tests/engine/test_cloud_extended.py::TestCloudModelDiscovery::test_no_api_key_empty_list
tests/engine/test_cloud_extended.py::TestCloudModelDiscovery::test_only_google_client
tests/engine/test_cloud_extended.py::TestCloudModelDiscovery::test_health_no_clients

Reproduce

$ export OPENROUTER_API_KEY=sk-or-...
$ uv run pytest tests/engine/test_cloud.py::TestCloudEngineHealth::test_health_no_keys
FAILED

$ OPENROUTER_API_KEY= uv run pytest tests/engine/test_cloud.py::TestCloudEngineHealth::test_health_no_keys
1 passed

That single variable accounts for the failure on my machine; a developer with a Gemini or DeepSeek key set would hit the same thing through a different variable.

Why it is worth fixing

These are exactly the tests most likely to be red for a contributor who actually uses the product — anyone with real cloud keys exported starts from a failing suite and has to work out which failures are theirs. I hit this while checking whether my own local changes had broken anything, and had to stash every change and re-run against a clean tree to establish that the failures were pre-existing.

Suggested fix

Clear the full set rather than an ad-hoc pair. A small autouse fixture keeps it in one place and stops the list drifting again as engines are added:

python
CLOUD_KEY_ENV_VARS = (
    "ANTHROPIC_API_KEY", "DEEPSEEK_API_KEY", "GEMINI_API_KEY", "GOOGLE_API_KEY",
    "MINIMAX_API_KEY", "OPENAI_API_KEY", "OPENAI_CODEX_API_KEY", "OPENROUTER_API_KEY",
)

@pytest.fixture
def no_cloud_keys(monkeypatch):
    for var in CLOUD_KEY_ENV_VARS:
        monkeypatch.delenv(var, raising=False)

Tests that want one key present then set it explicitly after requesting the fixture, which also makes their intent clearer than the current mix of partial delenv calls.

Happy to open a PR.

Note on a sibling case, already handled

tests/speech/test_tts_backends.py::test_kokoro_health_false_without_package had the same shape and origin/main already guards it by skipping when the optional kokoro package is installed. That is the pattern being asked for here, applied to the cloud key vars.

Environment

  • Observed at c1af3c94 (2026-09-03); confirmed still present on origin/main
  • Python 3.12.10, Windows 11