tests(engine): "no API keys" cloud tests clear 2 of the 8 key env vars CloudEngine reads, so the suite is red on any machine with real keys
Problem
The "no API keys" tests in tests/engine/test_cloud.py and tests/engine/test_cloud_extended.py clear only 2 of the 8 API-key environment variables that CloudEngine actually reads, so they assert "no keys are configured" while up to six keys are still visible.
CloudEngine reads:
ANTHROPIC_API_KEY DEEPSEEK_API_KEY GEMINI_API_KEY GOOGLE_API_KEY
MINIMAX_API_KEY OPENAI_API_KEY OPENAI_CODEX_API_KEY OPENROUTER_API_KEYThe tests clear:
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)So on any machine with, say, OPENROUTER_API_KEY exported, a client constructs, health() returns True, and the assertion fails.
Affected
tests/engine/test_cloud.py::TestCloudEngineHealth::test_health_no_keys
tests/engine/test_cloud.py::TestCloudEngineListModels::test_list_models_no_keys
tests/engine/test_cloud_extended.py::TestCloudModelDiscovery::test_no_api_key_empty_list
tests/engine/test_cloud_extended.py::TestCloudModelDiscovery::test_only_google_client
tests/engine/test_cloud_extended.py::TestCloudModelDiscovery::test_health_no_clientsReproduce
$ export OPENROUTER_API_KEY=sk-or-...
$ uv run pytest tests/engine/test_cloud.py::TestCloudEngineHealth::test_health_no_keys
FAILED
$ OPENROUTER_API_KEY= uv run pytest tests/engine/test_cloud.py::TestCloudEngineHealth::test_health_no_keys
1 passedThat single variable accounts for the failure on my machine; a developer with a Gemini or DeepSeek key set would hit the same thing through a different variable.
Why it is worth fixing
These are exactly the tests most likely to be red for a contributor who actually uses the product — anyone with real cloud keys exported starts from a failing suite and has to work out which failures are theirs. I hit this while checking whether my own local changes had broken anything, and had to stash every change and re-run against a clean tree to establish that the failures were pre-existing.
Suggested fix
Clear the full set rather than an ad-hoc pair. A small autouse fixture keeps it in one place and stops the list drifting again as engines are added:
CLOUD_KEY_ENV_VARS = (
"ANTHROPIC_API_KEY", "DEEPSEEK_API_KEY", "GEMINI_API_KEY", "GOOGLE_API_KEY",
"MINIMAX_API_KEY", "OPENAI_API_KEY", "OPENAI_CODEX_API_KEY", "OPENROUTER_API_KEY",
)
@pytest.fixture
def no_cloud_keys(monkeypatch):
for var in CLOUD_KEY_ENV_VARS:
monkeypatch.delenv(var, raising=False)Tests that want one key present then set it explicitly after requesting the fixture, which also makes their intent clearer than the current mix of partial delenv calls.
Happy to open a PR.
Note on a sibling case, already handled
tests/speech/test_tts_backends.py::test_kokoro_health_false_without_package had the same shape and origin/main already guards it by skipping when the optional kokoro package is installed. That is the pattern being asked for here, applied to the cloud key vars.
Environment
- Observed at c1af3c94 (2026-09-03); confirmed still present on
origin/main - Python 3.12.10, Windows 11
Source: open-jarvis/OpenJarvis