37 tools skip the description contract under a closed ticket (#5498)
Summary
37 tools skip test_description_contract under #5498, which is closed, so nothing tracks the gap and those tools are never checked.
Expected vs actual behavior
Expected: a quarantine in _DESCRIPTION_CONTRACT_ALLOWLIST points at an open ticket, and entries leave the list as tools are fixed.
Actual: #5498 closed with all 37 entries still in the list. Every one fails the same rule, use_cases is empty despite siblings in the same source, and is reported as a skip rather than a failure:
240 passed, 37 skippeduse_cases is part of what the agent reads when choosing between tools from the same source, so these are the tools it has the least to go on for. The list includes shell_run, slash_invoke, memory_recall and every work_task_* tool.
| Source | Tools |
|---|---|
| interactive_shell | 8 |
| knowledge | 6 |
| dagster, mariadb, mongodb, mongodb_atlas | 5 each |
| railway | 2 |
| slack | 1 |
None of the entries are stale; each still violates the rule.
Steps to reproduce
uv run python -m pytest tests/tools/test_description_contract.py -q -rsCan you reproduce it consistently?
Yes.
Additional context
Plan: add use_cases per source in small PRs and remove each tool from the allowlist as it is fixed, so its test runs again. The allowlist should end empty, or keep only entries with an open ticket.
Source: Tracer-Cloud/opensre