diagram_gen reports AVAILABLE without mermaid-cli and silently renders mermaid source as a text card
Summary
diagram_gen reports status: AVAILABLE when mermaid-cli (mmdc) is not installed, and
diagram_type: "mermaid" then returns success: True while rendering the mermaid source
code as a text card. An agent that trusts the registry status and the success flag ships images
of code instead of diagrams, with nothing in the result signalling the substitution except a
data["method"] field it has no reason to check.
This bit during a real production run: the scene plan called for 17 progressive pipeline diagrams
as the visual spine of an explainer. All 17 would have rendered as pictures of graph LR ....
Root cause
tools/graphics/diagram_gen.py:
def get_status(self) -> ToolStatus:
if self._has_mermaid() or self._has_pillow(): # <-- OR
return ToolStatus.AVAILABLE
def _has_mermaid(self) -> bool:
return shutil.which("mmdc") is not NonePillow is a hard dependency (requirements.txt:6 — Pillow>=10.0), so _has_pillow() is always true and the tool is
always AVAILABLE. But Pillow only backs the boxes/flowchart paths — it cannot render mermaid.
_render_mermaid handles the missing binary by falling back:
else:
# Fallback: render mermaid text as a styled text card
return self._render_text_card(definition, inputs)_render_text_card returns ToolResult(success=True, ...). mermaid-cli is not in
requirements.txt, package.json, or make setup, so the default install hits this path.
Reproduction
On a machine without mmdc (the default after make setup):
from tools.tool_registry import registry
registry.discover()
print(registry._tools['diagram_gen'].get_info()['status']) # available
r = registry._tools['diagram_gen'].execute({
'diagram_type': 'mermaid',
'definition': 'graph LR\n A[Research] --> B[Proposal]',
'output_path': '/tmp/d.png',
})
print(r.success, r.data['method']) # True text_cardThe PNG contains the literal text graph LR A[Research] --> B[Proposal].
Why this matters beyond one tool
AGENT_GUIDE.md makes the registry the source of truth for capability and instructs agents not to
hide degraded paths. This is a case where the registry itself reports a capability the tool does not
have, and the degradation is invisible at the call site. Its shape ("declared available, silently
substitutes") is the failure mode the governance contract is designed to prevent, so it seems worth
fixing at the contract level rather than only in this tool.
Suggested fix
Any one of these would close the hole; the first two seem cheapest:
- Per-capability status.
get_status()returnsAVAILABLEonly for the diagram types that actually have a backend. If only Pillow is present,mermaidshould beDEGRADED/UNAVAILABLEwhileboxesstays available. - Fail loudly instead of substituting. Return
success: Falsewith the existinginstall_instructions(npm install -g @mermaid-js/mermaid-cli) whenmermaidis requested andmmdcis missing. Keep the text-card fallback behind an explicit opt-in such asallow_text_fallback: true. - At minimum, surface it in
runtime_warningsfromprovider_menu_summary()— the same channel already used forhyperframes: npm package not resolvable— so preflight shows it.
Secondary observation
The Pillow boxes renderer is also weaker than its availability implies: it grids at
cols = min(len(boxes), 4), so a 7- or 8-node flow becomes a 4+3 grid with connector lines cutting
diagonally across the boxes, and ImageFont.truetype("arial.ttf", 18) falls back to the default
bitmap font on macOS (no arial.ttf), which is unreadable at 1920x1080. Happy to open that
separately if you'd prefer it tracked on its own.
Environment
- macOS 15 (Darwin 25.6.0), Apple M1
- Python 3.12.14, Node 22.14.0, FFmpeg 9.0.1
- Repo at
08e2151,mmdcnot installed
Source: calesthio/OpenMontage