MCP: declared property values can't be read back, so verifying state means diffing pixels
Split out from #13244, which reported four verification gaps together.
Problem
An MCP client can drive a UI but can't read back what a declared property holds.
get_element_properties returns absolutePosition, size, accessibleLabel, accessibleRole, computedOpacity and typeNamesAndIds.
set_element_value writes the accessible value only.
ElementHandle's accessors in internal/backends/testing/search_api.rs are accessibility plus structure, so the MCP layer isn't declining to expose something it already has.
Verifying that an interaction produced the intended state therefore means screenshotting before and after and diffing pixels. That answers "these pixels changed", not "this property holds this value", and it says nothing about state with no distinct visual. In the case that prompted #13244 each verification step moved ~350 KB of base64 PNG, and the diff only became reliable after zooming the UI so the change cleared a noise threshold.
Why this isn't an MCP-layer fix
Name-based property access exists only in the interpreter: ComponentInstance::get_property (internal/interpreter/component.rs:236, internal/interpreter/api.rs:1512).
For a compiled Rust or C++ application — what the embedded MCP server usually runs in — ItemTreeVTable exposes accessible_string_property and no generic property access.
There is no table mapping a property name to its storage in generated code, so there is nothing for the introspection layer to read.
Two consequences for the design:
- Supporting compiled applications means generating a property table, in the spirit of what
SLINT_EMIT_DEBUG_INFOalready emits for type names and ids. That is a codegen decision with a binary-size cost, and it needs an opt-in story. - Even under the interpreter,
get_propertyreaches the root component's exposed properties. The case in #13244 was aselectedflag on a repeated row, which is a property of a sub-component instance, so it isn't reachable by that path either.
Possible directions
- Emit a name-to-property table under the existing debug-info flag, and expose typed reads through
ElementHandleand apropertiesfield onget_element_properties. - Restrict the feature to interpreter-hosted applications, such as
slint-viewer --features mcp, and say so plainly in the tool description. - Decline it, and document the accessible-value surface plus
start_event_recordingas the supported verification path.
Value encoding is a real question in any of these, though set_element_value and accessible_value already establish a convention to extend.
Related
- #13244 — the original report of this and three other gaps.
Source: slint-ui/slint