#13251·slint

MCP: declared property values can't be read back, so verifying state means diffing pixels

Author: tilladamCreated Sep 5, 2026Updated Sep 14, 2026
Labelsneed triaginga:backend-testing

Split out from #13244, which reported four verification gaps together.

Problem

An MCP client can drive a UI but can't read back what a declared property holds. get_element_properties returns absolutePosition, size, accessibleLabel, accessibleRole, computedOpacity and typeNamesAndIds. set_element_value writes the accessible value only. ElementHandle's accessors in internal/backends/testing/search_api.rs are accessibility plus structure, so the MCP layer isn't declining to expose something it already has.

Verifying that an interaction produced the intended state therefore means screenshotting before and after and diffing pixels. That answers "these pixels changed", not "this property holds this value", and it says nothing about state with no distinct visual. In the case that prompted #13244 each verification step moved ~350 KB of base64 PNG, and the diff only became reliable after zooming the UI so the change cleared a noise threshold.

Why this isn't an MCP-layer fix

Name-based property access exists only in the interpreter: ComponentInstance::get_property (internal/interpreter/component.rs:236, internal/interpreter/api.rs:1512). For a compiled Rust or C++ application — what the embedded MCP server usually runs in — ItemTreeVTable exposes accessible_string_property and no generic property access. There is no table mapping a property name to its storage in generated code, so there is nothing for the introspection layer to read.

Two consequences for the design:

  1. Supporting compiled applications means generating a property table, in the spirit of what SLINT_EMIT_DEBUG_INFO already emits for type names and ids. That is a codegen decision with a binary-size cost, and it needs an opt-in story.
  2. Even under the interpreter, get_property reaches the root component's exposed properties. The case in #13244 was a selected flag on a repeated row, which is a property of a sub-component instance, so it isn't reachable by that path either.

Possible directions

  • Emit a name-to-property table under the existing debug-info flag, and expose typed reads through ElementHandle and a properties field on get_element_properties.
  • Restrict the feature to interpreter-hosted applications, such as slint-viewer --features mcp, and say so plainly in the tool description.
  • Decline it, and document the accessible-value surface plus start_event_recording as the supported verification path.

Value encoding is a real question in any of these, though set_element_value and accessible_value already establish a convention to extend.

Related

  • #13244 — the original report of this and three other gaps.