signals: follow-ups deferred from PR #903
Title
signals: follow-ups deferred from PR #903
Summary
PR #903 landed the Rust port of the Signals paper taxonomy into brightstaff
with 100% parity against the Python reference on 2,000 lmsys-chat-1m samples.
Three items came up during review that were intentionally scoped out of that
PR. This issue tracks them so they don't drop on the floor.
Follow-ups
1. Documentation for the new OTel attribute set — P1
The layered attribute keys (signals.interaction.misalignment.count,
signals.execution.failure.severity, etc.) and per-instance span events
(signal.<type> with confidence/snippet/metadata) added in #903 are not yet
documented. Legacy keys (signals.frustration.*, signals.escalation.requested,
etc.) are being kept in dual-emit for backward compatibility and should be
flagged as deprecated in the docs.
Acceptance:
-
docs/source/guides/observability/has a signals attributes reference listing all layered keys with descriptions and example values - Legacy keys section marks each as deprecated with a migration pointer
-
docs/source/observability/monitoring.rstlead example updated to show layered keys
2. Upstream Python reference: grouped summary format — P3, optional
In #903 the Rust generate_summary was refactored to produce a
taxonomy-grouped output. The Python reference at katanemo/signals
still emits the flat format. The parity comparator classifies summary
as Tier-C so this is not a correctness issue, but it's the one known
intentional divergence and worth aligning upstream.
Acceptance:
- Open a PR against
katanemo/signalsporting the grouped-summary format (same layer headers, same severity notation, same alert tail) - Re-run the parity harness and confirm Tier-C divergence drops to zero
3. Legacy OTel key sunset — P3
Decided offline during #903 review to keep signals.frustration.*,
signals.escalation.requested, signals.follow_up.repair.*,
signals.positive_feedback.count, signals.repetition.count for one
release for backward compatibility. This issue tracks the eventual removal.
Acceptance:
- Identify all internal consumers of legacy keys (dashboards, alerts, samplers, any downstream ingest)
- Migrate each consumer to the layered keys
- Remove
emit_legacy_attributesfromcrates/brightstaff/src/signals/otel.rs - Remove legacy constants from
crates/brightstaff/src/tracing/constants.rs - Bump version per the release process in
CLAUDE.md
Context
- PR: #903 (merged, commit SHAs
c7f31879..cee89950) - Python reference:
katanemo/signals@0.1.0 - Parity harness:
tests/parity/signals/+.github/workflows/parity-signals.yml - Last parity run: 100% Tier-A agreement, 0 Tier-B divergences, 2,000 samples, seed 42
Source: katanemo/plano