#909·plano

signals: follow-ups deferred from PR #903

Author: syedhashmiCreated Apr 23, 2026Updated Apr 23, 2026

Title

signals: follow-ups deferred from PR #903

Summary

PR #903 landed the Rust port of the Signals paper taxonomy into brightstaff with 100% parity against the Python reference on 2,000 lmsys-chat-1m samples. Three items came up during review that were intentionally scoped out of that PR. This issue tracks them so they don't drop on the floor.

Follow-ups

1. Documentation for the new OTel attribute set — P1

The layered attribute keys (signals.interaction.misalignment.count, signals.execution.failure.severity, etc.) and per-instance span events (signal.<type> with confidence/snippet/metadata) added in #903 are not yet documented. Legacy keys (signals.frustration.*, signals.escalation.requested, etc.) are being kept in dual-emit for backward compatibility and should be flagged as deprecated in the docs.

Acceptance:

  • docs/source/guides/observability/ has a signals attributes reference listing all layered keys with descriptions and example values
  • Legacy keys section marks each as deprecated with a migration pointer
  • docs/source/observability/monitoring.rst lead example updated to show layered keys

2. Upstream Python reference: grouped summary format — P3, optional

In #903 the Rust generate_summary was refactored to produce a taxonomy-grouped output. The Python reference at katanemo/signals still emits the flat format. The parity comparator classifies summary as Tier-C so this is not a correctness issue, but it's the one known intentional divergence and worth aligning upstream.

Acceptance:

  • Open a PR against katanemo/signals porting the grouped-summary format (same layer headers, same severity notation, same alert tail)
  • Re-run the parity harness and confirm Tier-C divergence drops to zero

3. Legacy OTel key sunset — P3

Decided offline during #903 review to keep signals.frustration.*, signals.escalation.requested, signals.follow_up.repair.*, signals.positive_feedback.count, signals.repetition.count for one release for backward compatibility. This issue tracks the eventual removal.

Acceptance:

  • Identify all internal consumers of legacy keys (dashboards, alerts, samplers, any downstream ingest)
  • Migrate each consumer to the layered keys
  • Remove emit_legacy_attributes from crates/brightstaff/src/signals/otel.rs
  • Remove legacy constants from crates/brightstaff/src/tracing/constants.rs
  • Bump version per the release process in CLAUDE.md

Context

  • PR: #903 (merged, commit SHAs c7f31879..cee89950)
  • Python reference: katanemo/signals @ 0.1.0
  • Parity harness: tests/parity/signals/ + .github/workflows/parity-signals.yml
  • Last parity run: 100% Tier-A agreement, 0 Tier-B divergences, 2,000 samples, seed 42