#1440 Controlled reproduction: raw-CSI presence detection sits at chance (AUC ≈ 0.58) once time-of-day is controlled — the two apparent successes were confounds · RuView

First, thanks for open-sourcing this and for being so responsive on the tracker. I'm building toward whole-home presence without any indoor cameras (local Home Assistant, cameras outside only), so WiFi CSI is the one modality that could give me per-room "someone's here" without a lens. I got the hardware up, streaming real CSI, and ran a controlled experiment to completion. Posting as an issue per your steer that reproductions belong here rather than in discussions.

TL;DR: the underlying CSI is real and streams fine at --edge-tier 0, but I could not get reliable boolean presence out of it, and the two results that looked like a breakthrough both collapsed under a same-time-of-day control. This lines up with what others are reporting (#1401 "flickers present in an empty room", #1422 breathing extractor pins to 30 BPM on an empty room). I'd like guidance on whether a phase-aware path or the new ruview-unified model is the intended fix.

Setup

3× ESP32-S3-DevKitC-1 (WROOM-1 N16R8), firmware v0.6.7, all re-provisioned --edge-tier 0 (raw CSI passthrough), unique --node-id, --tdm-slot 0/1/2 --tdm-total 3.
Server ruvnet/wifi-densepose:v0.8.4-esp32, --net=host, --source esp32, /health → "source":"esp32", all three nodes one bedroom, same AP.

Confirmed real hardware, not sim: tcpdump'd the UDP frames from the three ESP32 IPs and hand-decoded against firmware/.../csi_collector.c (ADR-018): magic 0xC5110001, 20-byte LE header, int8 I/Q, 64-subcarrier HT20 (some 128), subcarrier profiles with the expected DC-null shape and real temporal variation, ~35 fps aggregate. The pipeline genuinely works end to end.

Two corrections for others reading this thread

1. Raw CSI is only available at --edge-tier 0. The default (tier 2) does the FFT/DSP on-chip and ships derived scalars, so the recording API shows amplitude: [] and every recorded frame has zero CSI. Configuration gap, not architectural.

2. ruvnet/wifi-densepose-pretrained is not a raw-CSI presence model, and its 82.3% is not presence accuracy. From its own csi-embed-v2.py/config.json it's an 8→64→128 MLP on the 8 pre-extracted summary features, and the 82.3% is heldout_triplet_acc — a contrastive metric on which a random encoder scores 69.6% and raw features 66.4% (csi-embed-v2-metrics.json). Training adds ~13 pts on a representation task; nothing published claims it's a presence classifier.

The experiment — and why the control matters

A naive occupied-vs-empty test is confounded by time-of-day channel drift, so I ran three windows:

5.9 h occupied overnight (asleep in the room)
a fresh empty baseline after leaving
an occupied-again window 37 min after the empty one — same time of day, so drift is controlled

Amplitude window features, balanced classes, AUC (imbalance-robust), 20–30 draws:

Node	night-occupied vs empty (confounded)	occupied-again vs empty (drift-controlled)
node1	1.000	1.000
node2	0.657	0.607
node3	0.963	0.581
all 3	0.752	0.576

The confounded column looks great (node3 0.96!). The controlled column is the honest one:

node3's 0.96 was overnight drift — controlled, it drops to 0.581 (chance).
node1's perfect 1.000 was a link/PHY artifact. During the empty window node1 was on a different PHY: 128 subcarriers, versus 64 subcarriers in both occupied windows — a person does not change your subcarrier count, so the classifier was trivially separating "node1 on HT40" from "node1 on HT20" (a re-association; these nodes grab an AP at boot and occasionally re-associate). I originally also cited an RSSI swing here, but per #1348 the server's reported RSSI is parsed from the wrong byte offset (sequence-counter bytes, not byte 16), so I'm no longer relying on the dBm figures — the subcarrier-count change, which #1348 confirms parses correctly, is the reliable tell.
Controlled, stable-link, combined: AUC ≈ 0.576 — barely above a coin flip with amplitude-only features.

This matches the server's own adaptive classifier for me: after room calibration (recorded train_empty_room/train_still_presence/train_walking/train_active_movement, correctly split into the four classes), training accuracy was 0.45 on 4 classes, and a known-empty room still read "present" 31/31 frames — the same empty-room-reads-present symptom in #1401 and #1422.

Two robustness notes: the per-node AUCs don't depend on cross-node TDM sync, so imperfect TDM isn't the cause; and the same-time-of-day control neutralizes the "fan/AC false-positive" caveat, since a stationary interferer is present in both windows.

Where I think the signal might be — questions

I suspect the info is there and I'm not extracting it. Leading suspects, and I'd love to know which to prioritize:

Phase. I used amplitude only and discarded phase — presence/breathing often lives in phase and the 0.1–0.5 Hz band. Is phase the main lever? Recommended phase sanitization (linear detrend / conjugate-multiply between antennas) for these boards?
The ruview-unified model. I see the new feat(ruview-unified): unified RF spatial world model work (ADR-273..282, separable delay-Doppler, Gaussian world model). Is that the intended presence path going forward, and is it usable against a tier-0 ESP32-S3 capture today? If so I'll re-run against it.
A presence-specific head + the jsonl→rvf path. Is there a presence classifier/head I should load instead of the triplet encoder, and is there a working jsonl → rvf converter now (saw #1233/#1322)?
Placement / window. Mine were ~4 m apart, one bedroom, not rigorously to the placement guide. How much does correct placement actually move presence AUC in your experience, and would breathing-band spectral features beat my crude window stats?

Offer

I have the 3-node rig up, a clean overnight raw-CSI capture (749k frames, ADR-018 parsed), and Python parse/analysis scripts I'm happy to share so results are comparable. I'll gladly re-run with whatever placement/firmware/feature changes you suggest — or against ruview-unified — and post updated controlled numbers. If a phase-aware model on controlled data is the unlock, I've got idle GPUs to throw at training a room-specific one.

I'd genuinely like to be proven wrong by a phase-aware model on same-time-of-day-controlled captures. My one ask of anyone reporting positive results: run the same empty↔occupied control, because an uncontrolled occupied(night)-vs-empty(morning) comparison will happily hand you an AUC of 0.96 that means nothing.