Uncommitted RemoteRemoved TLC is skipped during on-chain timeout reconciliation
Scenario
Topology: A-B-C-D, payment direction A -> D. B force-closes the B-C channel, which is still in ShuttingDown(WAITING_COMMITMENT_CONFIRMATION) when D's TLC expires.
C propagates RemoveTlc to B. B receives the peer message and immediately calls set_offered_tlc_removed(), changing the downstream TLC to RemoteRemoved and setting removed_reason.
The subsequent CommitmentSigned is rejected because B-C is already in the shutdown state with WAITING_COMMITMENT_CONFIRMATION. The remove handshake therefore never reaches RemoveAckConfirmed or apply_remove_tlc_operation, and B never propagates the result to the upstream A-B TLC.
Failure
After the TLC is settled on-chain, timeout reconciliation skips it because removed_reason is already present:
.filter(|tlc| tlc.removed_reason.is_none())has_unresolved_onchain_tlcs() also treats every TLC with removed_reason.is_some() as already resolved, even when the off-chain remove commitment handshake was not confirmed.
As a result, the B-C actor considers on-chain reconciliation complete and stops, while the upstream A-B offered TLC remains permanently present. The integration test test_mid_node_shutdown_when_d_expiry eventually times out waiting for A's offered TLC balance to become zero.
The issue is intermittent because it depends on whether C's RemoveTlc reaches B while B-C is still shutting down. If the B-C closing transaction is confirmed first, the peer message is dropped for the closed channel and B's local removed_reason remains empty, so the on-chain timeout path can reconcile the TLC.
Failure sequence
sequenceDiagram
autonumber
participant A
participant B
participant C
participant D
participant Chain as On-chain settlement
A->>B: AddTlc
B->>C: AddTlc
C->>D: AddTlc
B->>B: Force-close B-C
Note over B,C: B-C = ShuttingDown<br/>WAITING_COMMITMENT_CONFIRMATION
D-->>C: TLC expires / RemoveTlc
C-->>B: RemoveTlc
B->>B: Mark TLC as RemoteRemoved
B->>B: removed_reason = Some(...)
C--xB: CommitmentSigned is rejected
Note over B,C: RemoveTlc is not committed<br/>RemoveAckConfirmed is never reached
B--xA: RemoveTlc is not propagated to A-B
Chain-->>B: B-C TLC is settled on-chain
B->>B: Filter by removed_reason.is_none()
Note right of B: The TLC is skipped
B->>B: has_unresolved_onchain_tlcs = false
B->>B: Treat reconciliation as complete<br/>and stop the actor
Note over A,B: A-B offered TLC remains permanently
A->>A: Wait 70 seconds
A--xA: Integration test times outSource: nervosnetwork/fiber