Optimistic Execution can retain stale height when ProcessProposal is skipped, causing misleading hash mismatch on next FinalizeBlock

Summary

We observed a reproducible Optimistic Execution edge case where OE remains initialized for height H, while ProcessProposal(H+1) is skipped by CometBFT.

When FinalizeBlock(H+1) arrives, Cosmos SDK compares the stale OE hash from height H against the FinalizeBlock hash from height H+1, producing:

OE aborted due to hash mismatch

The fallback path works correctly and the block is finalized successfully, so this does not appear to cause consensus failure or state corruption.

However, the log output is misleading because req_height currently prints oe.request.Height instead of the actual height of the incoming FinalizeBlock request.

We reproduced this multiple times on a non-validator synchronized node running:

• Cosmos SDK v0.54.4
• CometBFT v0.39.4
• IBC-Go v11.2.0
• Go 1.26.5

Optimistic Execution is enabled.

Observed behavior

Example at heights 323104 / 323105.

At height 323104, the node receives and processes the proposal normally:

received proposal ... height=323104 proposer=DEAB2F523DD97727B306373E9650F1A5B9B04B8F
received complete proposal block hash=56C44AECB60CDC6A42D99CF599D793DC1ADF7EDDDCE4375D1150B330DF62F768 height=323104
finalizing commit of block hash=56C44AECB60CDC6A42D99CF599D793DC1ADF7EDDDCE4375D1150B330DF62F768 height=323104

Optimistic Execution is started from ProcessProposal(323104) with:

oe_hash=56C44AECB60CDC6A42D99CF599D793DC1ADF7EDDDCE4375D1150B330DF62F768

Height 323104 finalizes successfully.

At height 323105, the node does not receive/process the proposal before the proposal timeout:

Timed out ... height=323105 step=RoundStepPropose

It then receives a commit for a block it does not yet know:

commit is for a block we do not know about; set ProposalBlock=nil
commit=6CEB23AC86BDC1E0514DEACE8ADECF1EE731238628D7828E4B5A265782B3830A
height=323105

The block is subsequently received:

received complete proposal block hash=6CEB23AC86BDC1E0514DEACE8ADECF1EE731238628D7828E4B5A265782B3830A height=323105
finalizing commit of block hash=6CEB23AC86BDC1E0514DEACE8ADECF1EE731238628D7828E4B5A265782B3830A height=323105

Immediately after:

ERR OE aborted due to hash mismatch
module=oe
oe_hash=56c44aecb60cdc6a42d99cf599d793dc1adf7edddce4375d1150b330df62f768
oe_height=323104
req_hash=6ceb23ac86bdc1e0514deace8adecf1ee731238628d7828e4b5a265782b3830a
req_height=323104

The block still finalizes successfully.

Hash verification

We verified both hashes independently against the RPC block store:

height 323104:
56C44AECB60CDC6A42D99CF599D793DC1ADF7EDDDCE4375D1150B330DF62F768

height 323105:
6CEB23AC86BDC1E0514DEACE8ADECF1EE731238628D7828E4B5A265782B3830A

Therefore the mismatch is not between two hashes for the same height.

It is:

OE request from H
vs
FinalizeBlock request from H+1

Second reproduction

The same behavior occurred at heights 322439 / 322440:

oe_hash=34d668232633c3cb434e7f8cd18dca011d207d568d665d05a7d7d7b69c84e40c
oe_height=322439

req_hash=11f8404b38c1aa963f45fbe1f57870912db8aee98f1cf03e81b6c22d42a93d4b
req_height=322439

RPC verification showed:

oe_hash  = block 322439
req_hash = block 322440

Again, height 322440 finalized normally after OE was aborted.

Relevant Cosmos SDK behavior

OptimisticExecution.Execute() stores the ProcessProposal hash and height in an internally constructed RequestFinalizeBlock.

Later, FinalizeBlock() calls:

aborted := app.optimisticExec.AbortIfNeeded(req.Hash)

AbortIfNeeded() compares:

if !bytes.Equal(oe.request.Hash, reqHash) {
    oe.logger.Error(
        "OE aborted due to hash mismatch",
        "oe_hash", hex.EncodeToString(oe.request.Hash),
        "req_hash", hex.EncodeToString(reqHash),
        "oe_height", oe.request.Height,
        "req_height", oe.request.Height,
    )

    oe.cancelFunc()
    return true
}

There are two notable observations.

1. req_height appears incorrect

The log currently uses:

"req_height", oe.request.Height

so both height fields represent the OE request.

In the reproduced example the log says:

oe_height=323104
req_height=323104

but RPC verification proves that req_hash belongs to height 323105.

Conceptually the log should report:

oe_height=323104
req_height=323105

2. Successful OE remains initialized

After a successful OE match, FinalizeBlock() returns the OE result directly.

The OE context is reset on the aborted path, but not after the successful path.

Normally the following ProcessProposal() aborts/replaces any previous OE before starting a new one.

However, if ProcessProposal(H+1) is skipped, the successful OE context for H remains initialized when FinalizeBlock(H+1) arrives.

The resulting flow is:

ProcessProposal(H)
    ↓
OE(H)
    ↓
FinalizeBlock(H) succeeds
    ↓
OE remains initialized
    ↓
ProcessProposal(H+1) is skipped
    ↓
FinalizeBlock(H+1)
    ↓
AbortIfNeeded compares OE(H) with H+1
    ↓
hash mismatch
    ↓
OE is aborted/reset
    ↓
FinalizeBlock(H+1) executes normally

Observed CometBFT sequence

For the affected next height we observe:

RoundStepNewHeight
    ↓
RoundStepPropose timeout
    ↓
commit is for a block we do not know about
    ↓
received complete proposal block
    ↓
FinalizeBlock

There is no proposal processing path before the commit for that height.

Impact

We have observed no consensus or state corruption.

After the mismatch Cosmos SDK correctly falls back to the normal FinalizeBlock execution path.

The node remains synchronized and commits the correct application state.

This therefore appears to be a stale Optimistic Execution lifecycle / observability edge case rather than a consensus safety issue.

Frequency

In our test network this is reproducible around a low-voting-power validator's proposer turn.

The observing node successfully processes that validator's block at height H, but the following height can progress before the node processes the next proposal.

We observed the pattern approximately every 132-133 blocks.

All inspected occurrences showed:

oe_hash  = block H
req_hash = block H+1

Expected behavior / questions

It may be useful for OE to explicitly handle a previous-height request as stale before comparing hashes.

Possible approaches could include:

• resetting consumed OE state after successful FinalizeBlock,
• including the height in the stale-request check,
• resetting OE when incoming FinalizeBlock.Height differs from oe.request.Height.

I am not sure which behavior is preferable because the current OE lifecycle may intentionally retain the context for another reason.

At minimum, reporting the actual incoming FinalizeBlock height would make this behavior much easier to diagnose.

Questions:

1. Is retaining a successfully consumed OE request after FinalizeBlock intentional?
2. Is it expected for FinalizeBlock(H+1) to encounter initialized OE state from H when ProcessProposal(H+1) was skipped?
3. Would explicitly treating a previous-height OE as stale be appropriate?
4. Can the log be changed so req_height represents the actual incoming FinalizeBlock height?

I can provide additional logs or test a proposed patch if useful.