[Bug] Omni reuses stale PLE input across successive text requests

Summary

With Gemma 4 E2B, the first text request completes normally, but subsequent requests after resetting the conversation on the same inference instance can produce unrelated or repetitive output until the token limit.

The text-only embedding path appears to reuse PLE input left over from the previous decode step.

Platform and version

• Device: HUAWEI MatePad Edge, Kirin X90, 32 GB RAM.
• OS: HarmonyOS, arm64-v8a.
• Inference backend: CPU.
• MNN revision tested: 2edeef91b425e98a93707840b6fffdd97980bdbe.

Model and reproduction

Model: MNN/gemma-4-E2B-it-MNN, using the same model files for all comparisons.

Reproduced through our application using MNN’s C++ LLM interface:

1. Load the model and send a text-only prompt.
2. Wait for generation to finish.
3. Reset the conversation without unloading the model.
4. Send another text prompt and repeat.

The first request completed normally; the following four produced abnormal output and reached the 512-token limit. A standalone llm_demo reproduction has not yet been verified.

Runtime logs and diagnosis

Diagnostic logging captured a new 109-token input entering the embedding path while the cached mPleInput had a sequence length of 1.

In Omni::embedding(), the text-only branch calls Llm::embedding() without clearing the previous mPleInput. The base implementation only recomputes PLE when:

mPleEmbedding && (!mPleInput.get() || seq_len == 1)

A subsequent multi-token prompt therefore bypasses recomputation when PLE remains non-null after the previous single-token decode step.

Preserving precomputed PLE is needed for multimodal prefill, but this condition does not distinguish current multimodal data from stale data belonging to an earlier request.

Locally verified fix

Inside Omni’s text-only branch, before calling Llm::embedding():

if (mPleEmbedding && input_ids.size() > 1) {
    mPleInput = nullptr;
}

This allows PLE to be recomputed for the current text input while leaving the multimodal branch unchanged.

With the same application code and model files, changing only the MNN library:

• Five fresh text requests completed normally after the fix.
• A two-turn recall test and image/text alternation also passed.

These results support this specific stale-PLE diagnosis; they do not establish that other Gemma output issues share the same cause.