[Bug]: LFM2.5 350M and 1.2B silently generate garbage on NPU DEVICE_ARCHITECTURE=4000 (Lunar Lake) — correct on 3720, CPU and GPU

OpenVINO Version

2026.3.0-22451-8a17657b995-releases/2026/3 (also reproduced on 2026.3.1 and on the 2026.5.0 nightly, 2026.5.0-22942-597262ef181)

Operating System

Windows System

Device used for inference

NPU

Framework

openvino-genai 2026.3.0.0-3277-bd8d6542e3c, Python 3.13

Model used

OpenVINO/LFM2.5-350M-int8-ov (Intel's own published export) and aweussom/LFM2.5-1.2B-Instruct-int4-cw-ov

Issue description

LFM2.5 350M and 1.2B generate garbage on an NPU whose DEVICE_ARCHITECTURE is 4000 (Lunar Lake), and correct text on 3720 (Arrow Lake) — with the model files, OpenVINO version, compiler and NPU driver all held constant. CPU and GPU are correct on the same machine, in the same venv, in the same session. There is no error, no warning: it compiles, it runs at full speed, and the tokens are wrong.

@Zulkifli-Intel asked me to open this as its own issue (openvinotoolkit/openvino#37322, 2026-09-12). It is the same failure he reported himself in that thread on 2026-08-13 — "the LFM2.5-1.2B model is supported on the NPU, but when I tested it on the NPU, it gave gibberish responses (GPU and CPU had no issues)" — and said he was checking internally. That thread is titled for a different bug (an unordered_map crash in LFM2.5-2.6B, since resolved by a nightly), so the accuracy defect never had a home of its own. Full evidence is in my comment there; this issue restates it standalone.

Two points from the release notes that make this worth a look:

1. 2026.3.1's notes say "Fixed accuracy issue for lfm2-1.2b, lfm2.5-1.2b-instruct models for NPU device target." That fix does not hold on 4000, with either driver.
2. The models the notes list as NPU-validated are the ones failing. Validation covers LFM2-1.2B and LFM2.5-1.2B; those are the builds that produce gibberish on 4000.

Scope — what this is not. @DavidDohmen reports LFM2.5-2.6B int8 running correctly at ~21 tok/s on a 256V, which is also 4000. So this is not "all LFM2 on this NPU generation". Every failure we have is on the 350M and 1.2B builds. That split may be the useful clue.

Step-by-step reproduction

import sys
import openvino_genai as ovg, openvino as ov

dev = sys.argv[1] if len(sys.argv) > 1 else "NPU"
core = ov.Core()
if dev == "NPU":
    print("arch", core.get_property("NPU", "DEVICE_ARCHITECTURE"),
          "| driver", core.get_property("NPU", "NPU_DRIVER_VERSION"))

# huggingface-cli download OpenVINO/LFM2.5-350M-int8-ov --local-dir ./lfm2.5-350m
kwargs = {"MAX_PROMPT_LEN": 1024} if dev == "NPU" else {}   # NPU-only property
pipe = ovg.LLMPipeline("./lfm2.5-350m", dev, **kwargs)
cfg = ovg.GenerationConfig(); cfg.max_new_tokens = 40; cfg.do_sample = False
for p in ("Say hello.", "What is 2+2?", "What is the capital of Norway? Answer in one word."):
    print(dev, repr(pipe.generate(p, cfg)))

Run it as python repro.py NPU, then python repro.py CPU, then python repro.py GPU, in that order, in one venv.

Relevant log output

Failing — Core Ultra 7 258V (Lunar Lake), DEVICE_ARCHITECTURE=4000, driver 32.0.100.5540:

arch 4000 | driver 1005540
NPU ''
NPU ' Tong、/神可乱 evidently shovexf 三 shove shove shove ruinedファ機主助{ stupid evidently shoveت terribly shove shove shove huh保出意文 shove shove shove huha shit'
NPU ' Nielsen  神意 terrific主ビqulocal�主三主 stupid shove shove shove shove shove shove shove randomised shove huh保二 ridiculous否 cried保 shove shove shove huh意 ridiculous消主意'

Same machine, same venv, same files, CPU and GPU:

CPU 'Hello! How can I help you today?'   GPU 'Hello! How can I help you today?'
CPU '2 + 2 = 4'                          GPU '2 + 2 = 4'
CPU 'Oslo'                               GPU 'Oslo'

Positive control on the other NPU generation — identical script and files on a Core Ultra 9 285K (Arrow Lake, DEVICE_ARCHITECTURE=3720), same OpenVINO 2026.3.0, same driver:

Only the NPU differs.

What does not change the result on NPU 4000

Every cell below produces byte-identical garbage — the same strings on every run:

It is silent numerical breakage, not a crash or a stall: decode runs at full speed (46–48 tok/s on the 1.2B, TTFT ~0.4 s). In the same session on the same NPU, SmolLM3-3B-int8-cw and Qwen3-8B-int4-cw are coherent — the device is healthy.

Two stopping signatures, and the prompt picks between them

Taken from DecodedResults.finish_reasons rather than inferred from token counts, at two caps so "hit the cap" is distinguishable from "the answer was that long". Driver 32.0.100.5540, 4000, OpenVINO 2026.3.0:

So on the same device: one model runs to the cap and never stops, and Intel's 350M emits EOS as its very first token on Say hello., giving the empty string — that is the NPU '' line above. I originally read that as "no output"; it is not. It is a confident immediate end-of-sequence.

The CPU rows are the control that makes this worth your time: same file, same script, same session, EOS emitted correctly after 10 and 3 tokens and independent of the cap. So neither signature is an artifact of genai, of the NPU pipeline, or of the measurement — the sampler reports STOP perfectly well on 4000 when the logits ask it to. Whatever is wrong sits upstream of the stopping logic and is corrupting the distribution itself, sometimes into a repeating loop and sometimes straight onto the EOS token.

To see this, the script above needs the list form of generate — a bare-string prompt returns a plain str with no metrics attached:

for cap in (24, 72):
    cfg = ovg.GenerationConfig(); cfg.max_new_tokens = cap; cfg.do_sample = False
    r = pipe.generate([prompt], cfg)          # list form -> DecodedResults
    print(cap, r.perf_metrics.get_num_generated_tokens(),
          [str(x) for x in r.finish_reasons], repr(r.texts[0][:60]))

Environment

Happy to run anything you want on either box — having both NPU generations on identical software makes this cheap to bisect from our side.

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, forum, etc. and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.