[BUG] Streaming zipformer silently produces no output on Snapdragon 8 Elite Gen 5 (SM8850): onnxruntime 1.27.0 miscomputes the encoder, 1.28.0 fixes it

Important Instructions ⚠️

• I am using the latest version of sherpa-onnx. If not, I will upgrade and check if the issue persists.
• I understand that issues not reproducible on the latest version may be closed without response.
• I have verified that the error is reproducible using only sherpa-onnx, not code from my own project.
• I will provide detailed steps to reproduce the issue.
• I will provide complete logs and error messages.
• I understand that issues not following these instructions may be closed or receive no response.

Notes on the checkboxes above, so you can judge how much to trust them:

• Latest version: tested with the v1.13.4 Android AAR (latest release), and I also checked master at f737aeb (2026-08-06) — build-android-arm64-v8a.sh still defaults to onnxruntime_version=1.27.0, and I found no commit bumping it, so this is not already fixed on master.
• Reproducible with only sherpa-onnx: yes — official model, official test WAV, official keyword file, default settings. More importantly, the decisive evidence below involves no sherpa-onnx code at all: I ran the three .onnx files straight through the onnxruntime Java API and compared numbers.

Describe the issue

On Snapdragon 8 Elite Gen 5 (SM8850) with Android 16, streaming zipformer models silently produce no output at all. KeywordSpotter never fires, and OnlineRecognizer returns empty text for the entire stream. There is no crash, no exception, no NaN, and nothing in the logs.

Offline models on the same device work correctly, so this is not a general onnxruntime failure on the device.

Root cause: onnxruntime 1.27.0 computes wrong values for the zipformer2 encoder on this SoC. onnxruntime 1.28.0 produces correct values. sherpa-onnx currently bundles 1.27.0 for Android, so any app using it for wake-word detection is broken on this chip.

The pipeline runs structurally correctly — I instrumented it, and the counts are identical on a working and a failing device:

That is why it is easy to misdiagnose: everything looks healthy except the numbers.

Steps to reproduce

A. Reproduce the symptom (sherpa-onnx only)

1. Take an SM8850 device (Xiaomi 17 / Android 16 in my case).
2. Use the KeywordSpotter API with the official model sherpa-onnx-kws-zipformer-zh-en-3M-2025-12-20 (int8 encoder/joiner, fp32 decoder).
3. Feed the official test_wavs/zh_4.wav (contains 女兒) in 1600-sample chunks, calling isReady/decode/getResult as usual.
4. Expected: 女兒 is detected. Actual: nothing, even with keywordsThreshold=0.05 and keywordsScore=2.0.
5. The same code and models on a Snapdragon 845 device detect it every time.

The same happens with OnlineRecognizer + sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20 (modelType=zipformer): empty text for the whole stream, including every intermediate partial result.

B. Isolate it to onnxruntime (no sherpa-onnx code)

Run the three .onnx files directly through the onnxruntime Java API on-device with deterministic inputs, and compare against the same run on x86_64:

• float inputs: ((i % 100) + 1) * 0.01
• int64 inputs: i % 10
• all dynamic dimensions = 1

Sum of all output elements:

Only the encoder is wrong. Not NaN, correct shape, just wrong values (min/max also differ: expected −1.776/1.910, observed −3.548/3.516). The first-layer cached state is only ~0.14% off and the error compounds through the layers, so the transducer emits blanks forever — hence the silent failure.

Expected behavior

KeywordSpotter should detect the keyword on SM8850 exactly as it does on older Snapdragon devices with the identical APK, models and settings.

Environment

• sherpa-onnx version: v1.13.4 (Android AAR); also checked master @ f737aeb
• OS: Android 16 (API 36), HyperOS OS3.0.302.0, kernel BP2A.250605.031.A3
• Device / SoC: Xiaomi 17 (25113PN0EG, pudding_global) / SM8850, Qualcomm Oryon. /proc/cpuinfo features include sme smei8i32 smef16f32 smeb16f32 smef32f32 — SME but no SME2
• ABI: arm64-v8a, page size 4KB
• Python version: N/A (Android, Kotlin/JNI)
• onnxruntime version: 1.27.0 (as bundled) — broken; 1.28.0 — fixed
• GPU / NPU: not used, CPU execution provider (also tried XNNPACK and NNAPI)
• Control device: Samsung Note 9 (SM-N960F, Snapdragon 845, Android 10) — works

Logs

There is no error output — that is the core problem. The session is created normally:

W/sherpa-onnx: .../jni/keyword-spotter.cc:Java_com_k2fsa_sherpa_onnx_KeywordSpotter_newFromAsset:93
W/sherpa-onnx: config:
W/sherpa-onnx: KeywordSpotterConfig(feat_config=FeatureExtractorConfig(sampling_rate=16000,
  feature_dim=80, ...), model_config=OnlineModelConfig(transducer=OnlineTransducerModelConfig(
  encoder="...encoder-epoch-13-avg-2-chunk-16-left-64.int8.onnx",
  decoder="...decoder-epoch-13-avg-2-chunk-16-left-64.onnx",
  joiner="...joiner-epoch-13-avg-2-chunk-16-left-64.int8.onnx"), ...
  provider_config=ProviderConfig(device=0, provider="cpu", ...)

My instrumented self-test output on the failing device (ORT 1.27.0):

Device: Xiaomi 25113PN0EG / Android 16 (API 36) / arm64-v8a / page size 4KB / threads 2
asset sizes: encoder int8 4599656, tokens.txt 1928, keywords.txt 28, zh_4.wav 147216
[fp32 / 2 threads / threshold 0.25] no detection (370ms, 46 chunks fed, 13 decode calls)
[fp32 / 1 thread  / threshold 0.05] no detection (340ms, 46 chunks fed, 13 decode calls)
[int8 / 2 threads / threshold 0.25] no detection (366ms, 46 chunks fed, 13 decode calls)
[int8 / 1 thread  / threshold 0.05] no detection (365ms, 46 chunks fed, 13 decode calls)

Same self-test after replacing libonnxruntime.so with 1.28.0:

[CPU     / int8] detected 女兒 (510ms, 46 chunks fed, 13 decode calls)
[CPU 1thr/ int8] detected 女兒 (514ms, 46 chunks fed, 13 decode calls)
[XNNPACK / int8] detected 女兒 (611ms, 46 chunks fed, 13 decode calls)
[NNAPI   / int8] detected 女兒 (515ms, 46 chunks fed, 13 decode calls)

Repeated 3+ times, identical every run. No regression on the Snapdragon 845 control device (wake word, offline ASR and everything else still fine).

Ruled out

These made no difference at all — identical failure, identical timings:

• mlas.disable_kleidiai=1 via the provider="cpu:<config>" mechanism. I verified the config was actually applied (with DEBUG=1 the native layer logged Provider config: SessionConfig.mlas.disable_kleidiai=1), so this is a real negative, not a silently-ignored setting.
• GraphOptimizationLevel=0
• Execution provider: CPU / XNNPACK / NNAPI
• Thread count (1 vs 2), keywordsThreshold (0.25 vs 0.05)
• int8 vs fp32 under 1.27.0 — both broken
• 16KB page alignment (all bundled .so are already 16KB-aligned; this device uses 4KB pages anyway)

What I could NOT explain

Stating this explicitly so nobody over-reads the results above:

• Under ORT 1.28.0 the fp32 encoder still fails in the real streaming pipeline, even though my single-step fixed-input probe of that same fp32 file matched the reference exactly under 1.28.0. So "1.28.0 fixes fp32" is not established — only int8 is verified end-to-end. It may be input-range dependent or specific to multi-step stateful execution; I did not chase it.
• I did not re-test OnlineRecognizer (streaming ASR) under 1.28.0, only KeywordSpotter.
• I have only one affected device, so I cannot tell whether this reproduces on every SM8850 unit or is specific to this OS build.

Request

Please bump the Android onnxruntime from 1.27.0 to 1.28.0 (build-android-arm64-v8a.sh line 93, plus the corresponding onnxruntime-android-1.28.0.zip in csukuangfj/onnxruntime-libs, which currently tops out at 1.27.1).

Worth noting for anyone hitting this before a release: swapping only libonnxruntime.so does not work as a user-side workaround, because libsherpa-onnx-jni.so carries a versioned symbol requirement (OrtGetApiBase@@VERS_1.27.0) and the loader rejects the newer library with cannot locate symbol "OrtGetApiBase". It has to be rebuilt against 1.28.0.

This matters beyond a single phone: SM8850 is this generation's flagship SoC and will ship in a large number of devices, and the failure is completely silent — no crash, no log, correct-looking control flow — so affected developers are likely to suspect their own code, their microphone, or their keywords long before they suspect the inference engine.

Related

• #3490 (SIGILL on the same SoC family, fixed by moving to ORT 1.24.3+)
• microsoft/onnxruntime#27884, #26361, #27282 (SIGILL on SM8850 / Android 16)
• microsoft/onnxruntime#26377 (KleidiAI enabled on SME-without-SME2 hardware)

Those are all crash reports. This is the silent-wrong-results variant, on a version where those crashes are already fixed. I could not find it reported.