Canary: lang2id is hardcoded to en/es/de/fr, so any other language silently falls back to English

sherpa-onnx v1.13.8 (and master as of 2026-09-17), Python API, CPU.

OfflineRecognizer.from_nemo_canary only honors four languages: en, es, de, fr. For any other src_lang/tgt_lang the initial decoder input silently falls back to the en token, so multilingual canary exports such as nvidia/canary-1b-v2 (25 European languages, sherpa-layout int8 export exists at Sarphix/canary-1b-v2-sherpa-onnx-int8) transcribe or translate into English regardless of the requested language.

Repro (Italian voice message, canary-1b-v2 int8 export):

import sherpa_onnx
r = sherpa_onnx.OfflineRecognizer.from_nemo_canary(
    encoder="encoder.int8.onnx", decoder="decoder.int8.onnx", tokens="tokens.txt",
    src_lang="it", tgt_lang="it", feature_dim=128, num_threads=4)
# -> English translation-style output ("Hello, my name is Carlo ..."),
#    identical to src_lang="it", tgt_lang="en" and to src_lang="en".
# fr/fr, es/es and de/de decode correctly on the same export.

Root cause: sherpa-onnx/csrc/offline-recognizer-canary-impl.h builds meta.lang2id from a hardcoded four-entry map (lines 303-306 at v1.13.8), and GetInitialDecoderInput falls back to lang2id.at("en") for any src/tgt missing from it (lines 263-275). The model's own tokens.txt carries the other language tokens (<|it|> etc.), and the encoder metadata is otherwise sound.

Suggested fix: derive lang2id from the model's token table (or an encoder-metadata language list) instead of the hardcoded map, so each export declares the languages it supports. At minimum, an unknown src_lang/tgt_lang could log a warning instead of silently switching the target language.

We ship sherpa-onnx on Android for on-device transcription and hit this while evaluating canary-1b-v2 for Italian; I can test a patch against the int8 export.

Environment

• sherpa-onnx 1.13.8 (Python, CPU), also present on master (2026-09-17)
• Model: Sarphix/canary-1b-v2-sherpa-onnx-int8 (nvidia/canary-1b-v2 base)