sherpa-ncnn

Supported functions

Real-time Speech recognition Speech synthesis Voice activity detection ✔️ ✔️ ✔️

Supported platforms

Architecture Android iOS Windows macOS linux x64 ✔️ ✔️ ✔️ ✔️ x86 ✔️ ✔️ arm64 ✔️ ✔️ ✔️ ✔️ ✔️ arm32 ✔️ ✔️ riscv64 ✔️

Supported programming languages

1. C++ 2. C 3. Python 4. JavaScript ✔️ ✔️ ✔️ ✔️ 5. Go 6. C# 7. Kotlin 8. Swift ✔️ ✔️ ✔️ ✔️

It also supports WebAssembly.

Introduction

This repository supports running the following functions locally

• Streaming speech-to-text (i.e., real-time speech recognition)
• Text to speech (e.g., vits models from piper)
• VAD (e.g., silero-vad)

on the following platforms and operating systems:

• x86, x86_64, 32-bit ARM, 64-bit ARM (arm64, aarch64), RISC-V (riscv64)
• Linux, macOS, Windows, openKylin
• Android, WearOS
• iOS
• NodeJS
• WebAssembly
• Raspberry Pi
• RV1126
• LicheePi4A
• VisionFive 2
• 旭日X3派
• etc

with the following APIs

• C++, C, Python, Go, C#
• Kotlin
• JavaScript
• Swift

We support all platforms that ncnn supports.

Everything can be compiled from source with static link. The generated executable depends only on system libraries.

HINT: It does not depend on PyTorch or any other inference frameworks other than ncnn.

Please see the documentation https://k2-fsa.github.io/sherpa/ncnn/index.html for installation and usages, e.g.,

• How to build an Android app
• How to download and use pre-trained models

We provide a few YouTube videos for demonstration about real-time speech recognition with sherpa-ncnn using a microphone:

• English: https://www.bilibili.com/video/BV1TP411p7dh/

• Chinese: https://www.bilibili.com/video/BV1214y177vu

• Multilingual (Chinese + English) with endpointing Python demo : https://www.bilibili.com/video/BV1eK411y788/

• Android demos

• Multilingual (Chinese + English) Android demo 1: https://www.bilibili.com/video/BV1Ge411A7XS

• Multilingual (Chinese + English) Android demo 2: https://www.bilibili.com/video/BV1eK411y788/

• Chinese (with background noise) Android demo : https://www.bilibili.com/video/BV1GR4y167fx

• Chinese Android demo : https://www.bilibili.com/video/BV1744y1Z76H

• Chinese poem with background music Android demo : https://www.bilibili.com/video/BV1vR4y1k7eo

Links for pre-built Android APKs

Description URL Streaming speech recognition Address

Links for pre-trained models

https://github.com/k2-fsa/sherpa-ncnn/releases/tag/models

Useful links

• Documentation: https://k2-fsa.github.io/sherpa/ncnn/
• Bilibili 演示视频: https://search.bilibili.com/all?keyword=%E6%96%B0%E4%B8%80%E4%BB%A3Kaldi

How to reach us

Please see https://k2-fsa.github.io/sherpa/social-groups.html for 新一代 Kaldi 微信交流群 and QQ 交流群.

See also

• https://github.com/k2-fsa/sherpa-onnx
• https://github.com/k2-fsa/sherpa