SoulX-FlashTalk is the first 14B model to achieve sub-second start-up latency (0.87s) while maintaining a real-time throughput of 32 FPS on an 8xH800 node.
SoulX-FlashTalk is the first 14B model to achieve sub-second start-up latency (0.87s) while maintaining a real-time throughput of 32 FPS on an 8xH800 node.
## Online Demos ## Examples ## Quickstart ### Installation #### 1. Create a Conda environment ```bash conda create -n flashtalk python=3.10 conda activate flashtalk ``` #### 2. Install PyTorch on CUDA ```bash pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128 ``` #### 3. Install other dependencies ```bash pip install -r requirements.txt ``` #### 4. Flash-attention installation: ```bash pip install ninja pip install flash_attn==2.8.0.post2 --no-build-isolation ``` #### 5. FFmpeg installation ```bash # Ubuntu / Debian apt-get install ffmpeg # CentOS / RHEL yum install ffmpeg ffmpeg-devel ``` or ```bash # Conda (no root required) conda install -c conda-forge ffmpeg==7 ``` ### Model download | Model Component | Description | Link | | :--- | :--- | :---: | | `SoulX-FlashTalk-14B` | Our 14b model| [Huggingface](https://huggingface.co/Soul-AILab/SoulX-FlashTalk-14B) | | `chinese-wav2vec2-base` | chinese-wav2vec2-base | [Huggingface](https://huggingface.co/TencentGameMate/chinese-wav2vec2-base) | ```bash # If you are in china mainland, run this first: export HF_ENDPOINT=https://hf-mirror.com pip install "huggingface_hub[cli]" huggingface-cli download Soul-AILab/SoulX-FlashTalk-14B --local-dir ./models/SoulX-FlashTalk-14B huggingface-cli download TencentGameMate/chinese-wav2vec2-base --local-dir ./models/chinese-wav2vec2-base ``` ### Inference ```bash # Infer on single GPU # Requires more than 64G of VRAM. Use --cpu_offload to reduce VRAM usage to 40G. bash inference_script_single_gpu.sh # Infer on multy GPUs # Real-time inference speed can only be supported on 8xH800 or higher graphics cards bash inference_script_multi_gpu.sh ``` ### Online Demo Coming Soon! ## Contact Us If you are interested in leaving a message to our work, feel free to email [email protected] or [email protected] or [email protected] or [email protected] or [email protected] Due to Group 1 reaching its capacity, we have opened a new WeChat group. Additionally, we represent **SoulApp** and warmly welcome everyone to download the app and join our Soul group for further technical discussions and updates! ## Citation If you find our work useful in your research, please consider citing: ``` @misc{shen2025soulxflashtalk, title = {{SoulX-FlashTalk}: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation}, author = {Shen, Le and Qiao, Qian and Yu, Tan and Zhou, Ke and Yu, Tianhang and Zhan, Yu and Wang, Zhenjie and Tao, Ming and Yin, Shunshun and Liu, Siyuan}, year = {2025}, eprint = {2512.23379}, archivePrefix = {arXiv}, primaryClass = {cs.CV}, doi = {10.48550/arXiv.2512.23379}, url = {https://arxiv.org/abs/2512.23379} } ``` ## Acknowledgement - [Infinitetalk](https://github.com/MeiGen-AI/InfiniteTalk) and [Wan](https://github.com/Wan-Video/Wan2.1): the base model we built upon. - [Self forcing](https://github.com/guandeh17/Self-Forcing): the codebase we built upon. - [DMD](https://github.com/tianweiy/DMD2) and [Self forcing++](https://github.com/justincui03/Self-Forcing-Plus-Plus): the key distillation technique used by our method. > [!TIP] > If you find our work useful, please also consider starring the original repositories of these foundational methods. ## Star History
如何每将每秒处理帧数从25降到12fps来提速
生成demo效果很差
Background jitter in Stage-1 when switching to short chunks
Is it possible to output a video with 16fps or 8fps?
感谢大佬工作，falshtalk非常猛，我们的Opentalking开源接入了flashtalk，适配了下昇腾910B！
unofficial training reproduction
Video2Video Inferencing generate lip sync in existing video input
最低什么配置可以推理？
45G显存也报错OutOfMemoryError
Will you release the code of "Online Demo"?