InfiniteTalk

> **TL; DR:** InfiniteTalk is an unlimited-length talking video generation​​ model that supports both audio-driven video-to-video and image-to-video generation

## Latest News * May 21, 2026: We release [***LongCat-Video-Avatar-1.5***](https://meigen-ai.github.io/LongCat-Video-Avatar-1.5-Page/), an upgraded open-source framework for audio-driven human video generation. v1.5 replaces Wav2Vec2 with Whisper-Large for more accurate lip synchronization, achieves production-ready physical rationality and temporal stability with robust long-video generation, generalizes to stylized domains (anime, animals, complex real-world conditions), supports both single-stream and multi-stream audio inputs, and accelerates inference to 8 steps via step distillation. [ [***code***](https://github.com/meituan-longcat/LongCat-Video) | [***weights***](https://huggingface.co/meituan-longcat/LongCat-Video-Avatar-1.5) | [***project page***](https://meigen-ai.github.io/LongCat-Video-Avatar-1.5-Page/) ] * Dec 16, 2025: We are excited to announce the release of **[LongCat-Video-Avatar](https://github.com/MeiGen-AI/LongCat-Video-Avatar)**, a unified model that delivers expressive and highly dynamic audio-driven character animation, supporting native tasks including Audio-Text-to-Video, Audio-Text-Image-to-Video, and Video Continuation with seamless compatibility for both single-stream and multi-stream audio inputs. The release includes our Technical Report, [code](https://github.com/meituan-longcat/LongCat-Video), [model weights](https://huggingface.co/meituan-longcat/LongCat-Video-Avatar), and [project page](https://meigen-ai.github.io/LongCat-Video-Avatar/). * August 19, 2025: We release the [Technique-Report](https://arxiv.org/abs/2508.14033) , weights, and code of **InfiniteTalk**. The Gradio and the [ComfyUI](https://github.com/MeiGen-AI/InfiniteTalk/tree/comfyui) branch have been released. * August 19, 2025: We release the [project page](https://meigen-ai.github.io/InfiniteTalk/) of **InfiniteTalk** ## ✨ Key Features We propose **InfiniteTalk**​​, a novel sparse-frame video dubbing framework. Given an input video and audio track, InfiniteTalk synthesizes a new video with ​​accurate lip synchronization​​ while ​​simultaneously aligning head movements, body posture, and facial expressions​​ with the audio. Unlike traditional dubbing methods that focus solely on lips, InfiniteTalk enables ​​infinite-length video generation​​ with accurate lip synchronization and consistent identity preservation. Beside, InfiniteTalk can also be used as an image-audio-to-video model with an image and an audio as input. - ​​Sparse-frame Video Dubbing​​ – Synchronizes not only lips, but aslo head, body, and expressions - ⏱️ ​​Infinite-Length Generation​​ – Supports unlimited video duration - ✨ ​​Stability​​ – Reduces hand/body distortions compared to MultiTalk - ​​Lip Accuracy​​ – Achieves superior lip synchronization to MultiTalk ## Community Works - [Wan2GP](https://github.com/deepbeepmeep/Wan2GP/): Thanks [deepbeepmeep](https://github.com/deepbeepmeep) for integrating InfiniteTalk in Wan2GP that is optimized for low VRAM and offers many video edtiting option and other models (MMaudio support, Qwen Image Edit, ...). - [ComfyUI](https://github.com/kijai/ComfyUI-WanVideoWrapper): Thanks for the comfyui support of [kijai](https://github.com/kijai). ## Todo List - [x] Release the technical report - [x] Inference - [x] Checkpoints - [x] Multi-GPU Inference - [ ] Inference acceleration - [x] TeaCache - [x] int8 quantization - [ ] LCM distillation - [ ] Sparse Attention - [x] Run with very low VRAM - [x] Gradio demo - [x] ComfyUI ## Video Demos ### Video-to-video (HQ videos can be found on [Google Drive](https://drive.google.com/drive/folders/1BNrH6GJZ2Wt5gBuNLmfXZ6kpqb9xFPjU?usp=sharing) ) ### Image-to-video ## Quick Start ### ️Installation #### 1. Create a conda environment and install pytorch, xformers ``` conda create -n multitalk python=3.10 conda activate multitalk pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu121 pip install -U xformers==0.0.28 --index-url https://download.pytorch.org/whl/cu121 ``` #### 2. Flash-attn installation: ``` pip install misaki[en] pip install ninja pip install psutil pip install packaging pip install wheel pip install flash_attn==2.7.4.post1 ``` #### 3. Other dependencies ``` pip install -r requirements.txt conda install -c conda-forge librosa ``` #### 4. FFmeg installation ``` conda install -c conda-forge ffmpeg ``` or ``` sudo yum install ffmpeg ffmpeg-devel ``` ### Model Preparation #### 1. Model Download | Models | Download Link | Notes | | --------------|-------------------------------------------------------------------------------|-------------------------------| | Wan2.1-I2V-14B-480P | [Huggingface](https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-480P) | Base model | chinese-wav2vec2-base | [Huggingface](https://huggingface.co/TencentGameMate/chinese-wav2vec2-base) | Audio encoder | MeiGen-InfiniteTalk | [Huggingface](https://huggingface.co/MeiGen-AI/InfiniteTalk) | Our audio condition weights Download models using huggingface-cli: ``` sh huggingface-cli download Wan-AI/Wan2.1-I2V-14B-480P --local-dir ./weights/Wan2.1-I2V-14B-480P huggingface-cli download TencentGameMate/chinese-wav2vec2-base --local-dir ./weights/chinese-wav2vec2-base huggingface-cli download TencentGameMate/chinese-wav2vec2-base model.safetensors --revision refs/pr/1 --local-dir ./weights/chinese-wav2vec2-base huggingface-cli download MeiGen-AI/InfiniteTalk --local-dir ./weights/InfiniteTalk ``` ### Quick Inference Our model is compatible with both 480P and 720P resolutions. > Some tips > - Lip synchronization accuracy:​​ Audio CFG works optimally between 3–5. Increase the audio CFG value for better synchronization. > - FusionX： While it enables faster inference and higher quality, FusionX LoRA exacerbates color shift over 1 minute and reduces ID preservation in videos. > - V2V generation: Enables unlimited length generation. The model mimics the original video's camera movement, though not identically. Using SDEdit improves camera movement accuracy significantly but introduces color shift and is best suited for short clips. Improvements for long video camera control are planned. > - I2V generation: Generates good results from a single image for up to 1 minute. Beyond 1 minute, color shifts become more pronounced. One trick for the high-quailty generation beyond 1 min is to copy the image to a video by translating or zooming in the image. Here is a script to [convert image to video](https://github.com/MeiGen-AI/InfiniteTalk/blob/main/tools/convert_img_to_video.py). > - Quantization model: If your inference process is killed due to insufficient memory, we suggest using the quantization model, which can help **reduce memory usage**. #### Usage of InfiniteTalk ``` … ``` #### 1. Inference ##### 1) Run with single GPU ``` python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir 'weights/chinese-wav2vec2-base' \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res ``` ##### 2) Run with 720P If you want run with 720P, set `--size infinitetalk-720`: ``` python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir 'weights/chinese-wav2vec2-base' \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-720 \ --sample_steps 40 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_720p ``` ##### 3) Run with very low VRAM If you want run with very low VRAM, set `--num_persistent_param_in_dit 0`: ``` python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir 'weights/chinese-wav2vec2-base' \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --num_persistent_param_in_dit 0 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_lowvram ``` ##### 4) Multi-GPU inference ``` GPU_NUM=8 torchrun --nproc_per_node=$GPU_NUM --standalone generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir 'weights/chinese-wav2vec2-base' \ --infinitetalk_dir weights/InfiniteTalk/single/infinitetalk.safetensors \ --dit_fsdp --t5_fsdp \ --ulysses_size=$GPU_NUM \ --input_json examples/single_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_multigpu ``` ##### 5) Multi-Person animation ``` python generate_infinitetalk.py \ --ckpt_dir weights/Wan2.1-I2V-14B-480P \ --wav2vec_dir 'weights/chinese-wav2vec2-base' \ --infinitetalk_dir weights/InfiniteTalk/multi/infinitetalk.safetensors \ --input_json examples/multi_example_image.json \ --size infinitetalk-480 \ --sample_steps 40 \ --num_persistent_param_in_dit 0 \ --mode streaming \ --motion_frame 9 \ --save_file infinitetalk_res_multiperson ``` #### 2. Run with FusioniX or Lightx2v(Require only 4~8 steps) [FusioniX](https://huggingface.co/vrgamedevgirl84/Wan14BT2VFusion