MOSS-TTSD

MOSS-TTSD

English | 简体中文

Start here: Installation · Quick start · Try the demo · v1.0 model weights · Evaluation

Overview

MOSS-TTSD is the long-form dialogue specialist within our open-source MOSS‑TTS Family. While foundational models typically prioritize high-fidelity single-speaker synthesis, MOSS-TTSD is architected to bridge the gap between isolated audio samples and cohesive, continuous human interaction. The model represents a paradigm shift from "text-to-speech" to "script-to-conversation." By prioritizing the flow and emotional nuances of multi-party engagement, MOSS-TTSD transforms static dialogue scripts into dynamic, expressive oral performances. It is designed to serve as a robust backbone for creators and developers who require a seamless transition between distinct speaker personas without sacrificing narrative continuity. Whether it is capturing the spontaneous energy of a live talk show or the structured complexity of a multilingual drama, MOSS-TTSD provides the stability and expressive depth necessary for professional-grade, long-form content creation in an open-source framework.

Highlights

• From Monologue to Dialogue: Unlike traditional TTS which optimizes for reading, MOSS-TTSD masters the rhythm of conversation. It supports 1 to 5 speakers with flexible control, handling natural turn-taking, overlapping speech patterns, and distinct persona maintenance.
• Extreme Long-Context Modeling: moving beyond short-sentence generation, the model is architected for stability over long durations, supporting up to 60 minutes of coherent audio in a single session with consistent identity.
• Diverse Scenario Adaptation: fine-tuned for high-variability scenarios including conversational media (AI Podcasts), dynamic commentary (Sports/Esports), and entertainment (Audiobooks, Dubbing, and Crosstalk).
• Multilingual & Zero-Shot Capabilities: features state-of-the-art zero-shot voice cloning requiring only short reference audio, with robust cross-lingual performance across major languages including Chinese, English, Japanese, and European languages.

News

• [2026-03-18] We support efficient end-to-end SGLang inference for MOSS-TTSD v1.0.
• [2026-03-06] We added end-to-end SGLang inference support for MOSS-TTSD v0.7. For detailed instructions, please see the legacy v0.7 docs.
• [2026-02-10] MOSS-TTSD v1.0 is released! MOSS-TTSD v1.0 is officially released! This milestone version redefines long-form synthesis with 60-minute single-session context and support for multi-party interactions. It significantly expands multilingual capabilities and diverse usage scenarios.

• [2025-11-01] MOSS-TTSD v0.7 is released! v0.7 significantly improves audio quality, voice cloning capability, and stability, adds support for 32 kHz high‑quality output, greatly extends single‑pass generation length (960s→1700s).
• [2025-09-09] We supported SGLang inference engine to accelerate model inference by up to 16x.
• [2025-08-25] We released the 32khz version of XY-Tokenizer.
• [2025-08-12] We add support for streaming inference in MOSS-TTSD v0.5.
• [2025-07-29] We provide the SiliconFlow API interface and usage examples for MOSS-TTSD v0.5.
• [2025-07-16] We open-source the fine-tuning code for MOSS-TTSD v0.5, supporting full-parameter fine-tuning, LoRA fine-tuning, and multi-node training.
• [2025-07-04] MOSS-TTSD v0.5 is released! v0.5 has enhanced the accuracy of timbre switching, voice cloning capability, and model stability.
• [2025-06-20] MOSS-TTSD v0 is released! Moreover, we provide a podcast generation pipeline named Podever, which can automatically convert PDF, URL, or long text files into high-quality podcasts.

Note: For MOSS-TTSD v0.7 (including end-to-end SGLang inference), please refer to the legacy v0.7 docs for detailed instructions.

Supported Languages

MOSS-TTSD currently supports 20 languages:

Installation

To run MOSS-TTSD, you need to install the required dependencies. You can use pip and conda to set up your environment.

Using conda

conda create -n moss_ttsd python=3.12 -y && conda activate moss_ttsd
pip install -r requirements.txt
pip install flash-attn

Usage

Quick Start

MOSS-TTSD uses a continuation workflow: provide reference audio for each speaker, their transcripts as a prefix, and the dialogue text to generate. The model continues in each speaker's identity.

…

Batch Inference

You can use the provided inference script for batch inference. The script automatically uses all visible GPUs. You can control GPU visibility via export CUDA_VISIBLE_DEVICES=<device_ids>.

python inference.py \
  --model_path OpenMOSS-Team/MOSS-TTSD-v1.0 \
  --codec_model_path OpenMOSS-Team/MOSS-Audio-Tokenizer \
  --input_jsonl /path/to/input.jsonl \
  --save_dir outputs \
  --mode voice_clone_and_continuation \
  --batch_size 1 \
  --text_normalize

Parameters:

• --model_path: Path or HuggingFace model ID for MOSS-TTSD.
• --codec_model_path: Path or HuggingFace model ID for MOSS-Audio-Tokenizer.
• --input_jsonl: Path to the input JSONL file containing dialogue scripts and speaker prompts.
• --save_dir: Directory where the generated audio files will be saved.
• --mode: Inference mode. Choices: generation, continuation, voice_clone, voice_clone_and_continuation. We recommend using voice_clone_and_continuation for the best voice cloning experience.
• --batch_size: Number of samples per batch (default: 1).
• --max_new_tokens: Maximum number of new tokens to generate. Controls total generated audio length (1s ≈ 12.5 tokens).
• --temperature: Sampling temperature (default: 1.1).
• --top_p: Top-p sampling threshold (default: 0.9).
• --top_k: Top-k sampling threshold (default: 50).
• --repetition_penalty: Repetition penalty (default: 1.1).
• --text_normalize: Normalize input text (recommended to always enable).
• --sample_rate_normalize: Resample prompt audios to the lowest sample rate before encoding (recommended when using 2 or more speakers).

JSONL Input Format

The input JSONL file should contain one JSON object per line. MOSS-TTSD supports 1 to 5 speakers per dialogue. Use [S1]–[S5] tags in the text field and provide corresponding prompt_audio_speakerN / prompt_text_speakerN pairs for each speaker:

{
  "base_path": "/path/to/audio/files",
  "text": "[S1]Speaker 1 dialogue[S2]Speaker 2 dialogue[S3]...[S4]...[S5]...",
  "prompt_audio_speaker1": "path/to/speaker1_audio.wav",
  "prompt_text_speaker1": "Reference text for speaker 1 voice cloning",
  "prompt_audio_speaker2": "path/to/speaker2_audio.wav",
  "prompt_text_speaker2": "Reference text for speaker 2 voice cloning",
  "...": "...",
  "prompt_audio_speaker5": "path/to/speaker5_audio.wav",
  "prompt_text_speaker5": "Reference text for speaker 5 voice cloning"
}

Accelerate Inference with SGLang

MOSS-TTSD v1.0 supports running the fused MOSS-TTSD and MOSS-Audio-Tokenizer model with the deeply extended SGLang from OpenMOSS, enabling efficient inference for audio generation.

Environment Setup

First, clone the SGLang branch compatible with MOSS-TTSD v1.0.

git clone https://github.com/OpenMOSS/sglang -b moss-ttsd-v1.0-with-cat

Using venv

python -m venv moss_ttsd_sglang
source moss_ttsd_sglang/bin/activate
pip install ./sglang/python[all]

Using conda

conda create -n moss_ttsd_sglang python=3.12
conda activate moss_ttsd_sglang
pip install ./sglang/python[all]

End-to-End Inference Service

Start the inference server

Before starting the service, first download MOSS-TTSD-v1.0 and MOSS-Audio-Tokenizer.

git clone https://huggingface.co/OpenMOSS-Team/MOSS-TTSD-v1.0
git clone https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer

Or:

hf download OpenMOSS-Team/MOSS-TTSD-v1.0 --local-dir ./MOSS-TTSD-v1.0
hf download OpenMOSS-Team/MOSS-Audio-Tokenizer --local-dir ./MOSS-Audio-Tokenizer

After the download is complete, run the following command to fuse MOSS-TTSD v1.0 and MOSS-Audio-Tokenizer into a single-directory model that can be loaded by SGLang. After fusion, the model uses voice_clone_and_continuation inference mode by default:

python scripts/fuse_moss_tts_delay_with_codec.py \
  --model-path <path-to-moss-ttsd-v1.0> \
  --codec-model-path <path-to-moss-audio-tokenizer> \
  --save-path <path-to-fused-model>

Then start the inference server with:

sglang serve \
  --model-path <path-to-fused-model> \
  --delay-pattern \
  --trust-remote-code \
  --port 30000 --host 0.0.0.0

The first service startup may take longer due to compilation. Once you see The server is fired up and ready to roll!, the service is ready. The first request after startup may still trigger a lengthy compilation, which is expected behavior, so please be patient.

Tip: The end-to-end inference service may cause some VRAM fragmentation during runtime. If GPU memory is tight, we recommend using --mem-fraction-static when starting SGLang to reserve enough space for intermediate tensors.

Send a generation request

The service API is compatible with the standard multimodal text-generation interface. The text field in the returned JSON contains the base64-encoded WAV audio.

The repository currently provides a minimal request example script:

python scripts/request_sglang_generation.py

This script will:

• send requests to http://localhost:30000/generate by default
• use asset/reference_02_s1.wav and asset/reference_02_s2.wav in the repository as reference audio
• save the returned audio to outputs/output.wav

If you need to change the reference audio, input text, sampling parameters, or server URL, you can directly edit the corresponding constants in scripts/request_sglang_generation.py.

Evaluation

Objective Evaluation(TTSD-eval)

We introduce a robust evaluation framework leveraging MMS-FA for word-level alignment and utterance segmentation and wespeaker for embedding extraction to derive Speaker Attribution Accuracy (ACC) and Speaker Similarity (SIM). Please refer to TTSD-eval for the code and data.