How to generate custom voice embeddings for TTS 1.6B? (mimi_voice model not available)

Due diligence

• I have done my due diligence in trying to find the answer myself.

Topic

Other / All

Question

Hi team,

I'm using Kyutai TTS 1.6B with moshi-server for a voice assistant project. Everything works great with the pre-built voices from kyutai/tts-voices.

I'd like to use a custom voice (from an MP3 recording) as a permanent voice for my assistant. I've tried several approaches but none produce working embeddings:

What I tried:

1. tts_make_voice.py with --hf-repo kyutai/tts-1.6b-en_fr — fails because 1e68beda_240_mimi_voice.safetensors is not in the repo (404).
2. Using standard Mimi encoder (mimi.encode_to_latent(wav, quantize=False)) — produces embeddings with correct shape [1, 512, 125] but wrong value distribution (std≈0.067 vs std≈0.715 for working voices). The output is noise/garbled audio.
3. Voice cloning server (voice-cloning.toml) — references hf://kyutai/unmute-voice-cloning/e9d43d50_500_mimi_voice.safetensors which returns 401/404.

It seems generating correct voice embeddings requires a special _mimi_voice.safetensors model that is not publicly available.

Questions:

1. Is there a supported way to generate custom voice embeddings for TTS 1.6B from audio files?
2. Are there plans to release the _mimi_voice model or the voice cloning server weights?
3. Would you be so nice to close the voice from my recording if other options are not available?

The voice cloning works perfectly on unmute.sh — I'm looking for a way to achieve the same result with a self-hosted setup. Thank you for the amazing work on this project!

Cheers, Almir

sophie_voice.mp3