Text-Only Inference Report for Upstream

Summary

We attempted to run VibeVoice-1.5B for text-to-speech locally (without SGLang) using the published vibevoice pip package. The model loads correctly and the tokenizer works after a minor patch, but the inference pipeline cannot produce audio without SGLang — the generate() method's speech-processing path is never initialized for text-only input, and speech_outputs returns [None].

This report documents the full investigation, the specific code paths involved, and two concrete suggestions for enabling standalone text-to-speech.

---

What We Tried

We used the VibeVoiceForConditionalGenerationInference class from vibevoice.modular.modeling_vibevoice_inference with direct imports (bypassing AutoConfig, since vibevoice is not in CONFIG_MAPPING). We used the Qwen2 tokenizer vocabulary files since the VibeVoice HF repo contains no tokenizer config.

Step 1: Model loading — WORKS

from vibevoice.modular.configuration_vibevoice import VibeVoiceConfig
from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference

cfg = VibeVoiceConfig.from_pretrained("microsoft/VibeVoice-1.5B")
model = VibeVoiceForConditionalGenerationInference.from_pretrained(
    "microsoft/VibeVoice-1.5B", config=cfg,
    device_map="cuda", torch_dtype=torch.bfloat16)

Result: Model loads in 3-8 seconds, 5.4 GB VRAM on an RTX 5060 Ti. No state dict errors, no OOM.

Step 2: Tokenizer — WORKS (with patch)

The VibeVoiceTextTokenizer.__init__() passes add_special_tokens=True to PreTrainedTokenizerBase.__init__(). In transformers 4.51.3, add_special_tokens is both a constructor parameter and a method on the base class — causing an AttributeError collision.

Workaround:

import transformers.tokenization_utils_base as tub
_orig_init = tub.PreTrainedTokenizerBase.__init__
def _patched_init(self, **kwargs):
    kwargs.pop("add_special_tokens", None)
    return _orig_init(self, **kwargs)
tub.PreTrainedTokenizerBase.__init__ = _patched_init

Since the HF repo has no tokenizer files, we used Qwen2's vocabulary:

vocab_file = hf_hub_download("Qwen/Qwen2-0.5B", "vocab.json")
merges_file = hf_hub_download("Qwen/Qwen2-0.5B", "merges.txt")
tok = VibeVoiceTextTokenizer(vocab_file=vocab_file, merges_file=merges_file)

Result: Tokenizer constructs and tokenizes correctly.

Step 3: Inference — FAILS (two issues)