Proposal: Native support for bfloat16 / float16 inference (VRAM and speed optimization)

Hi. First of all, congratulations on the amazing work with Chatterbox; the results are fantastic. I am using it intensively to generate complete audiobooks from PDFs through a custom script that processes the text in batches using an RTX 5080.

I'm writing to leave a suggestion for future updates: adding native support or an official parameter to run inference in half-precision (bfloat16 or float16).

I tried loading the modules (T3, S3Gen, VoiceEncoder) in bfloat16 using dtype=torch.bfloat16 to optimize the process, but currently, the generation pipeline breaks due to tensor mismatch errors (internally there are operations or inputs, such as the reference audio, that strictly require float32).

Implementing support for half-precision (whether by converting the inputs under the hood or using torch.autocast) would bring massive advantages to the community:

Drastic VRAM reduction: The model would take up half the memory. This would allow users with more modest GPUs (8 GB or less) to run Chatterbox much more comfortably and without crashes.

Massive batching: For those of us with more VRAM (16 GB or more), that freed-up memory would allow us to significantly increase the batch size, multiplying the generation speed when processing long texts.

Tensor Core acceleration: On modern NVIDIA architectures (Ampere, Ada, etc.), operating in 16-bit would finally squeeze the full potential out of the Tensor Cores, radically reducing inference times without any perceptible drop in voice quality.

I think this would be a huge performance leap for the project. Thank you very much for your time and for sharing this model with the community!