Emotions, non speech sounds and training?
Hey there!
I've been playing with this tool for voice cloning purposes and the first thing I want to say is that I'm really impressed by the results. I've played around with voice cloning tools for sometime and this is one of the best I've seen so far, without any training or finetuning on top of that, so congrats!
I have three "issues" though, or maybe wishes for improvement.
The first one is that I find the emotions very hard to use, and they have barely an effect on the output. I saw in this post that one of the users had exposed more of the parameters, like the temperature, to the interface. Yet the last comment on this thread is that this development is obsolete, but still I don't see these parameters in the main branch. Are they implemented elsewhere? Would these parameters help with generating the right emotions and with enough intensity?
The second issue is this one : in all the voice cloning projects I use, I need to be able to generate non speech sounds : sighs, laughs, hesitation, shushing, breath intake....All that sort of thing. In MRQ ai voice cloning repo, forked by Jarod Mica, I've been able to train part of these sounds to very good results (still struggling with laughter though but technically it's possible). In Zonos, the non speech sounds are not recognized at all and give bad results if I write them "phonetically", like "Shh!" will output "Shoo!". I was wondering if there was a way to generate these sounds correctly, or maybe if there was a way to train them?
The second question opens the third : is there any way to finetune the model? What would the requirements be?
Anyway this is clearly a very promising tool, thank you very much for sharing it under Apache, it is clearly worth the results of Eleven Labs on a lot of aspects.
Source: Zyphra/Zonos