Issues with get_conditioning_latents.py

[mcbobbo]

Working with get_conditioning_latents.py as it seems to speed things up a bit, and I've noticed three issues. Haven't tested these deeply, just reporting them here in case others want to explore, as I believe I have workarounds for them.

1. It doesn't like new paths in voices. If you put your own files in a path it knows, e.g. 'pat2' it seems happy. Otherwise you get...

Traceback (most recent call last):
  File "C:\tortoise-tts-main\tortoise\get_conditioning_latents.py", line 23, in <module>
    cond_paths = voices[voice]
KeyError: 'newvoice'

2. It doesn't process .wav files, but seems happy with .mp3 files. The latter do throw an error, but they seem to work. Here's the message I get twice:

C:\Users\mcbob\.conda\envs\tortise-tts\lib\site-packages\librosa-0.9.2-py3.9.egg\librosa\util\decorators.py:88: UserWarning: PySoundFile failed. Trying audioread instead.
  return f(*args, **kwargs)

3. Output isn't working as expected. It runs and appears to do nothing. If it's putting files somewhere, I don't see where. This is an acceptable workaround:

--output .\

[MV799]

Ok, you are using anaconda as me and I noticed after putting some print commands here and there that the get_conditioning_latents.py is looking for the voices in the anaconda env copy (yes there is one, created by the install), so for that to work you either need to fix the get_conditioning_latents.py (not sure how) or copy the voices also to your conda env folders .
Also the get_conditioning_latents.py has wrong output default path - it will save the latents one level above your results folder - just remove ..\

[bbecausereasonss]

Can you guys explain what this even does? Just creates a model to use instead of always running a new on on the wav files? What's the proper way to use it syntax wise?

[neonbjb]

If you all fix bugs like this, please submit PRs to help others! I'd be glad to review them.

@bbecausereasonss When you feed the model a voice to duplicate, it "compiles" it into something I call a "conditioning latent". This is is vector of numbers that describes the voice in a way that the Tortoise model understands. It is not a model.

[bbecausereasonss]

Gotcha, so in essence it can save time during inference?

[neonbjb]

Yep. A little tiny bit of time at that.

[bbecausereasonss]

Are you able to fix the .py to ensure it looks for voices in the right folder opposed to the venv?

[FurkanGozukara]

If you all fix bugs like this, please submit PRs to help others! I'd be glad to review them.

@bbecausereasonss When you feed the model a voice to duplicate, it "compiles" it into something I call a "conditioning latent". This is is vector of numbers that describes the voice in a way that the Tortoise model understands. It is not a model.

How much time is good? Can we say more training data is better?