Ahead-of-time API requests to improve overvall performance? #448
-
|
Currently, by configuring Read Aloud to use OpenAI compatible API endpoints, I'm able to generate sound by using my own https://github.com/remsky/Kokoro-FastAPI instance. But because of limit of hardware power, it must be wait a while before each paragraph is read in turn. Is there a way that while reading this paragraph, the other paragraphs after that will be requested first and everything can be seamless? Is it possible to set some interval for that.? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
All the engines are already doing prefetch, i. e. synthesizing the next section while the current one is being read. You can look in tts-engines.js However, for this to work, the inference must be faster than realtime (realtime factor < 1) . Otherwise the next section wont finish synthesizing before the current one finishes playing. Kokoro is quite demanding, unfortunately. See discussion in ken107/piper-browser-extension#10 |
Beta Was this translation helpful? Give feedback.
All the engines are already doing prefetch, i. e. synthesizing the next section while the current one is being read. You can look in tts-engines.js
However, for this to work, the inference must be faster than realtime (realtime factor < 1) . Otherwise the next section wont finish synthesizing before the current one finishes playing.
Kokoro is quite demanding, unfortunately. See discussion in ken107/piper-browser-extension#10