I think chatterbox in tts crash after first generation

[asaniom]

The issue is that every time I generate audio using the Chatterbox extension, it works fine the first time. However, after that, it stops functioning.

For example:
Clicking the “Generate” button does nothing—neither in the UI nor in the console log.
or
Refreshing the browser results in no UI appearing.

{'text': 'Chatterbox is an expressive text-to-speech model with reference audio support for voice cloning.', 'exaggeration': 0.5, 'cfg_weight': 0.5, 'temperature': 0.8, 'audio_prompt_path': None, 'seed': '4030282067', 'device': 'auto', 'dtype': 'float32', 'model_name': 'just_a_placeholder', 'chunked': False, 'cpu_offload': False, 'cache_voice': False, 'desired_length': 200, 'max_length': 300, 'halve_first_chunk': False, 'initial_forward_pass_backend': 'eager', 'generate_token_backend': 'cudagraphs-manual', 'max_new_tokens': 1000, 'max_cache_len': 1500}
Generating: '''Chatterbox is an expressive text-to-speech model w...'''
Chatterbox(
exaggeration=0.5,
cfg_weight=0.5,
temperature=0.8,
audio_prompt_path=None,
seed='4030282067',
device='auto',
dtype='float32',
model_name='just_a_placeholder',
chunked=False,
cpu_offload=False,
cache_voice=False,
desired_length=200,
max_length=300,
halve_first_chunk=False,
initial_forward_pass_backend='eager',
generate_token_backend='cudagraphs-manual',
max_new_tokens=1000,
max_cache_len=1500
)
Using device: cuda
Loading model 'Chatterbox on cuda with torch.float32'...
G:\Project\Python\webuitts\installer_files\env\lib\site-packages\diffusers\models\lora.py:393: FutureWarning: LoRACompatibleLinear is deprecated and will be removed in version 1.0.0. Use of LoRACompatibleLinear is deprecated. Please switch to PEFT backend by installing PEFT: pip install peft.
deprecate("LoRACompatibleLinear", "1.0.0", deprecation_message)
loaded PerthNet (Implicit) at step 250,000
Moving model to cuda, torch.float32
Generating chunk: Chatterbox is an expressive text-to-speech model with reference audio support for voice cloning.
Estimated token count: 130
Input embeds shape before padding: torch.Size([2, 101, 1024])
Sampling: 0%| | 0/1000 [00:00<?, ?it/s]Capturing CUDA graph for bucket 250 (max_position: 250)
Sampling: 14%|█████████▋ | 140/1000 [00:03<00:19, 44.38it/s]
Generated in 31.108 seconds
Saving generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.wav
Saving metadata to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.json
Average execution time: 31.129
Saving waveform plot to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.png
Saving generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.ogg
Saved generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.ogg
Saving generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.flac
Saved generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.flac

[rsxdalv]

Thanks for the report. Do you change the dtype between generations? I see that it's float 32 while the default is bfloat16. I'm not sure if it will handle the change correctly when using cudagraphs manual. I have seen a few cases where gradio and the server froze altogether but I thought that I fixed it.

…

On Fri, Aug 29, 2025, 05:05 asaniom ***@***.***> wrote: *asaniom* created an issue (rsxdalv/TTS-WebUI#558) <#558> The issue is that every time I generate audio using the Chatterbox extension, it works fine the first time. However, after that, it stops functioning. For example: Clicking the “Generate” button does nothing—neither in the UI nor in the console log. or Refreshing the browser results in no UI appearing. {'text': 'Chatterbox is an expressive text-to-speech model with reference audio support for voice cloning.', 'exaggeration': 0.5, 'cfg_weight': 0.5, 'temperature': 0.8, 'audio_prompt_path': None, 'seed': '4030282067', 'device': 'auto', 'dtype': 'float32', 'model_name': 'just_a_placeholder', 'chunked': False, 'cpu_offload': False, 'cache_voice': False, 'desired_length': 200, 'max_length': 300, 'halve_first_chunk': False, 'initial_forward_pass_backend': 'eager', 'generate_token_backend': 'cudagraphs-manual', 'max_new_tokens': 1000, 'max_cache_len': 1500} Generating: '''Chatterbox is an expressive text-to-speech model w...''' Chatterbox( exaggeration=0.5, cfg_weight=0.5, temperature=0.8, audio_prompt_path=None, seed='4030282067', device='auto', dtype='float32', model_name='just_a_placeholder', chunked=False, cpu_offload=False, cache_voice=False, desired_length=200, max_length=300, halve_first_chunk=False, initial_forward_pass_backend='eager', generate_token_backend='cudagraphs-manual', max_new_tokens=1000, max_cache_len=1500 ) Using device: cuda Loading model 'Chatterbox on cuda with torch.float32'... G:\Project\Python\webuitts\installer_files\env\lib\site-packages\diffusers\models\lora.py:393: FutureWarning: LoRACompatibleLinear is deprecated and will be removed in version 1.0.0. Use of LoRACompatibleLinear is deprecated. Please switch to PEFT backend by installing PEFT: pip install peft. deprecate("LoRACompatibleLinear", "1.0.0", deprecation_message) loaded PerthNet (Implicit) at step 250,000 Moving model to cuda, torch.float32 Generating chunk: Chatterbox is an expressive text-to-speech model with reference audio support for voice cloning. Estimated token count: 130 Input embeds shape before padding: torch.Size([2, 101, 1024]) Sampling: 0%| | 0/1000 [00:00<?, ?it/s]Capturing CUDA graph for bucket 250 (max_position: 250) Sampling: 14%|█████████▋ | 140/1000 [00:03<00:19, 44.38it/s] Generated in 31.108 seconds Saving generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.wav Saving metadata to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.json Average execution time: 31.129 Saving waveform plot to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.png Saving generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.ogg Saved generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.ogg Saving generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.flac Saved generation to outputs\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a\2025-08-29_05-27-36__chatterbox__Chatterbox_is_a.flac — Reply to this email directly, view it on GitHub <#558>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTRXI6MGPIYZN2JUTYMHNT3P6YM3AVCNFSM6AAAAACFDEXN3OVHI2DSMVQWIX3LMV43ASLTON2WKOZTGM3DKMJYGY4TIMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[asaniom]

I've tried it both ways, but the program freezes regardless of whether I change the dtype. Is there any way to fix this?

Thanks for the report. Do you change the dtype between generations? I see
that it's float 32 while the default is bfloat16. I'm not sure if it will
handle the change correctly when using cudagraphs manual. I have seen a few
cases where gradio and the server froze altogether but I thought that I
fixed it.
…

[rsxdalv]

What's your system? I'm not sure what's causing the issue. What if you try a different option than cudagraphs-manual?

…

On Sat, Aug 30, 2025, 19:55 asaniom ***@***.***> wrote: *asaniom* left a comment (rsxdalv/TTS-WebUI#558) <#558 (comment)> I've tried it both ways, but the program freezes regardless of whether I change the dtype. Is there any way to fix this? Thanks for the report. Do you change the dtype between generations? I see that it's float 32 while the default is bfloat16. I'm not sure if it will handle the change correctly when using cudagraphs manual. I have seen a few cases where gradio and the server froze altogether but I thought that I fixed it. … <#m_9212378291182592966_> — Reply to this email directly, view it on GitHub <#558 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTRXI53IEYA5ASVM2K4AET3QHJPVAVCNFSM6AAAAACFDEXN3OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMZZGQYDAOBUGE> . You are receiving this because you commented.Message ID: ***@***.***>

[asaniom]

amd 3900x- rtx 3060 ti - 16gb ram - Widnows 11

[rsxdalv]

8gb VRAM is enough, 16 GB Ram too. 3060 is good and should run best with bfloat16.
What about the other options instead of cudagraphs-manual?

[asaniom]

i tried eager and it was ok
so i guess cudagraphs-manual was the problem

[rsxdalv]

You can try to use cudagraphs (not manual) and inductor as well. Those might give you decent performance too. If cudagraphs-manual has this stability problem, I might release a few speedups for the other ones. It would be nice to understand the core issue, but it requires a lot of debugging.