Audio Output Always Contains Rauschen (Noise) in Stable Audio Generation

[PierrunoYT]

[BUG] Audio Output Always Contains Rauschen (Noise) in Stable Audio Generation

Bug Description

When generating audio with Stable Audio, the output always consists entirely of noise ("rauschen" or static). No part of the generated audio contains the expected content based on the input prompt.

Error Manifestation

There is no error message or stack trace. The audio file is generated successfully, but listening to it reveals that the entire output is just noise.

Steps to Reproduce

1. Run the Stable Audio generator.
2. Enter any text prompt.
3. Set the seed to any value (including -1 for random).
4. Click "Generate Audio".
5. Listen to the output: the entire audio is just noise.

Technical Details

This issue occurs with any prompt or seed value and on all tested platforms. No errors are logged during the generation process.

Workaround Applied

No workaround is currently available, as the output is always noise regardless of input or settings.

Suggested Fix

Investigate the audio generation pipeline to determine why the output is always noise. Check model initialization, input handling, and the inference process for possible causes.

Additional Context

This issue is consistent and affects all users and prompts. If needed, I can provide a sample audio file demonstrating the problem.

[zqevans]

Did you properly load the model checkpoint? This sounds like what would happen if you didn't properly load the weights.

[AlexPengis]

Confirm issue. Try finetune several models and can confirm that output sound usually contain some "clipping" noise. Especially it appears when generated sound not so far from source music tracks from train dataset that use full range of loudness ( -0 dB clipping) . It sounds kind of distortion.

[Taikakim]

I see people here now talk about two different issues, but about the latter, IIRC I had some of that problem when I was finetuning SAO with my own loud-ish music, and I think the results were better when I normalised everything to something like -3dB before training.