I have many questions

[instak1ll]

Hello, I have many questions. The first is that I can't generate audios longer than 47 seconds. Moving any parameter, nothing happens.

And the last question is whether there is any video or documentation that, by following it step by step (as if I were stupid), I could get some use out of this. Because I see that many people train it, and from what I understand, this improves the quality of the samples and, above all, modifies the duration. But it turns out that, since I'm "stupid" (my IQ is -100), I'm asking for help so they can teach me how to train this with my music dataset and how to modify the duration or create a continuation of those 47 seconds to create longer audios.

[Taikakim]

It's quite technical if you're not experienced in setting up virtual machines in the cloud, etc. I'm working on an easier solution and instructions for musicians easier to adopt it but it's going to take time. If you want longer samples, you'll need to finetune the model with lots of good data. There are some other ways potentially but they get very technical and I haven't seen a good public implementation.

[Taikakim]

You can try to begin with things like interval_min around 0.1 to 0.2 and max something above 0.8 or so. This might make the model a bit more creative. If you do this, you could use CFG values up to 16 to even 32 maybe. A high CFG will make the model follow your prompt better, and those CFG interval settings limit it a bit, since a too high value will cause glitches. With high CFG values try also setting a CFG Rescale value between 0.25 and 0.75, this might also mitigate the negative effects of a high CFG value.

Sigma_min is good to keep between 0.01 and 0.05 for generic things I've found. 100 steps is fine for most things, and if you find an especially good sample, you can try to crank that up to 300 or so for maybe a slight increase in quality.

Sigma max between 150 and 250 should be good for most purposes.

Some other samplers, like K-Heun might sometimes produce different results.

One thing is also to use init audio samples to guide the model a bit in the direction you want. I havenät actually experimented so much with this, but from the image world I know it helps out to make the outputs less average-sounding.

[sskalnik]

This video by Lyra should cover everything in depth:
https://www.youtube.com/watch?v=ex4OBD_lrds

[sskalnik]

High CFG value = the model will try to adhere more strictly to the guidance provided by your prompt, but this may also produce more artifacts.
Low CFG value = the model will not adhere as strictly to the prompt, meaning that the results might not be quite what you intended, but you might have fewer artifacts. I usually leave the CFG scaling at 1.0 (100%) or whatever the defaults are.

I recommend using DPM++ 2M or 3M for the sampler.
This page is for Stable Diffusion for image generation, but the same concepts apply to stable-audio-tools:
https://stable-diffusion-art.com/samplers/

[sskalnik]

Hope this helps a bit:
https://github.com/sskalnik/stable-audio-tools/blob/main/TRAINING_FOR_NOOBS.md

[Taikakim]

Hi, don't have a lot of time now, but there were a few issues in your documentation:

Listen to the demos and decide when it sounds like the model has learned enough about your dataset, then kill the training process with Ctrl-C.

The user should first check that there is a recent checkpoint

Weight decay = how much the Learning Rate decreases (decays) over time during training. You might have a Learning Rate of 0.01, and a Decay of 0.001, meaning that after each Step the Learning Rate decreases by 0.001: 0.01, 0.099, 0.098, 0.096 ... 0.003, 0.002, 0.001, done!

This is completely wrong. It's a multiplier used when updating the model weights during training.

Learning Rate Optimizers = algorithms for optimizing the Learning Rate. Usually combined with a Learning Rate Scheduler. A typical choice is AdamW. AdamW8bit is a viable option for saving VRAM. Many other options exist, like Lion and Prodigy, but you should stick with AdamW or another Adam derivative unless you want to navigate unexplored territory and perform experiments.

True, but Lion and Prodigy are not supported without code changes unless this has been added recently.

Try to not use extremely large Batch Size values, such as 512, because the model is more likely to learn well from Batch Sizes of about 8 to 32.

Are you sure? IIRC the original was trained at 64 GPUs at a batch size of four for an effective batch size of 256, if I understood the paper correctly.

One thing worth mentioning is that with a CFG of 1.0 the inference will run at double-speed when the classifier-free guidance part is bypassed.

[sskalnik]

Thanks, I'm updating that file now!