Low steps increase when computing strength

[Danamir]

With the recent adding of Qwen-Image, I often find myself limited by the 4-steps SQDV model or Ligthning LoRA when doing refining. With such a low step count, the only refining strengths available steps are 100, 75, 50, 25 ; which are not enough for some fine refining adjustment.

To fix that you can switch to a 8-steps preset and have a strength range of 100, 88, 75, 62, 50, 38, 25, 12 , but now in many cases the number of steps when refining is bigger than the first 4 steps and we now have a slower refining than the original render !

Another way is to keep the 4-steps model or LoRA but with a 5 steps preset. It works pretty well an give an alternative strengths range of 100, 80, 60, 40, 20 ; and the low steps count keeps things fast.

Proposed change :

This PR is a proof of concept of adding a new variable when computing strength and snapping to percent to allow a dynamic total steps increase to add more available strengths to the valid range.

In this version I fixed the values so that when the target number of steps is 4 or less, we allow a 1 step increase when computing strength. The new dynamic strength range is then : 100, 80, 75, 60, 50, 40, 25, 20 ; which is not linear but still useful I think.

Thoughts ?

NB : I also tied with a 2 steps increase, but using a target of 6 steps with a 4-steps Lightning can produce very noisy results, so it's better to keep the increase low. [Edit]: Don't pay attention to this, I think I made a setup error when testing.

[Danamir]

Note : When on 4 steps, it's a bad idea to decrease the total number of steps to 3 or less because it produce bad results. But arguably this could be used with 8-steps Lightning to go as low as 6 steps to add even more available strengths.

[Danamir]

There, I changed the steps increase into a tuple to have more control over it. I added some test increase/decrease strength values range for 4-steps, 5-steps, and 8-steps.

[Danamir]

Can't find good settings for 8-steps, but anyway it was not that useful for those.

I made a change to 4-steps to change it internally to 5 steps with -1, 0 ,+1 dynamic steps. It feels right, and only the uppermost strength is actually using more steps than the original rendering.

The dynamic strengths range is now smoother : 83.3, 80, 75, 66.6, 60, 50, 40, 33.3, 25, 20, 16.6 .

[edit] : damn I'm dumb, it's exactly as using a 0, +1, +2 steps on 4-steps like I mentioned in the first post. But this time I have no bad results. I think I made an error in the first post and was wrongly using a 8-steps preset.

I'd have liked more strengths around 50%, but I'm not sure there is a combination of steps allowing this without going out of the 4-steps Lightning range.

[Acly]

Hm I'm a bit hesitant to add further complexity to the strength/step comutation. From reading the code I don't really have any kind of intuition what happens with the steps anymore.

Losing flexibility to fine adjust strength with low total steps is an issue, I've always preferred to go with 8ish step distills where possible as they are less "baked". That being said, minimum steps was added to also deal with this problem, and it works reasonably well for me.

Minimum steps = 4 you scale to 4-of-5, 4-of-6, 4-of-8 which seems to give decent results with Qwen image lighting 4step merge.
Minimum steps = 3 is better for a low-step model like this. It results in cut-offs at 35% -> 41% -> 49% -> 58% -> 75% -> 100% all of which give meaningful difference without looking overbaked.

I feel like noise schedules that assume >4 steps are not actually a big problem for this model, only actually performing >4 steps makes it deteriorate.

[Danamir]

My problem with the minimum steps = 3 setting, and why I added a preset without it in the first place, it that it increases the total steps too much on low denoise, and producing noisy results when using a Nunchaku SVDQ model merged with 4-steps Lightning.

This problem does not affect the normal of GGUF models + Lightning LoRA. On this setup the minimum 3-steps is perfectly suitable. But on my system it's the difference between a 25-40% strength refining to denoise an image taking 18s for 3 steps vs. taking 2s to 6s for 1-3 steps...

Here are the various options available for now when refining an image with Qwen-Image Nunchaku Lightning 4-steps. In each grid, the first image is the original, and the last the 100% Qwen generated image. In-between, the various strength settings.

The current "default", 4 steps total, 1 minimum step :

Rendered steps	Total steps	Strength	Notes
1	4	25%	Can be used as a "cleanup" pass on an image. Better eyes, fingers...
2	4	50%	Same, with a stronger effect. The style starts to be altered.
3	4	75%	Mix between Qwen-style and the old style

The proposed PR, 4 steps total, 1 minimum step, +0/+1/+2 dynamic max steps :

Rendered steps	Total steps	Strength	Notes
1	6	17%	Little bit noisy because of the 6 steps, don't intend to use this setting.
1	5	20%	Very close to the original, low cleanup pass.
1	4	25%	Nice cleanup.
2	6	33%	Stronger cleanup, but kinda noisy. Again, 6 steps is a little bit much for Nunchaku+Lightning.
2	5	40%	Good improvements, keep most of the style.
2	4	50%	More "improvements", the style starts to change.
3	5	60%	Same but stronger.
4	6	67%	Closer to Qwen "anime" style. The 6 steps are no longer a problem here, very subtle noise.
3	4	75%	The pose changes.
4	5	80%	More.
5	6	83%	And more. Little bit of noise due to the 6 steps, but nothing outrageous.

The old code with 4 steps total, 3 minimum steps :

Rendered steps	Total steps	Strength	Notes
3	12	25%	So... much... noise. 12 steps is wrecking the Nunchaku+Lightning model.
3	10	30%	Still to much noise to be used as a cleanup pass.
3	9	35%	Noisy.
3	8	40%	Noisy.
3	6	45%	Strong cleanup, the style starts to change.
3	6	50%	Exact same as 45% because of the nearest strength calculation.
3	5	55%	More style change, clean.
3	5	60%	Same rendering as 55%...
3	4	75%	Mix between Qwen-style and the old style.

Conclusions

So, when using 3 minimum steps on a 4-steps Nunchaku+Lightning :

• The rendering at lower strength are slower because of the 3 steps rendered where it would only take 1 to 2 steps with the dynamic max steps method.
• Below 40% the results are unusable because the max steps are way above the intended 4.
• Some of the strength changes produce the same exact results (45-50%, 55-60%). But the UI does not behave the same at each time, ie. when starting from 0% or 100%, some times the duplicate strength are not shown, sometimes they are.

I much prefer my method, when using a Nunchaku model. And even on normal models it allows a much greater range of available strengths, at the cost of more code complexity and potentially confusing steps count.

That being said, I'm also happy to keep this dev on my fork for my own usage if you think it's not worth the trouble pulling it in the main repository.

[Danamir]

Good news, the noise at 6 steps on 4-steps Nunchaku Lightning was due to the beta57 scheduler I had in my presets. With the beta scheduler there is no more noise on specific strengths.

It also helps with the "minimum 3 steps" method, but only to a certain extent. Once you get to 30% strength and below, there is still too much noise.

[Acly]

Maybe hardcoding minimum steps into the sampler presets is not that useful. In the end it's usually just 3 or 4, and when the intended number of steps of a preset is changed to fit a low-step model it no longer fits.

I agree the current method stops working at some point when low strength cranks up total number of steps too high. For SDXL low-step this didn't cause quality issues, looks like it's more of an issue for Qwen. Haven't tested Flux.

This PR feels like a pin-point patch for a very specific situation on top of already not very intuitive mechanism. Maybe it's just the way its implemented. I feel it should look a bit simpler, with all the logic in one function. I don't mind changing the minimum steps logic, as long as the new method still works for older architectures too.

[Danamir]

Yeah I think you're right, it still feels clunky.
I'll close this PR for now, until I find a nicer way to handle those low steps denoise.