Allow unknown schedule-free optimizers to continue to module loader

[rockerBOO]

To support ProdigyPlusScheduleFree

pip install git+http://github.com/LoganBooker/prodigy-plus-schedule-free

--optimizer_type prodigyplus.ProdigyPlusScheduleFree

                    INFO     use prodigyplus.ProdigyPlusScheduleFree | {'weight_decay': 0.1}                                         train_util.py:4111
[Prodigy+ScheduleFree] Optimiser contains single param_group -- 'split_groups' has been disabled.

Related #1796

[kohya-ss]

Thank you for this! And I'm very sorry, but for the schedule-free optimizer, I received a new PR #1600, so I merged it into the sd3 branch (and refactored in #1605). Therefore, I plan to delete this branch in the future. Sorry for the confusion.

I think can make this fix to the sd3 branch myself, however I would be very grateful if you could make the same fix to the sd3 branch.

[rockerBOO]

No problem @kohya-ss, I'll make a new PR against the SD3 branch.

[FurkanGozukara]

@rockerBOO i did trainings with prodigyplus.ProdigyPlusScheduleFree yesterday but it didnt learn anything

i must be missing optimizer arguments can you give me solid example? thank you

i trained up to 5600 steps and another model 2800 steps