Support for 5D latents (video models) and scheduling #62
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Allow scheduling the Tiled Diffusion effect Other minor cleanups
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main thing this pull adds is support for 5D latents so Tiled Diffusion can be used with video models and the ability to schedule the Tiled Diffusion effect.
Some refactoring was necessary to make applying settings function in a relatively sane way. Some of it, I just couldn't help myself. :) I tried to preserve the existing code style as much as possible.
One change I probably should explain is instead of having to track existing TiledDiffusion instances with a WeakSet, one can just keep a sequence number and update it whenever the instances need to reset. That way, instances can just save the sequence number from the last reset (or when they're created) and know to reset themselves when it changes.
These changes are lightly tested, as far as I can tell they don't break existing functionality. There is one case where they would break existing workflows: when a user is applying SpotDiffusion parameters before the TiledDiffusion model patch is applied. These changes require applying parameters after TiledDiffusion.
This pull also adds a TiledDiffusion Advanced Parameters node that currently allows configuring the start/end sigmas for the Tiled Diffusion effect. Adding other options, like allowing the user to specify a list of bboxes would be possible in the future. It would be pretty nice to be able to use custom bboxes instead of a fixed-size grid.
I'd like to add the ability to do temporal tiling but I figured I would make that a separate pull to keep these changes relatively manageable.
Small example of what this can enable:
AnimateDiff_00161.mp4