Animating masks in 2d and 3d modes

[Funofabot]

I recently implemented animating masks similarly to the notebook. I don't know how effective it is. For example when zooming, a centered mask will just become the entire screen. Or when a mask moves off the screen when animating, the mask does not retain its information if it is moved back into the screen.

This is one area that I am unsure how it would be useful for others.

If the mask stops diffusion at borders when animating, the borders will suffer from not diffusing causing stripes or other artefacts. An additional area can be added to the existing mask when animating, that would diffuse the borders that are effected, however currently that is not implemented.

What thoughts do others have? Please enlighten me.
Is this something that we really care about?

[Zarxrax]

I've not used the notebook to try this feature, but I assume it just follows your animation parameters. Given the large motions in a typical deforum animation, I'm imagining that the mask would typically end up off-screen very quickly in most cases, right? It seems to me like animating the mask in external software would be the idea way to go in almost all cases. Are there any examples anyone has made to put this feature to good use?

[Funofabot]

I've not used the notebook to try this feature, but I assume it just follows your animation parameters. Given the large motions in a typical deforum animation, I'm imagining that the mask would typically end up off-screen very quickly in most cases, right? It seems to me like animating the mask in external software would be the idea way to go in almost all cases. Are there any examples anyone has made to put this feature to good use?

When I implemented the way the notebook did it, the mask wasn't changing at all. So maybe the notebook implementation is not complete. But I came up with what I thought was intended. There are other notebook syncs that I need to work on in the end of the day so this one can sit and fester.

[Funofabot]

It was one of the things that I was attempting to sync, however the results are not that great over a long animation.

example with a fixed seed, that will get cartooney from overfitting. But notice the mask, If I change it to zoom in, the mask will not zoom out after it reaches the edges..

[Funofabot]

@kabachuha Remind me about this catch later plz.
This experiment led to one good catch, if the mask is static, we dont need to load it every frame =). So I will incorporate that change in an upcoming PR.