ControlNet video

[andyxr]

Hey all,
Has anyone figured out how to use ControlNet within Deforum, i.e video to video?
Do you use Video Input mode, or 2D/3D?
What's the difference between ControlNet video input and ControlNet video mask input?
If anyone has a rough guide, I'd greatly appreciate it. It's kinda breaking my brain trying to figure out how it's supposed to work...!

A

[kabachuha]

It works in all the modes: 2D, 3D, video input. I haven't yet tested ControlNet masks, I suppose they're just limiting the scope of CN-guidance to the region, so before that just put your source images into CN video input.

I recently rendered this video with the help of Deforum and ControlNet, it has the workflow and the settings included in the description

https://youtu.be/U9lRAyHwBiQ

[andyxr]

Thanks @kabachuha . Appreciate the response. I guess I'm trying to get my head around using Video Input mode, and using a video in the CN tab. I'm assuming that the CN video guides the video specified in Video Input mode. I'll play. Thanks again.

[Andie-Squirrel]

Hi andyxr, I am interested in using the Video Input mode but I do not know how to use relative paths. Any input is really appreciated.