v6

[anothermartz]

v6 colab:

Whoops! forgot to put v5 as a release! I shared it with anyone that would've used it anyway. At least I hope I did.

v5 notes:

Waaaay way faster inference time, and quality improvements!

I utelised the optimisations and improvements of wav2lip from this project: https://github.com/devxpy/cog-Wav2Lip

In my short test clip, processing time improved by about 83% compared to v4!
The new tracking method also fixed many visual bugs too!

I removed other upscaling methods than gfpgan because I couldn't find a use case where those were better.
If you disagree, let me know and I'll add them back in.

v6 notes:

Changed the masking from being the whole face to a feathered mask around the mouth, it should look a lot more natural and won't have the fake looking eyes when upscaling from v5.

You can see what the mask is actually covering by using the debug_mask checkbox.

(left: v5, middle: v6 mask_debug, right: v6

Added preview_settings to process only a single frame for checking settings so you don't have to render the entire video to check settings.

Reduced processing time when you process the same video multiple times by saving the face detection process.

Significantly reduced the overall processing time by pre-loading certain things in step 1.

These two factors make it much better to tweak the settings on the same clip, especially if you use preview_settings.

---

This discussion was created from the release v6.