Spatial TTA, Temporal TTA and UHD mode

[kikoferrer]

What do these options do? Is there a documentation anywhere that explains them in RIFE application? Thanks

[lightSpeedBubble]

• TTA stands for Test-Time Augmentation. Quoting RIFE's research paper:

  Model Scaling. To scale our models that can be compared with existing methods, we introduce two modifications following: test-time augmentation and resolution multiplying. 1) We flip the input images horizontally and vertically to get augmented test data. We infer and average (with flipping) these two results finally. This model is denoted as RIFE-2T.

  In my tests, enabling both spatial and temporal TTA yielded better results but I don't know if it does so in all situations.

• As for UHD:
  in RIFE's code, there is this line:

  '--UHD', dest='UHD', action='store_true', help='support 4k video'
  

  So it's something to do with 4K Ultra High Definition.
  I tested it on both 1920x1080 and 3840x2160 image sequences and it made no difference. I assume it's only relevant if your input or output is in a UHD video format but idk what it does exactly, because if it's just a matter of getting RIFE to work at all, then it should be baked in the code, not as an option.

[TNTwise]

TTA calculates warp different ways and merges them. UHD saves on processing power by inferencing at a lower flow resolution.

[lightSpeedBubble]

Ultra High Definition = lower flow resolution, got it, makes perfect sense 👍️😋 (just kidding, as long as it's explained in RIFE's documentation everything's fi... oh wait...)
PS: RIFE's repo points to a DeepWiki which made me discover what Deepwiki is and it's crazy, and yeah if you search for "UHD" there it explains what it does fairly well by analyzing the code. Also it's just setting a scale to 0.5 so I guess it has nothing to do with UHD? Like you can probably use it to halve the flow resolution on any input resolution. They certainly named it this way bc you don't need it on lower res footage but it makes the option a bit obscure. Anyway, outside of the scope of this repo.