Is there a way to speed this realesrgan upscale up? #1253
Description
Using the example command from the --help gives a framerate of approx 0.4 fps. A 4k upscale on my M1Pro Mac for 60m of tv footage in Topaz video AI takes approx 12 hours and video2x is estimating ~40 hours. I had assumed upscaling with an Nvidia GPU would be orders of magnitude faster.
Command used:
video2x -d 0 -i input.mkv -o output.mkv -s 4 -p realesrgan --realesrgan-model realesrgan-plus -c libx264rgb -e crf=17 -e preset=veryfast -e tune=film
Here's the output I am getting:
video2x -d 0 -i input.mkv -o output.mkv -s 4 -p realesrgan --realesrgan-model realesrgan-plus -c libx264rgb -e crf=17 -e preset=veryfast -e tune=film
[2024-12-13 11:40:44.705] [info] Video2X version 6.2.0
[2024-12-13 11:40:44.705] [info] Processing file: input.mkv
[2024-12-13 11:40:45.623] [info] Using Vulkan device: NVIDIA GeForce RTX 3090 (0x2204)
[2024-12-13 11:40:45.624] [info] Press [space] to pause/resume, [q] to abort.
[libx264 @ 0x7f3a081b2b00] using SAR=1/1
[libx264 @ 0x7f3a081b2b00] DPB size (16 frames, 921600 mbs) > level limit (12 frames, 696320 mbs)
[libx264 @ 0x7f3a081b2b00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f3a081b2b00] profile High, level 6.2, 4:2:0, 8-bit
[libx264 @ 0x7f3a081b2b00] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:-1:-1 analyse=0x3:0x133 me=umh subme=10 psy=1 psy_rd=1.00:0.15 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-3 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=60 rc=crf mbtree=1 crf=17.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[0 NVIDIA GeForce RTX 3090] queueC=2[8] queueG=0[16] queueT=1[2]
[0 NVIDIA GeForce RTX 3090] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0
[0 NVIDIA GeForce RTX 3090] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 3090] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1
[1 llvmpipe (LLVM 15.0.7, 256 bits)] queueC=0[1] queueG=0[1] queueT=0[1]
[1 llvmpipe (LLVM 15.0.7, 256 bits)] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0
[1 llvmpipe (LLVM 15.0.7, 256 bits)] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)] subgroup=8 basic=1 vote=1 ballot=1 shuffle=1
[0 NVIDIA GeForce RTX 3090] queueC=2[8] queueG=0[16] queueT=1[2]
[0 NVIDIA GeForce RTX 3090] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0
[0 NVIDIA GeForce RTX 3090] fp16-p/s/u/a=1/1/1/1 int8-p/s/u/a=1/1/1/1
[0 NVIDIA GeForce RTX 3090] subgroup=32 basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 3090] fp16-8x8x16/16x8x8/16x8x16/16x16x16=0/1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)] queueC=0[1] queueG=0[1] queueT=0[1]
[1 llvmpipe (LLVM 15.0.7, 256 bits)] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0
[1 llvmpipe (LLVM 15.0.7, 256 bits)] fp16-p/s/u/a=1/1/1/1 int8-p/s/u/a=1/1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)] subgroup=8 basic/vote/ballot/shuffle=1/1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)] fp16-8x8x16/16x8x8/16x8x16/16x16x16=0/0/0/0
[2024-12-13 11:40:48.537] [warning] Estimating the total number of frames using duration * fps
frame=38/72609 (0.05%); fps=0.46; elapsed=00:01:23; remaining=44:01:50
I'm using Nvidia's beta 1.4 Vulkan driver on Ubuntu 22.04 and --list-devices shows the GPU as device 0. I got what I could from the --help info and tried a preset of 'veryfast' but the framerate is still very low. Video2x version is the latest (6.2). Are there some other command line args I can try which will speed this up? Thanks!