Not sure how to specify num frames | No such file or directory error

[gurubac]

I have this code which is trying to interpolate to 480 fps then render to 60 fps. (Ignore the comments they are from the sample code)

mkdir output_frames

# find the source fps and format with ffprobe, for example 24fps, AAC
ffprobe input.mp4

# extract audio
ffmpeg -i input.mp4 -vn -acodec copy audio.m4a

# decode all frames
ffmpeg -i input.mp4 input_frames/frame_%08d.png

# interpolate 2x frame count
./rife-ncnn-vulkan -i input_frames -o output_frames -n 480

# encode interpolated frames in 48fps with audio
ffmpeg -framerate 60 -i output_frames/%08d.png -i audio.m4a -c:a copy -crf 20 -c:v libx264 -pix_fmt yuv420p output.mp4

I keep getting the error
[image2 @ 0x157f05860] Could find no file with path 'output_frames/%08d.png' and index in the range 0-4
output_frames/%08d.png: No such file or directory

If i take out the frames argument it works fine, and does the default of multiplying by 2. I have also tried doing -n 8, which does not work either.

Here is the full log.

➜  rife-ncnn-vulkan-20221029-macos sh run.sh
ffprobe version 6.0 Copyright (c) 2007-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2023-04-20T04:13:28.000000Z
    encoder         : Blackmagic Design DaVinci Resolve
  Duration: 00:00:21.40, start: 0.000000, bitrate: 111037 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 111128 kb/s, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      timecode        : 01:00:00:00
  Stream #0:1[0x2](und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : TimeCodeHandler
      timecode        : 01:00:00:00
Unsupported codec with id 0 for input stream 2
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2023-04-20T04:13:28.000000Z
    encoder         : Blackmagic Design DaVinci Resolve
  Duration: 00:00:21.40, start: 0.000000, bitrate: 111037 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 111128 kb/s, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      timecode        : 01:00:00:00
  Stream #0:1[0x2](und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : TimeCodeHandler
      timecode        : 01:00:00:00
[ipod @ 0x154705d30] track 0: codec frame size is not set
Output #0, ipod, to 'audio.m4a':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=     840kB time=00:00:21.33 bitrate= 322.8kbits/s speed=3.98e+03x
video:0kB audio:836kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.562616%
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2023-04-20T04:13:28.000000Z
    encoder         : Blackmagic Design DaVinci Resolve
  Duration: 00:00:21.40, start: 0.000000, bitrate: 111037 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 111128 kb/s, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      timecode        : 01:00:00:00
  Stream #0:1[0x2](und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : TimeCodeHandler
      timecode        : 01:00:00:00
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> png (native))
Press [q] to stop, [?] for help
[swscaler @ 0x141540000] [swscaler @ 0x110908000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110918000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110928000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110938000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110948000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110958000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110968000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110978000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x141540000] [swscaler @ 0x110988000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x118008000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x1620e0000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x1620f0000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x162100000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x162110000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x162120000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x162130000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x162140000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x110988000] [swscaler @ 0x162150000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118008000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118018000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118028000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118038000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118048000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118058000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118068000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118078000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x118088000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x120360000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x120370000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x120380000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x160450000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x160460000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x160470000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x160480000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x160490000] No accelerated colorspace conversion found from yuv420p to rgb24.
[swscaler @ 0x162150000] [swscaler @ 0x1620e0000] No accelerated colorspace conversion found from yuv420p to rgb24.
Output #0, image2, to 'input_frames/frame_%08d.png':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Stream #0:0(und): Video: png, rgb24(pc, gbr/bt709/bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn (default)
    Metadata:
      creation_time   : 2023-04-20T04:13:28.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      timecode        : 01:00:00:00
      encoder         : Lavc60.3.100 png
frame= 1279 fps=8.3 q=-0.0 Lsize=N/A time=00:00:21.30 bitrate=N/A speed=0.138x    s/s speed=N/A
video:6651105kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
only rife-v4 model support custom numframe and timestep
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[image2 @ 0x157f05860] Could find no file with path 'output_frames/%08d.png' and index in the range 0-4
output_frames/%08d.png: No such file or directory

[arendtio]

As far as I understand the num frames parameter, you can use it to specify the number of frames you want to have in the end. So if you have a 5-second video with 24 fps you have 5*24=120 frames in the beginning.

If you want to interpolate from 24 to 480 fps it means you increase your number of frames by a factor of 20. So you have to multiply your initial frames by this factor to find out how many frames you will require in the end: 120*20=2400

However, there seems to be a bug. Somehow the interpolation is wrong and the second half is always the final frame. You should be able to circumvent the bug by increasing the num frames by 2 and deleting the second half of the interpolated frames... Maybe it is easier to fix the bug, which should be somewhere around this line:

https://github.com/nihui/rife-ncnn-vulkan/blob/master/src/main.cpp#L713

EDIT: IMHO changing that line to the following produces much better results:

double scale = (double)(count - 1.0) / (numframe - 1.0);

[2blackbar]

i knew there was abug! what up devs!!! are you sleeping or what ?
Do You have it compiled with that fix ?