Ffmpeg help needed with varible framerate

Software & Operating Systems Open Source & Web-Based
,
1

Iam kinda new to ffmpeg so bear with me.
I’m using the software Topaz video Enhance AI to upscale some videos. It creates .pngs of each upscaled frame and then I merge them with ffmpeg into a h265 video.
My problem now is Iam currently upscaling a .mp4 with varible framerate:
Frame rate : 29.970 FPS
Minimum frame rate : 29.960 FPS
Maximum frame rate : 29.970 FPS
It probably won’t be a huge issue in this example but how can I tell ffmpeg to encode with this exact variable framerate so the audio isn’t out of sync?

My current ffmpeg command is:
ffmpeg -framerate 30000/1001 -i %06d.png -c:v libx265 -crf 16 -preset faster -pix_fmt yuv420p10le E:\movies\no.mkv

Thanks in advance

2

The Mediainfo for the source file:

General
Complete name               : 
Format                      : MPEG-4
Format profile              : Base Media
Codec ID                    : isom (isom/iso2/avc1/mp41)
File size                   : 5.02 GiB
Duration                    : 2 h 0 min
Overall bit rate            : 5 979 kb/s
Writing application         : Lavf57.46.100

Video
ID                          : 1
Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile              : Main@L4
Format settings             : CABAC / 3 Ref Frames
Format settings, CABAC      : Yes
Format settings, Reference  : 3 frames
Format settings, GOP        : M=4, N=60
Codec ID                    : avc1
Codec ID/Info               : Advanced Video Coding
Duration                    : 2 h 0 min
Bit rate                    : 5 744 kb/s
Width                       : 1 920 pixels
Height                      : 1 080 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Variable
Frame rate                  : 29.970 FPS
Minimum frame rate          : 29.960 FPS
Maximum frame rate          : 29.970 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.092
Stream size                 : 4.80 GiB (96%)
Writing library             : x264 core 157 r2945 72db437
Encoding settings           : cabac=1 / ref=3 / deblock=1:1:0 / analyse=0x1:0x131 / me=hex / subme=7 / psy=1 / psy_rd=0.50:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=60 / keyint_min=31 / scenecut=0 / intra_refresh=0 / rc_lookahead=40 / rc=cbr / mbtree=1 / bitrate=5744 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=69 / qpstep=4 / vbv_maxrate=5744 / vbv_bufsize=12000 / nal_hrd=none / filler=0 / ip_ratio=1.41 / aq=1:1.00
Codec configuration box     : avcC

Audio
ID                          : 2
Format                      : AAC LC
Format/Info                 : Advanced Audio Codec Low Complexity
Codec ID                    : mp4a-40-2
Duration                    : 2 h 0 min
Bit rate mode               : Constant
Bit rate                    : 256 kb/s
Channel(s)                  : 2 channels
Channel layout              : L R
Sampling rate               : 48.0 kHz
Frame rate                  : 46.875 FPS (1024 SPF)
Compression mode            : Lossy
Stream size                 : 220 MiB (4%)
Default                     : Yes
Alternate group             : 1
3

Good question, I only know how to do constant rate factor. I know it will do some VBR automagically but I don’t know how to force a max and min frame rate. At this point the man pages or gnu info articles will be your friend. Or ask on the ffpmeg website if you cannot find on their forvm or wiki.

4

The problem is that Topaz Video Enhance AI can’t/won’t handle Variable Frame rate files to upscale, you need to convert your file to Constant Frame Rate first, then upscale, then turn the image set into a video to re-mux the audio without it going out of sync.

5

Variable frame rate is incredibly common on Android source videos. Nintendo Switch NVENC encoder videos are variable in MediaInfo, but actually run at 30.00fps. Same with anything you shoot with your Android video camera function.

Since the variation is not that great, I have a feeling it’s a Handbrake source file. You should not have major desync issues if you convert it to constant frame rate. Where you experience desync is iPhone files that drop the framerate mid-file to 23.976 for low light. or if it’s an Android file where minimum is 29.970 and maximum is 30.333. (Because Android works in Fractional frame rates, not integers)

BTW, Video2X might be a better solution that does not require PNG conversion.

6

Thanks, I wasn’t aware that Video Enhance AI could have problems with it.

7

The time-stamps, specifying which video frame to display at which instant, is stored in the container (e.g. mkv). When reencoding you want to use the ffmpeg option -copyts to preserve it. Once you separate the video from the container, such as converting to a series of png images, or converting to a less advanced container format, you’ve lost that timing information, and there’s realistically no way to get it back.

That’s why ffmpeg has its own filter component. You should find a libavfilter component to do whatever filtering you need, so that it can be done inside the video encoding pipeline and preserve those time-stamps and related metadata.