FFmpeg, the encoding and processing suite that quietly powers most of the video on the internet, has merged support for NVIDIA’s Frame Rate Up-Conversion technology. The new filter is called fruc_vulkan, and it hands the job of inventing in-between frames to the Optical Flow Accelerator built into GeForce RTX graphics cards rather than grinding through it on the CPU. The commits landed in FFmpeg Git over the weekend of August 29 and 30, 2026, authored by longtime contributor Philip Langdale.
The practical result is straightforward. A 24 FPS cinematic clip can be transcoded out at 48 FPS, and a 30 FPS capture can become 60 FPS, with the graphics card synthesizing the frames that were never shot. Software interpolation has been possible in FFmpeg for years through filters like minterpolate, but it is punishing on the CPU. Moving the motion estimation onto dedicated silicon changes the cost profile of the whole operation.
Why Vulkan Was the Route That Worked
This is not the first attempt to get FRUC into FFmpeg. An earlier patch surfaced in 2023 and never made it upstream, because it leaned on NVIDIA’s proprietary Optical Flow SDK as a build dependency, which is a hard sell for a project that ships as free software. The new implementation sidesteps that problem entirely by talking to the hardware through VK_NV_optical_flow, a standard Vulkan extension exposed by the driver. FFmpeg gets direct access to the optical flow block without carrying a closed-source library.
The optical flow hardware itself is not new. NVIDIA introduced the Optical Flow Accelerator, or NVOFA, with the GeForce RTX 20 series, and it is the same class of motion estimation logic used elsewhere in the graphics stack. NVIDIA’s own developer material describes it as a dedicated unit for computing optical flow between a pair of images, and states that output quality is consistent whether the block is driven through Vulkan, DirectX or CUDA. The company also notes that the Ada generation delivers roughly 2.5 times the throughput of Ampere on this workload alongside quality improvements, so the RTX 40 and RTX 50 families should be considerably faster at it than an RTX 30 card.
How the Filter Actually Builds a Frame
The commit series is unusually readable about its own internals. The filter began life as a copy of the existing vf_framerate filter and was converted into a Vulkan filter, then had the optical flow machinery bolted on in stages.
- Grayscale extraction: a compute shader pulls a luma-only representation of each frame, since the optical flow engine works on grayscale. For RGB inputs the filter applies hardcoded BT.709 weights to derive luma.
- Optical flow session: the filter opens a VK_NV_optical_flow session with persistent input and flow images, then uses cross-queue submission with binary semaphore ordering to run compute, then optical flow, then compute again in the correct sequence.
- Motion-compensated warping: the first pass shipped as a simple proportional blend that ignored the flow data. A follow-up commit replaced it with real motion compensation, using forward and backward flow fields to warp both source frames before blending them.
- Picard iteration: for each output pixel the filter recovers the corresponding source position on both neighbouring frames using fixed-point Picard iteration. The commit sets 6 iterations with a relaxation factor of 0.5, values the author describes as empirically established on a small set of ad-hoc samples and openly invites others to improve.
Supported pixel formats are deliberately narrow at this stage, covering planar and semi-planar YUV, grayscale, and packed RGB in component order. Because it is a Vulkan filter, it slots into a Vulkan filter chain and expects frames to be uploaded to GPU memory rather than fed in from system memory, so the exact invocation depends on how your build and filter graph are set up. Anyone planning to use it should check the filter documentation shipped with their specific build for current option names.
Hardware Support at a Glance
| GPU Generation | Consumer Series | NVOFA Hardware | Vulkan FRUC Path |
|---|---|---|---|
| Turing | GeForce RTX 20 | Yes | Not listed as supported |
| Ampere | GeForce RTX 30 | Yes | Supported |
| Ada Lovelace | GeForce RTX 40 | Yes | Supported, roughly 2.5x Ampere throughput |
| Blackwell | GeForce RTX 50 | Yes | Supported |
The RTX Video Filters Branch That Came and Went
There is a second half to this story that did not stick. Around the same window, a much larger body of NVIDIA code briefly appeared in the FFmpeg tree carrying the company’s RTX Video Filters. According to reporting on the merge, that set included Dynamic Vibrance built on the DeepDVC network, DLPP-based upscaling, NGX Image Super Resolution, Smooth Motion frame generation, RTX TrueHDR and RTX Video Super Resolution. It was backed out within days and is no longer present in the repository, with no public explanation offered for the reversal. Only the FRUC work remains.
Not Frame Gen, FYI
It is worth setting expectations here, because the phrase “frame generation” carries a lot of baggage from the gaming side. This is a transcoding filter. It runs as part of an FFmpeg processing chain to produce a higher frame rate output file, and it is not a real-time playback smoothing feature for a media player. The mpv project has had an open request for NVOFA-driven FRUC since 2022, and an upstream FFmpeg filter is a reasonable foundation for that kind of work, but nothing has shipped on that front yet.
There is also no independent performance or quality data available at this point. No benchmarks, no side-by-side comparisons against software interpolation, and no assessment of how the filter handles the usual hard cases for interpolation such as fast panning, occlusion, subtitles, or animation on twos. We have not tested it ourselves, and the algorithm constants are, by the author’s own admission, first-pass values. Treat this as a promising foundation that has just landed rather than a finished consumer feature.
The code is in FFmpeg Git now, which means it will reach general users when it appears in a tagged release and filters down through distribution packages and third-party builds. Anyone who compiles their own FFmpeg with Vulkan enabled can try it today, assuming an Ampere or newer GeForce RTX card and a current driver.
Related: Our DLSS 5 technical preview provides another look at NVIDIA’s newer GPU rendering pipeline.
FULL STORY: NVIDIA GPU Frame Interpolation Lands in FFmpeg via Vulkan Optical Flow: https://b2g.to/t29fs