The NVIDIA Optical Flow Accelerator (NVOFA) is a dedicated hardware unit on newer NVIDIA GPUs for computing optical flow between a pair of images at high…
The NVIDIA Optical Flow Accelerator (NVOFA) is a dedicated hardware unit on newer NVIDIA GPUs for computing optical flow between a pair of images at high performance. The NVIDIA Optical Flow SDK exposes developer APIs that enable you to leverage the power of NVOFA hardware in your applications.
We are excited to announce the availability of Optical Flow SDK 5.0, which adds support for generating optical flow in Vulkan applications.
Vulkan enables explicit GPU control
Vulkan is a low-overhead, cross-platform, open standard API for 3D graphics and computation. Vulkan offers higher performance and more efficient CPU and GPU usage compared to the previous-generation APIs, such as OpenGL and Direct3D 11.
You can take advantage of the explicit GPU access provided by Vulkan to optimize your applications, with full control over resource management and synchronization. Since its release in 2016, Vulkan adoption has rapidly grown among game and graphics application developers.
What’s new in the Optical Flow SDK?
The NVIDIA Optical Flow SDK 5.0 now enables Vulkan applications to use the NVIDIA Optical Flow engine when running on NVIDIA Ampere architecture and newer generation GPUs.
Optical flow computed with NVOFA creates a smoother experience for games and videos by increasing frame rates and enabling efficient object tracking for video analytics.
To increase frame rate, Frame Rate Up Conversion (FRUC) techniques insert interpolated frames between original frames. Interpolation algorithms use the flow between frame pairs to generate the intermediate frame.
With the new Optical Flow SDK, you can now use the NVIDIA Optical Flow FRUC technique (NvOFFRUC) in your Vulkan application to improve visual fluidity.
The Optical Flow SDK package contains headers, sample applications that demonstrate usage, C++ wrapper classes that can be re-used or modified as required, and documentation. All other ‘under the cover’ components for accessing the Optical Flow hardware are included in the NVIDIA display driver, which contains the NVIDIA Optical Flow Vulkan extension.
You don’t have to use the Vulkan extension directly. Instead, you can use the Vulkan Optical Flow SDK APIs, which are supported on Windows 10, 11, and Linux.
The Vulkan Optical Flow API is designed to be as similar as possible to the other interfaces that are already available in the SDK, including CUDA, DirectX 11, and DirectX 12. It consists of three core functions: initialization, flow estimation, and destruction.
typedef NV_OF_STATUS(NVOFAPI* PFNNVOFINIT) (NvOFHandle hOf, const NV_OF_INIT_PARAMS* initParams); typedef NV_OF_STATUS(NVOFAPI* PFNNVOFEXECUTEVK) (NvOFHandle hOf, const NV_OF_EXECUTE_INPUT_PARAMS_VK* executeInParams, NV_OF_EXECUTE_OUTPUT_PARAMS_VK* executeOutParams); typedef NV_OF_STATUS(NVOFAPI* PFNNVOFDESTROY) (NvOFHandle hOf);
The initialization and destruction APIs are the same across all interfaces, but the Vulkan Execute API requires explicit semaphore and semaphore values input parameters to provide direct control over synchronization. These semaphore objects are used to synchronize both CPU-to-GPU and GPU-to-GPU operations.
The buffer management API interface in Vulkan also needs fence objects for synchronization. For more information, see the NVOFA Programming Guide.
The Optical Flow output quality is the same whether you use Vulkan, DX11, DX12, or CUDA, with comparable performance across all the interfaces.
Get started
Harness the NVIDIA Optical Flow accelerator with the Optical Flow SDK in your Vulkan application with Optical Flow SDK 5.0.
For more information about getting started using Vulkan, see the following resources:
- Vulkan resources at The Khronos Group
- NVIDIA Optical Flow Vulkan extension: VK_NV_optical_flow
For more information about the NVIDIA Optical Flow SDK, see the following resources:
- Harnessing the NVIDIA Ada Architecture for Frame-Rate Up-Conversion in the NVIDIA Optical Flow SDK
- AV1 Encoding and Optical Flow: Video Performance Boosts and Higher Fidelity on the NVIDIA Ada Architecture
Source:: NVIDIA