VideoProcessingFramework

Deprecation notice

VPF is being replaced by PyNvVideoCodec library with leaner API and pip install support. The library offers easy-to-use Python APIs, granting access to the core C/C++ video encode/decode capabilities of the Video Codec SDK.

PyNvVideoCodec library is distributed in two formats: binary distribution via PyPI and source code distribution via NVIDIA NGC. In both cases, it can be installed using a single pip install command. The library is distributed under the MIT license and is officially supported by NVIDIA. PyNvVideoCodec supports all features of VPF (except software encode/decode and surface format conversion).

For more information, please visit the Get Started with PyNvVideoCodec page.

We'd like to thank Roman Arzumanyan, original author of VPF, for his support and efforts along these years. Roman will continue his work on VALI also leveraging NVIDIA GPUs.

VideoProcessingFramework

VPF stands for Video Processing Framework. It’s set of C++ libraries and Python bindings which provides full HW acceleration for video processing tasks such as decoding, encoding, transcoding and GPU-accelerated color space and pixel format conversions.

VPF also supports exporting GPU memory objects such as decoded video frames to PyTorch tensors without Host to Device copies.

Prerequisites

VPF works on Linux(Ubuntu 20.04 and Ubuntu 22.04 only) and Windows

• NVIDIA display driver: 525.xx.xx or above

• CUDA Toolkit 11.2 or above

  • CUDA toolkit has driver bundled with it e.g. CUDA Toolkit 12.0 has driver 530.xx.xx. During installation of CUDA toolkit you could choose to install or skip installation of the bundled driver. Please choose the appropriate option.

• FFMPEG

  • Compile FFMPEG with shared libraries
  • or download pre-compiled binaries from a source you trust.
    • During VPF’s “pip install”(mentioned in sections below) you need to provide a path to the directory where FFMPEG got installed.
  • or you could install system FFMPEG packages (e.g. apt install libavfilter-dev libavformat-dev libavcodec-dev libswresample-dev libavutil-dev on Ubuntu)

• Python 3 and above

• Install a C++ toolchain either via Visual Studio or Tools for Visual Studio.

  • Recommended version is Visual Studio 2017 and above (Windows only)

Linux

We recommend Ubuntu 20.04 as it comes with a recent enough FFmpeg system packages. If you want to build FFmpeg from source, you can follow https://docs.nvidia.com/video-technologies/video-codec-sdk/12.0/ffmpeg-with-nvidia-gpu/index.html

…

To check whether VPF is correctly installed run the following Python script

import PyNvCodec

If using Docker via Nvidia Container Runtime, please make sure to enable the video driver capability: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/user-guide.html#driver-capabilities via the NVIDIA_DRIVER_CAPABILITIES environment variable in the container or the --gpus command line parameter (e.g. docker run -it --rm --gpus 'all,"capabilities=compute,utility,video"' nvidia/cuda:12.1.0-base-ubuntu22.04).

Please note that some examples have additional dependencies that need to be installed via pip (pip install .[samples]). Samples using PyTorch will require an optional extension which can be installed via

pip install src/PytorchNvCodec  # install Torch extension if needed (optional), requires "torch" to be installed before

After resolving those you should be able to run make run_samples_without_docker using your local pip installation.

Windows

• Install a C++ toolchain either via Visual Studio or Tools for Visual Studio (https://visualstudio.microsoft.com/downloads/)
• Install the CUDA Toolkit: https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64
• Compile FFMPEG with shared libraries or download pre-compiled binaries from a source you trust
• Install from the root directory of this repository indicating the location of the compiled FFMPEG in a Powershell console

# Indicate path to your FFMPEG installation (with subfolders `bin` with DLLs, `include`, `lib`)
$env:SKBUILD_CONFIGURE_OPTIONS="-DTC_FFMPEG_ROOT=C:/path/to/your/ffmpeg/installation/ffmpeg/" 
pip install .

To check whether VPF is correctly installed run the following Python script

import PyNvCodec

Please note that some examples have additional dependencies (pip install .[sampels]) that need to be installed via pip. Samples using PyTorch will require an optional extension which can be installed via

pip install src/PytorchNvCodec  # install Torch extension if needed (optional), requires "torch" to be installed before

Docker

For convenience, we provide a Docker images located at docker that you can use to easily install all dependencies for the samples (docker and nvidia-docker are required)

DOCKER_BUILDKIT=1 docker build \
                --tag vpf-gpu \
                -f docker/Dockerfile \
                --build-arg PIP_INSTALL_EXTRAS=torch \
                .
docker run -it --rm --gpus=all vpf-gpu

PIP_INSTALL_EXTRAS can be any subset listed under project.optional-dependencies in pyproject.toml.

Documentation

A documentation for Video Processing Framework can be generated from this repository:

pip install . # install Video Processing Framework
pip install src/PytorchNvCodec  # install Torch extension if needed (optional), requires "torch" to be installed before
pip install sphinx  # install documentation tool sphinx
cd docs
make html

You can then open _build/html/index.html with your browser.

Community Support

If you did not find the information you need or if you have further questions or problems, you are very welcome to join the developer community at NVIDIA. We have dedicated categories covering diverse topics related to video processing and codecs.

The forums are also a place where we would be happy to hear about how you made use of VPF in your project.