How GPU hardware acceleration works with Linux

The processor of current PCs is usually powerful enough to work smoothly with all types of content. However, some processes put a higher load on the processor — for example, when you watch videos.

Still, playback is usually smooth because streaming providers automatically reduce the video quality if the internet connection is not fast enough or the PC’s performance is insufficient.

Nevertheless, it makes sense to reduce the load on the CPU because this reduces power consumption and the battery lasts longer on laptops.

The graphics unit (GPU) of the PC is located in the main processor, in a separate graphics chip, or on a dedicated graphics card. The GPU often doesn’t have much to do apart from displaying the desktop. But the graphics chip is often more powerful than the CPU when it comes to playing (decoding) videos.

The GPU can also help when converting videos into other formats (encoding). The overall power consumption is reduced, videos can be played in higher quality levels or converted faster.

IDG

GPU utilization under Linux

Whether and how hardware acceleration can be utilized by the graphics chip depends on several factors under Linux:

• The model of the graphics chip and its capabilities
• The driver used
• The display protocol (X11 or Wayland)
• The respective software (browser, video player, video editor)

IDG

There are also several interfaces for different application areas that can be used for hardware acceleration:

• Vulkan is mainly used by games and can also be used when playing videos.
• Open GL is the predecessor of Vulkan and offers support for 2D/3D vector graphics on the desktop and in games.
• Vdpau offers hardware acceleration for video playback, comes from Nvidia, and is also supported by AMD graphics drivers.
• CUDA is a programming interface (API) from Nvidia with which program parts can be processed by the graphics processor (GPU). The API is used, for example, in AI applications and when playing (NVDEC) and converting (NVENC) videos.
• VA-API is an interface for decoding and encoding videos with AMD and Intel GPUs.
• Quick Sync Video is the name of the hardware acceleration of Intel GPUs, which is accessed via the VA-API.

Media players usually recognize the available interfaces automatically and use hardware acceleration by default. Web browsers, on the other hand, usually do not offer this function with common graphics chips.

Current graphics chips provide hardware acceleration for playback with the codecs H264 (AVCHD), H265 (HEVC), and VP9. This means you are well equipped because YouTube, for example, usually uses H264 or VP9.

You can find out what a chip can do from Nvidia, Intel and AMD.

IDG

In the foreseeable future, YouTube will increasingly switch to the license fee-free AV1 codec, the improved successor to VP9. Netflix will also be using AV1 in the future.

However, only newer graphics chips such as Intel Tiger Lake (11th generation) or Nvidia GeForce from RTX 2050 offer hardware acceleration (decoder) for AV1. If you want to use the AV1 encoder for conversion, you need a graphics card from GeForce RTX 4050, AMD Radeon RX 7000, or Intel Arc.

Even if the graphics chip supports a format, drivers and interfaces must also be compatible. VLC, for example, can use Vdpau for hardware acceleration if the Nvidia driver is installed.

However, browsers such as Mozilla Firefox or Google Chrome rely on VA-API, which the Nvidia driver does not support. A suitable driver with VA-API (“nvidia-vaapi-driver”) is only officially available in the distribution repository from Ubuntu 23.04 onwards. Support for VA-API is included as standard for Intel graphics; AMD drivers support VA-API and Vdpau.

The open source driver Nouveau for Nvidia chips also supports VA-API and VDPAU, but only for older chipsets such as GeForce 700 or 600.

Check support for Intel chips

The graphics units from Intel are usually found in the CPU of the PC. Quick Sync Video has been included since around 2011, starting with the Sandy Bridge processors (Intel HD Graphics 2000 and 3000). Linux distributions contain the necessary drivers as standard.

IDG

You can find out what the chipset has to offer in the terminal with the vainfo tool, which can be installed via the package of the same name. It displays the version numbers of the VA-API and the Intel driver.

You can also see the supported codec profiles, such as “VAProfileH264Main” and “VAProfileVP8Version0_3”. The addition “VAEntrypointEncSliceLP” indicates that encoding, i.e. accelerated conversion to this format, is also offered for this codec.

You can check whether a program actually uses the VA-API with another tool, which you can install with

sudo apt install intel-gpu-tools

in the terminal. Then start

sudo intel_gpu_top

and in another terminal call the media player VLC in the form

vlc 

If the video is encoded with a codec such as VP9 or H264, VLC provides information on the use of VA-API and the associated driver, intel_gpu_top shows the utilization of the GPU.

Note: If a dedicated graphics card from Nvidia is installed in the PC, the functions of the Intel graphics unit cannot be used. For laptops with hybrid graphics, you can switch to the Intel chip via the Nvidia X Server Settings tool under “PRIME profiles.” If the option is not available, use the following in the terminal

sudo prime-select intel

You must then log out and log in again. Replace “intel” with “nvidia” or “on-demand” (if required) to switch back to the more powerful Nvidia chip.

Play videos in Firefox with VA-API

Firefox offers support for the VA-API under X11 (Linux Mint) and under Wayland (Ubuntu) by default.

Call up the internal address “about:support” to check the function. Below the “Graphics” section, you will see “WebRenderer” after “Composing.” If “WebRenderer (software)” is displayed here, hardware acceleration is not active. Under “Media” you can find out for which codecs hardware decoding is offered.

You can call up a list of YouTube example videos in high resolution here.

Show “Statistics for interested parties” via the context menu of the video. If the line after “Codec” begins with “av01,” hardware acceleration is only available if the graphics chip already supports the format.

Try out different resolutions after clicking on the cogwheel symbol behind “Quality” until “vp09” appears behind “Codec.”

You can also install the “Your Codecs” extension in Firefox and use it to block the VP8, VP9, or AV1 formats. If all are blocked, YouTube will deliver videos with the AVC1 codec (H.264), but usually only in a lower resolution of up to 1080p.

In the YouTube settings, which you can access via the hamburger menu (three horizontal lines) at the top left, the behavior can be influenced under “Playback and performance.” If the graphics chip does not support AV1, activate the option “Prefer AV1 for SD” under “AV1 settings.”

YouTube then only uses AV1 at low resolutions, which reduces the CPU load. Otherwise it will switch to VP9 or another format. In the terminal you can use

sudo intel_gpu_top

to see how much support from the GPU reduces the load on the CPU.