How librav1e Handles 10-bit and 12-bit Color

This article explains how librav1e, the C-compatible library interface for the Rust-based AV1 encoder rav1e, processes and encodes 10-bit and 12-bit color depths. We will examine how the encoder configures high bit-depth formats, manages internal pixel representation, utilizes SIMD hardware acceleration, and applies these color depths to improve video quality.

Native Support and Configuration

librav1e provides native support for 8-bit, 10-bit, and 12-bit color depths. Rather than converting high bit-depth sources down to 8-bit, the encoder preserves the wider color space and dynamic range of the source material.

To handle 10-bit or 12-bit color, developers configure the encoder using the librav1e API. This is achieved by setting the pixel format and bit depth parameters within the encoder configuration structure (RaConfig). librav1e supports several chroma subsampling formats—such as YUV420, YUV422, and YUV444—paired with 10-bit and 12-bit color depths. The library automatically adjusts its internal quantization and coding tools based on these settings.

Internal Pixel Representation

Because standard 8-bit CPU registers cannot hold 10 or 12 bits of data, librav1e changes how it stores and processes pixels in memory when handling high bit-depth content:

• 8-bit Encoding: Uses standard 8-bit unsigned integers (u8) for pixel representation.
• 10-bit and 12-bit Encoding: Uses 16-bit unsigned integers (u16) to store pixel data.

When a 10-bit or 12-bit frame is passed to the encoder, the pixels are read into 16-bit arrays. The encoder keeps the active bits left-aligned or right-aligned depending on the specific AV1 coding tool requirements, ensuring that no precision is lost during spatial and temporal predictions.

SIMD Acceleration for 16-bit Pipelines

Processing video using 16-bit integers instead of 8-bit integers doubles the amount of data the CPU must compute. To prevent a significant drop in encoding speed, librav1e leverages assembly-optimized SIMD (Single Instruction, Multiple Data) instructions.

The underlying rav1e engine includes handwritten assembly routines for x86-64 (AVX2, AVX-512) and ARM (NEON) architectures. These assembly paths are specifically optimized to process 16-bit data types. This ensures that tasks like motion estimation, intra-prediction, and discrete cosine transforms (DCT) run efficiently when encoding 10-bit or 12-bit video.

In-Loop Filtering and Precision

AV1 utilizes three in-loop filters to reduce compression artifacts: the deblocking filter, the Constrained Directional Enhancement Filter (CDEF), and the Loop Restoration filter.

When processing 10-bit or 12-bit color, librav1e scales the filter thresholds and strengths to match the higher bit-depth range. Because CDEF and Loop Restoration operate on a 16-bit pipeline, they retain fractional precision during calculation. This prevents rounding errors, resulting in smoother gradients and a reduction in color banding.

Benefits of 10-bit Encoding in AV1

Even when the source material is 8-bit, encoding video in 10-bit with librav1e is highly recommended. The internal 10-bit precision reduces the accumulation of rounding errors across deep compression structures. This results in cleaner gradients, particularly in dark scenes or sky backgrounds, and often yields a better compression ratio (higher visual quality at a lower bitrate) compared to native 8-bit encoding.