Best Benchmarking Tools for librav1e Performance

Testing the performance of the Rust-based AV1 encoder, librav1e, requires specialized benchmarking tools to accurately measure encoding speed, CPU efficiency, and video quality. This article explores the top tools suited for benchmarking librav1e—including specialized codec testing suites like AreWeCompressedYet (AWCY), command-line profilers like Hyperfine, and integration tools like FFmpeg—to help you choose the right tool for your testing workflow.

AreWeCompressedYet (AWCY)

AreWeCompressedYet (AWCY) is the premier benchmarking platform used directly by the rav1e and Alliance for Open Media (AOM) development communities. It is designed specifically for video codec comparison, allowing users to run automated tests that measure both encoding speed and objective quality metrics.

AWCY computes Bjøntegaard-Delta (BD-rate) savings, comparing librav1e against other AV1 encoders like SVT-AV1 or reference encoders like libaom. It runs encodes across standardized test sets (like the Derf video collection) and generates detailed plots comparing quality metrics (PSNR, SSIM, VMAF) against encoding time.

Hyperfine

For quick, command-line-based execution time benchmarking, Hyperfine is an exceptional tool. Written in Rust, it is a command-line benchmarking tool that can measure how long the rav1e CLI takes to encode a specific video file.

Hyperfine automatically handles multiple runs to gather statistical data, performs warm-up runs to eliminate disk cache latency, and provides a clear statistical analysis of execution speed. It is ideal for developers testing local code changes to see how optimization patches impact encode times.

FFmpeg with librav1e

FFmpeg remains the industry standard for media processing integration. When compiled with --enable-librav1e, FFmpeg can be used to benchmark the encoder’s performance in real-world pipelines.

By using FFmpeg’s built-in benchmarking flags (such as -benchmark), you can easily measure: * Frames per second (FPS): The speed of the encoding process. * System resource usage: CPU utilization and maximum resident set size (memory usage). * Bitrate target accuracy: How closely librav1e adheres to requested bitrates.

This approach is highly valuable for testing how librav1e performs when decoding input streams on the fly using different FFmpeg demuxers and filters.

VMAF (Video Multi-Method Assessment Fusion)

When benchmarking an encoder, speed is only half of the equation; you must also measure the visual quality achieved at a given bitrate. Netflix’s VMAF is the industry-standard tool for this purpose.

By combining VMAF with librav1e outputs, you can calculate the efficiency curve of the encoder. This helps determine which librav1e speed presets (ranging from 0 to 10) offer the best trade-off between encoding speed and subjective visual quality.