Skip to content

cuda: add psnr_cuda, ssim_cuda and ciede_cuda feature extractors - #1563

Open
BardieJoensen wants to merge 14 commits into
Netflix:masterfrom
BardieJoensen:cuda-psnr-ssim
Open

cuda: add psnr_cuda, ssim_cuda and ciede_cuda feature extractors#1563
BardieJoensen wants to merge 14 commits into
Netflix:masterfrom
BardieJoensen:cuda-psnr-ssim

Conversation

@BardieJoensen

@BardieJoensen BardieJoensen commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

This PR adds three CUDA feature extractors so PSNR, SSIM and CIEDE2000 can run in the same GPU pass as the VMAF features, instead of forcing a separate CPU pass (and, with libvmaf_cuda in ffmpeg, a second decode of both inputs):

  • psnr_cuda — GPU port of psnr. Integer SSE reduction on device (vectorized grid-stride loads, one atomic per block), scoring math unchanged on the host, so scores are bit-exact vs the CPU extractor — now asserted for PSNR and MSE at 8/10/12/16-bit across YUV420P/YUV422P/YUV444P, with APSNR aggregates compared as well. It is also verified on 500-frame real SDR/HDR clips, y/cb/cr. Supports the CPU extractor's options (enable_chroma, enable_mse, enable_apsnr, reduced_hbd_peak, min_sse). The APSNR flush emits only via vmaf_feature_collector_set_aggregate, so it is idempotent under the threaded+CUDA double flush (cuda: don't flush CUDA feature extractors twice when a thread pool is present #1553).
  • ssim_cuda — GPU port of float_ssim, replicating the iqa reference operation-for-operation: box decimation with symmetric borders (fused with the float conversion, so the full-resolution float image is never materialized), 11-tap separable valid-mode Gaussian with double accumulation and float rounding between passes, and the exact float/double mixing of _iqa_ssim (float-typed reference operations use explicit __f*_rn intrinsics so nvcc cannot contract them into fma). Also bit-exact vs the CPU extractor in all testing, and deterministic run-to-run (per-block tree reduction + sequential host sum over block partials). Supports enable_lcs, enable_db, clip_db, scale.
  • ciede_cuda — GPU port of ciede. One fused kernel does the nearest-neighbor chroma upsample, YUV→Lab conversion and per-pixel ΔE, with per-block double partials pooled sequentially on the host into the 45 − 20·log10(mean ΔE) score (identical frames yield +inf, matching the CPU extractor — note this also means the pooled mean/max serialize as null in JSON output whenever any frame is identical, for the CPU and CUDA extractors alike, while harmonic_mean stays finite). The shared chroma-upsample index math now correctly treats horizontal and vertical subsampling independently, with YUV420P and YUV422P regression coverage. Unlike the other two, this port is tolerance-validated rather than bit-exact: ciede is dominated by libm transcendentals, which round differently between glibc and CUDA regardless of precision, and FP64 throughput makes a faithful double port impractical on consumer GPUs (1/64 rate). Device math runs in float32 — the CPU reference truncates every intermediate to float anyway — and measured agreement is within 3e-5 on the per-frame score against the CPU extractor on real content (the included test asserts 1e-3). For reference, the CPU extractor measures ~4 fps at 4K with 16 threads (scalar libm, no SIMD variant); the CUDA extractor makes the metric usable full-file.

Performance (RTX 5060 Ti, two 4K HEVC inputs via the ffmpeg libvmaf_cuda filter, vmaf + vmaf_neg models): adding all three features costs ~8% vs models-only. The extractors launch on their own CUDA stream and overlap with the model extractors' work instead of extending the shared picture stream's critical path; picture lifetime is handled by recording a consumed event after the last picture-reading kernel and making both picture streams wait on it, so the pool's existing finished-event reuse protocol transitively covers the own-stream reads. compute-sanitizer racecheck reports 0 hazards (run with per-frame host synchronization in place of the two-slot pipelining — racecheck cannot retire its tracking state without host syncs and runs out of memory on the pipelined variant; the cross-stream handshake under test is identical). The pipelined variant is additionally verified deterministic across repeated runs and bit-identical to the serialized implementation. One subtlety worth reviewer attention: the work streams are deliberately created CU_STREAM_DEFAULT (legacy-blocking) — producers like the ffmpeg filter fill device pictures with synchronous-API D2D copies that queue on the legacy NULL stream without blocking the host, and only blocking-flavor streams are implicitly ordered after NULL-stream work. With a non-blocking stream, ~4% of frames scored nondeterministically through the ffmpeg path; blocking flavor restores the ordering while still overlapping with the other extractors' created streams.

Supporting changes:

  • New VMAF_FEATURE_EXTRACTOR_CUDA_CHROMA flag: the host→device picture upload (translate_picture_host) copied only the luma plane because all existing CUDA extractors are luma-only. The upload plane mask now widens to all planes only when a registered CUDA extractor declares it reads chroma — existing configurations upload exactly what they did before. The device→host direction (translate_picture_device) now downloads all planes unconditionally, since host pictures always carry every plane and CPU extractors that read chroma (psnr, ciede) expect them.
  • All three extractors provide a flush that drains their async score callback, so the final frame's score is in the collector before scores are read after flushing the context.
  • Tests: test_cuda_psnr_ssim_parity drives deterministic 8/10/12/16-bit YUV420P/YUV422P/YUV444P frames through the CPU and CUDA extractors via the public API. It asserts bit-exact PSNR, MSE, SSIM and L/C/S scores, compares APSNR aggregates, covers odd dimensions and automatic/explicit scales, and exercises the remaining scoring options. test_cuda_ciede_parity covers YUV420P and YUV422P in both 8-bit and high-bit-depth paths, asserts the 1e-3 tolerance, and checks the identical-frame +inf case. Both exit 77 (meson SKIP) when no CUDA device is available, so CI without a GPU reports them as skipped rather than silently passing. Dispatch assertions in test_feature_extractor.c are extended to the new extractors.

Naming follows the existing convention (vif_cuda/motion_cuda/adm_cuda): unique .name for --feature psnr_cuda / ffmpeg feature=name=psnr_cuda, with provided_features matching the CPU extractors so model-driven flag-aware selection works. The option tables mirror the CPU extractors' tables, following the existing CUDA extractors' pattern (they embed offsetof into each extractor's own state struct) — happy to factor the scoring math into a shared header instead if you'd prefer.

Notes:

  • Rebased onto master at f85a8536. cuda: fix row stride in calculate_motion_score_kernel_16bpc (wrong scores for >8-bit input) #1552 has landed and its duplicate commit has been removed; this branch remains stacked on cuda: don't flush CUDA feature extractors twice when a thread pool is present #1553 for correct CUDA operation with a thread pool.
  • While validating through ffmpeg's libvmaf_cuda filter I found that vf_libvmaf.c's copy_picture_data_cuda breaks out of its plane loop after plane 0, so chroma never reaches libvmaf and chroma-reading features report the psnr_max cap. That is an ffmpeg bug, not a libvmaf one (I patch it locally and plan to report it there), but it explains why chroma features through the filter need an ffmpeg fix to be useful.
  • Possible follow-up (not in this PR): the model extractors (vif/adm/motion) still serialize on the shared picture streams; moving them to per-extractor streams with the same consumed-event handshake should overlap them the same way.

Tested on RTX 5060 Ti (sm_120), driver 595.71.05, CUDA 12.9/13.3.

@BardieJoensen BardieJoensen reopened this Jul 12, 2026
@BardieJoensen BardieJoensen changed the title cuda: add psnr_cuda and ssim_cuda feature extractors cuda: add psnr_cuda, ssim_cuda and ciede_cuda feature extractors Jul 12, 2026
@BardieJoensen
BardieJoensen marked this pull request as ready for review July 12, 2026 09:44
motion_cuda is declared TEMPORAL | CUDA. With a thread pool present,
flush_context_threaded() flushes every TEMPORAL extractor and the
HAVE_CUDA block in flush_context() then flushes every CUDA extractor,
so motion_cuda gets flushed twice. The second flush re-appends the
final motion2 score at the same picture index, which
feature_vector_append() rejects, and the resulting -EINVAL is folded
into the same err as the cuCtxSynchronize() calls, so any CUDA run
with --threads N aborts with a misleading "context could not be
synchronized" even though no CUDA call failed.

Skip CUDA extractors in the temporal flush loop and leave them to the
HAVE_CUDA block, which is the only thing that flushes them on the
non-threaded path already. Verified across --threads 0/1/4 x
default/--gpumask 0: every combination now completes, where before
any --threads >= 1 run aborted with exit 234. On 8-bit input all six
combinations produce identical scores; 10-bit needs the 16bpc motion
stride fix (submitted separately) on top for full score equality,
since that bug's out-of-bounds reads vary with allocation layout.
Computes psnr_y/psnr_cb/psnr_cr on the GPU for CUDA device pictures,
bit-exact with the CPU psnr extractor (integer SSE reduction on device,
identical double-precision scoring on the host). Supports 8/10/12/16 bpc,
enable_mse, enable_apsnr (aggregate-only flush, idempotent), min_sse and
reduced_hbd_peak, mirroring the CPU extractor's options.

Since existing CUDA feature extractors are luma-only, the host-to-device
picture upload only copied plane 0. Adds a CUDA_CHROMA extractor flag and
widens the upload plane mask only when a registered CUDA extractor
declares it reads chroma, so existing configurations upload exactly what
they did before.
GPU port of float_ssim producing per-frame float_ssim scores bit-exact
with the CPU extractor: the kernels replicate the iqa reference
operation-for-operation (box decimation with symmetric borders, 11-tap
separable valid-mode Gaussian with double accumulation and float rounding
between passes, the zli-nflx l/c/s per-pixel math with its exact
float/double mixing, guarded against nvcc fma contraction with explicit
__f*_rn intrinsics). Frame mean uses per-block tree reduction plus a
sequential host-side sum over blocks, so scores are deterministic
run-to-run. Supports 8/10/12/16 bpc and the enable_lcs/enable_db/clip_db/
scale options of the CPU extractor. Luma-only, non-temporal.

Verified bit-exact against float_ssim on 8-bit/10-bit, scale 1/3/4 (auto),
odd dimensions and yuv444.
Drives identical deterministic synthetic frames (8- and 10-bit, at a
resolution that exercises float_ssim's decimation path) through the CPU
and CUDA extractors via the public API and asserts per-frame equality.
Runs the CUDA pass with a thread pool and enable_apsnr as a regression
for flush idempotency under the threaded+CUDA double-flush. Skips with a
notice when no CUDA device is available. Also extends the feature-name
dispatch assertions to the new extractors.
Per-warp atomics to the single per-plane SSE accumulator serialized at
~2.6 ms/frame on 4K yuv444p16 (~700k atomics/frame). Reduce within the
block via shared memory and issue one atomicAdd per block. Integer sums
are order-independent, so scores remain bit-exact.
- ssim_cuda: add a flush that drains the async write_scores host
  callback; without it the final frame's score could still be pending
  when scores are read after flushing the context (every other CUDA
  extractor already synchronizes here)
- translate_picture_device: download all planes, not just luma — CPU
  feature extractors that read chroma (psnr, ciede) expect complete
  host pictures, mirroring the upload-side plane-mask fix
- parity test: exit 77 on missing CUDA device so meson reports SKIP
  instead of a silent pass on CI without a GPU
All extractors used to launch on the shared per-picture stream, so each
one's kernels extended the serial critical path. psnr_cuda and ssim_cuda
now launch on their own non-blocking stream and overlap with the model
extractors' work.

Correctness relies on the picture pool's existing reuse protocol: a
picture is recycled only after the 'finished' event its own stream
records post-extraction has completed. The extractors record a
'consumed' event after their last picture-reading kernel and make both
picture streams wait on it, so 'finished' transitively covers the
own-stream reads (verified with compute-sanitizer racecheck: 0 hazards).
ssim_cuda releases the pictures right after its normalize stage - the
rest of its pipeline runs on private buffers.

The head-of-extract stream synchronization is replaced by a two-slot
ring over the write_scores parameters and pinned readback buffers, so
back-to-back frames pipeline instead of stalling the host; scores stay
deterministic (verified identical across repeated runs) and bit-exact.
The 16x16 block layout made each warp read only 16 contiguous bytes per
row, half-filling memory sectors. Switch both kernels to a 1-D
grid-stride loop over uchar4/ushort2 vectors (row starts are aligned by
cuMemAllocPitch); rows whose width isn't a multiple of the vector width
get their tail pixels from a scalar per-row loop. Integer sums are
order-independent, so scores are unchanged.
When decimation is active, the full-resolution float images were only
ever read at every factor-th sample by the box filter. Read the raw
picture plane in the decimate kernel and convert inline instead; the
conversion is exact per sample (power-of-two divide), so the filter sees
bit-identical floats. Saves two full-resolution float buffers and their
round trip through memory; the factor==1 path keeps the separate
normalize kernels.
Producers can fill device pictures with synchronous-API copies that are
queued on the legacy NULL stream (device-to-device memcpy does not block
the host) - the ffmpeg libvmaf_cuda filter does exactly this. Only
blocking-flavor streams are implicitly ordered after NULL-stream work,
so a non-blocking work stream could read a picture before the producer's
copy landed (observed as nondeterministic scores on ~4% of frames
through the ffmpeg filter). CU_STREAM_DEFAULT restores the ordering
while keeping overlap with the other extractors' created streams.
Verified deterministic across repeated ffmpeg-filter runs and
bit-identical to the pre-overlap implementation.
CONTRIBUTING.md asks new files to carry the original author's copyright;
the seven files added by this branch had headers copied verbatim from
their templates.
GPU port of the ciede (CIEDE2000) extractor: a single fused kernel per
bit depth does the nearest-neighbor chroma upsample (replicating
scale_chroma_planes' index math), YUV->Lab conversion and per-pixel
delta-E, with per-block double partial sums pooled sequentially on the
host into the 45 - 20*log10(mean) score (identical frames yield +inf,
matching the CPU extractor).

Unlike psnr_cuda/ssim_cuda this port is tolerance-validated rather than
bit-exact: ciede is dominated by libm transcendentals, which round
differently between glibc and CUDA regardless of precision, and FP64
throughput makes a faithful double port impractical on consumer GPUs.
Device math runs in float32 (the CPU reference truncates every
intermediate to float anyway); measured agreement is within 3e-5 on the
per-frame score (test asserts 1e-3). Uses the CUDA_CHROMA flag and the
same own-stream/lifetime-handshake/two-slot pattern as the other CUDA
extractors. Includes a CPU/CUDA parity test (skips without a GPU).
Use the horizontal subsampling flag for column scaling and the vertical flag for row scaling. YUV420 and YUV444 masked the mix-up because both flags have the same value for those formats. Add 8-bit and high-bit-depth YUV422 CPU/CUDA parity coverage.
Require bit-exact per-frame PSNR, MSE, SSIM, luminance, contrast and structure scores. Compare serialized APSNR aggregates and cover 12/16-bit, YUV444/YUV422, odd dimensions, explicit scales, dB output, clipping, reduced peak and min_sse options.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant