cuda: add psnr_cuda, ssim_cuda and ciede_cuda feature extractors - #1563
Open
BardieJoensen wants to merge 14 commits into
Open
cuda: add psnr_cuda, ssim_cuda and ciede_cuda feature extractors#1563BardieJoensen wants to merge 14 commits into
BardieJoensen wants to merge 14 commits into
Conversation
BardieJoensen
force-pushed
the
cuda-psnr-ssim
branch
from
July 10, 2026 17:17
5c62c97 to
cc04252
Compare
BardieJoensen
marked this pull request as ready for review
July 12, 2026 09:44
motion_cuda is declared TEMPORAL | CUDA. With a thread pool present, flush_context_threaded() flushes every TEMPORAL extractor and the HAVE_CUDA block in flush_context() then flushes every CUDA extractor, so motion_cuda gets flushed twice. The second flush re-appends the final motion2 score at the same picture index, which feature_vector_append() rejects, and the resulting -EINVAL is folded into the same err as the cuCtxSynchronize() calls, so any CUDA run with --threads N aborts with a misleading "context could not be synchronized" even though no CUDA call failed. Skip CUDA extractors in the temporal flush loop and leave them to the HAVE_CUDA block, which is the only thing that flushes them on the non-threaded path already. Verified across --threads 0/1/4 x default/--gpumask 0: every combination now completes, where before any --threads >= 1 run aborted with exit 234. On 8-bit input all six combinations produce identical scores; 10-bit needs the 16bpc motion stride fix (submitted separately) on top for full score equality, since that bug's out-of-bounds reads vary with allocation layout.
Computes psnr_y/psnr_cb/psnr_cr on the GPU for CUDA device pictures, bit-exact with the CPU psnr extractor (integer SSE reduction on device, identical double-precision scoring on the host). Supports 8/10/12/16 bpc, enable_mse, enable_apsnr (aggregate-only flush, idempotent), min_sse and reduced_hbd_peak, mirroring the CPU extractor's options. Since existing CUDA feature extractors are luma-only, the host-to-device picture upload only copied plane 0. Adds a CUDA_CHROMA extractor flag and widens the upload plane mask only when a registered CUDA extractor declares it reads chroma, so existing configurations upload exactly what they did before.
GPU port of float_ssim producing per-frame float_ssim scores bit-exact with the CPU extractor: the kernels replicate the iqa reference operation-for-operation (box decimation with symmetric borders, 11-tap separable valid-mode Gaussian with double accumulation and float rounding between passes, the zli-nflx l/c/s per-pixel math with its exact float/double mixing, guarded against nvcc fma contraction with explicit __f*_rn intrinsics). Frame mean uses per-block tree reduction plus a sequential host-side sum over blocks, so scores are deterministic run-to-run. Supports 8/10/12/16 bpc and the enable_lcs/enable_db/clip_db/ scale options of the CPU extractor. Luma-only, non-temporal. Verified bit-exact against float_ssim on 8-bit/10-bit, scale 1/3/4 (auto), odd dimensions and yuv444.
Drives identical deterministic synthetic frames (8- and 10-bit, at a resolution that exercises float_ssim's decimation path) through the CPU and CUDA extractors via the public API and asserts per-frame equality. Runs the CUDA pass with a thread pool and enable_apsnr as a regression for flush idempotency under the threaded+CUDA double-flush. Skips with a notice when no CUDA device is available. Also extends the feature-name dispatch assertions to the new extractors.
Per-warp atomics to the single per-plane SSE accumulator serialized at ~2.6 ms/frame on 4K yuv444p16 (~700k atomics/frame). Reduce within the block via shared memory and issue one atomicAdd per block. Integer sums are order-independent, so scores remain bit-exact.
- ssim_cuda: add a flush that drains the async write_scores host callback; without it the final frame's score could still be pending when scores are read after flushing the context (every other CUDA extractor already synchronizes here) - translate_picture_device: download all planes, not just luma — CPU feature extractors that read chroma (psnr, ciede) expect complete host pictures, mirroring the upload-side plane-mask fix - parity test: exit 77 on missing CUDA device so meson reports SKIP instead of a silent pass on CI without a GPU
All extractors used to launch on the shared per-picture stream, so each one's kernels extended the serial critical path. psnr_cuda and ssim_cuda now launch on their own non-blocking stream and overlap with the model extractors' work. Correctness relies on the picture pool's existing reuse protocol: a picture is recycled only after the 'finished' event its own stream records post-extraction has completed. The extractors record a 'consumed' event after their last picture-reading kernel and make both picture streams wait on it, so 'finished' transitively covers the own-stream reads (verified with compute-sanitizer racecheck: 0 hazards). ssim_cuda releases the pictures right after its normalize stage - the rest of its pipeline runs on private buffers. The head-of-extract stream synchronization is replaced by a two-slot ring over the write_scores parameters and pinned readback buffers, so back-to-back frames pipeline instead of stalling the host; scores stay deterministic (verified identical across repeated runs) and bit-exact.
The 16x16 block layout made each warp read only 16 contiguous bytes per row, half-filling memory sectors. Switch both kernels to a 1-D grid-stride loop over uchar4/ushort2 vectors (row starts are aligned by cuMemAllocPitch); rows whose width isn't a multiple of the vector width get their tail pixels from a scalar per-row loop. Integer sums are order-independent, so scores are unchanged.
When decimation is active, the full-resolution float images were only ever read at every factor-th sample by the box filter. Read the raw picture plane in the decimate kernel and convert inline instead; the conversion is exact per sample (power-of-two divide), so the filter sees bit-identical floats. Saves two full-resolution float buffers and their round trip through memory; the factor==1 path keeps the separate normalize kernels.
Producers can fill device pictures with synchronous-API copies that are queued on the legacy NULL stream (device-to-device memcpy does not block the host) - the ffmpeg libvmaf_cuda filter does exactly this. Only blocking-flavor streams are implicitly ordered after NULL-stream work, so a non-blocking work stream could read a picture before the producer's copy landed (observed as nondeterministic scores on ~4% of frames through the ffmpeg filter). CU_STREAM_DEFAULT restores the ordering while keeping overlap with the other extractors' created streams. Verified deterministic across repeated ffmpeg-filter runs and bit-identical to the pre-overlap implementation.
CONTRIBUTING.md asks new files to carry the original author's copyright; the seven files added by this branch had headers copied verbatim from their templates.
GPU port of the ciede (CIEDE2000) extractor: a single fused kernel per bit depth does the nearest-neighbor chroma upsample (replicating scale_chroma_planes' index math), YUV->Lab conversion and per-pixel delta-E, with per-block double partial sums pooled sequentially on the host into the 45 - 20*log10(mean) score (identical frames yield +inf, matching the CPU extractor). Unlike psnr_cuda/ssim_cuda this port is tolerance-validated rather than bit-exact: ciede is dominated by libm transcendentals, which round differently between glibc and CUDA regardless of precision, and FP64 throughput makes a faithful double port impractical on consumer GPUs. Device math runs in float32 (the CPU reference truncates every intermediate to float anyway); measured agreement is within 3e-5 on the per-frame score (test asserts 1e-3). Uses the CUDA_CHROMA flag and the same own-stream/lifetime-handshake/two-slot pattern as the other CUDA extractors. Includes a CPU/CUDA parity test (skips without a GPU).
Use the horizontal subsampling flag for column scaling and the vertical flag for row scaling. YUV420 and YUV444 masked the mix-up because both flags have the same value for those formats. Add 8-bit and high-bit-depth YUV422 CPU/CUDA parity coverage.
Require bit-exact per-frame PSNR, MSE, SSIM, luminance, contrast and structure scores. Compare serialized APSNR aggregates and cover 12/16-bit, YUV444/YUV422, odd dimensions, explicit scales, dB output, clipping, reduced peak and min_sse options.
BardieJoensen
force-pushed
the
cuda-psnr-ssim
branch
from
September 4, 2026 17:52
5f03c6a to
3f64bed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds three CUDA feature extractors so PSNR, SSIM and CIEDE2000 can run in the same GPU pass as the VMAF features, instead of forcing a separate CPU pass (and, with
libvmaf_cudain ffmpeg, a second decode of both inputs):psnr_cuda— GPU port ofpsnr. Integer SSE reduction on device (vectorized grid-stride loads, one atomic per block), scoring math unchanged on the host, so scores are bit-exact vs the CPU extractor — now asserted for PSNR and MSE at 8/10/12/16-bit across YUV420P/YUV422P/YUV444P, with APSNR aggregates compared as well. It is also verified on 500-frame real SDR/HDR clips, y/cb/cr. Supports the CPU extractor's options (enable_chroma,enable_mse,enable_apsnr,reduced_hbd_peak,min_sse). The APSNR flush emits only viavmaf_feature_collector_set_aggregate, so it is idempotent under the threaded+CUDA double flush (cuda: don't flush CUDA feature extractors twice when a thread pool is present #1553).ssim_cuda— GPU port offloat_ssim, replicating the iqa reference operation-for-operation: box decimation with symmetric borders (fused with the float conversion, so the full-resolution float image is never materialized), 11-tap separable valid-mode Gaussian with double accumulation and float rounding between passes, and the exact float/double mixing of_iqa_ssim(float-typed reference operations use explicit__f*_rnintrinsics so nvcc cannot contract them into fma). Also bit-exact vs the CPU extractor in all testing, and deterministic run-to-run (per-block tree reduction + sequential host sum over block partials). Supportsenable_lcs,enable_db,clip_db,scale.ciede_cuda— GPU port ofciede. One fused kernel does the nearest-neighbor chroma upsample, YUV→Lab conversion and per-pixel ΔE, with per-block double partials pooled sequentially on the host into the45 − 20·log10(mean ΔE)score (identical frames yield +inf, matching the CPU extractor — note this also means the pooledmean/maxserialize asnullin JSON output whenever any frame is identical, for the CPU and CUDA extractors alike, whileharmonic_meanstays finite). The shared chroma-upsample index math now correctly treats horizontal and vertical subsampling independently, with YUV420P and YUV422P regression coverage. Unlike the other two, this port is tolerance-validated rather than bit-exact: ciede is dominated by libm transcendentals, which round differently between glibc and CUDA regardless of precision, and FP64 throughput makes a faithful double port impractical on consumer GPUs (1/64 rate). Device math runs in float32 — the CPU reference truncates every intermediate to float anyway — and measured agreement is within 3e-5 on the per-frame score against the CPU extractor on real content (the included test asserts 1e-3). For reference, the CPU extractor measures ~4 fps at 4K with 16 threads (scalar libm, no SIMD variant); the CUDA extractor makes the metric usable full-file.Performance (RTX 5060 Ti, two 4K HEVC inputs via the ffmpeg
libvmaf_cudafilter, vmaf + vmaf_neg models): adding all three features costs ~8% vs models-only. The extractors launch on their own CUDA stream and overlap with the model extractors' work instead of extending the shared picture stream's critical path; picture lifetime is handled by recording aconsumedevent after the last picture-reading kernel and making both picture streams wait on it, so the pool's existingfinished-event reuse protocol transitively covers the own-stream reads. compute-sanitizer racecheck reports 0 hazards (run with per-frame host synchronization in place of the two-slot pipelining — racecheck cannot retire its tracking state without host syncs and runs out of memory on the pipelined variant; the cross-stream handshake under test is identical). The pipelined variant is additionally verified deterministic across repeated runs and bit-identical to the serialized implementation. One subtlety worth reviewer attention: the work streams are deliberately createdCU_STREAM_DEFAULT(legacy-blocking) — producers like the ffmpeg filter fill device pictures with synchronous-API D2D copies that queue on the legacy NULL stream without blocking the host, and only blocking-flavor streams are implicitly ordered after NULL-stream work. With a non-blocking stream, ~4% of frames scored nondeterministically through the ffmpeg path; blocking flavor restores the ordering while still overlapping with the other extractors' created streams.Supporting changes:
VMAF_FEATURE_EXTRACTOR_CUDA_CHROMAflag: the host→device picture upload (translate_picture_host) copied only the luma plane because all existing CUDA extractors are luma-only. The upload plane mask now widens to all planes only when a registered CUDA extractor declares it reads chroma — existing configurations upload exactly what they did before. The device→host direction (translate_picture_device) now downloads all planes unconditionally, since host pictures always carry every plane and CPU extractors that read chroma (psnr,ciede) expect them.flushthat drains their async score callback, so the final frame's score is in the collector before scores are read after flushing the context.test_cuda_psnr_ssim_paritydrives deterministic 8/10/12/16-bit YUV420P/YUV422P/YUV444P frames through the CPU and CUDA extractors via the public API. It asserts bit-exact PSNR, MSE, SSIM and L/C/S scores, compares APSNR aggregates, covers odd dimensions and automatic/explicit scales, and exercises the remaining scoring options.test_cuda_ciede_paritycovers YUV420P and YUV422P in both 8-bit and high-bit-depth paths, asserts the 1e-3 tolerance, and checks the identical-frame +inf case. Both exit 77 (meson SKIP) when no CUDA device is available, so CI without a GPU reports them as skipped rather than silently passing. Dispatch assertions intest_feature_extractor.care extended to the new extractors.Naming follows the existing convention (
vif_cuda/motion_cuda/adm_cuda): unique.namefor--feature psnr_cuda/ ffmpegfeature=name=psnr_cuda, withprovided_featuresmatching the CPU extractors so model-driven flag-aware selection works. The option tables mirror the CPU extractors' tables, following the existing CUDA extractors' pattern (they embedoffsetofinto each extractor's own state struct) — happy to factor the scoring math into a shared header instead if you'd prefer.Notes:
masteratf85a8536. cuda: fix row stride in calculate_motion_score_kernel_16bpc (wrong scores for >8-bit input) #1552 has landed and its duplicate commit has been removed; this branch remains stacked on cuda: don't flush CUDA feature extractors twice when a thread pool is present #1553 for correct CUDA operation with a thread pool.libvmaf_cudafilter I found thatvf_libvmaf.c'scopy_picture_data_cudabreaks out of its plane loop after plane 0, so chroma never reaches libvmaf and chroma-reading features report the psnr_max cap. That is an ffmpeg bug, not a libvmaf one (I patch it locally and plan to report it there), but it explains why chroma features through the filter need an ffmpeg fix to be useful.Tested on RTX 5060 Ti (sm_120), driver 595.71.05, CUDA 12.9/13.3.