Skip to content

fix(core): harvest and fix nine stale upstream Netflix/vmaf reports (ADR-1166) - #1223

Merged
lusoris merged 8 commits into
masterfrom
fix/upstream-harvest-2026-09-03
Sep 4, 2026
Merged

fix(core): harvest and fix nine stale upstream Netflix/vmaf reports (ADR-1166)#1223
lusoris merged 8 commits into
masterfrom
fix/upstream-harvest-2026-09-03

Conversation

@lusoris

@lusoris lusoris commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Harvests a batch of long-open Netflix/vmaf reports, verifies each one against this tree rather than trusting the report, and fixes the subset that still bites. The fork diverged far enough (ADR-0700's libvmaf/core/ rename, several C-to-C++ conversions, four fork-added GPU backends) that an upstream issue is neither automatically applicable nor automatically stale — three of these were already fixed here, two were never applicable, and several are wider here than upstream because a fork-added backend or SIMD path copied the defective shape.

Three of the fixes close memory-safety defects that are reachable today from the public C API with supported input. The complete triage table — including the ALREADY-FIXED and NOT-APPLICABLE verdicts, which are the expensive ones to re-derive — is in docs/research/1166-upstream-issue-harvest-2026-09-03.md.

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

What is fixed, and the test that proves it

Upstream ref What was wrong here Regression test
Netflix/vmaf#1582 Two OOB accesses in the float convolution. (a) convolution_edge_s / _sq_s / _xy_s bounced an out-of-range reflect-101 tap once, which only lands in range for size >= radius + 1; at size 2 a tap of −2 folds to +2 and +3 folds to −1 (heap-buffer-overflow READ). (b) convolution_x_c_s / _y_c_s derived borders_right = dim - (filter_width - radius), negative for a plane narrower than the filter, so the trailing loop started at a negative index and wrote dst[i * dst_stride - 1] (heap underflow WRITE). Reachable via --feature float_vif on any 9..15 px frame and via --feature float_motion with motion_add_uv on a 4x4 YUV420P frame. core/test/test_convolution_edge_small.c — planes embedded in NaN-poisoned buffers; an escaping tap taints the output, an escaping write replaces a poison NaN. Fails on the pre-fix tree, verified by stashing the two source files and rebuilding. test_large_plane_bit_identical additionally asserts bit equality against an explicit single-bounce reference at 24x24, so nothing in contract moved.
Netflix/vmaf#1581 Same mirror. The chroma half is the live CPU path above: motion_check_min_dim() validated luma only while motion_blur_plane() runs per plane at ref_pic->w[c] / h[c]. core/test/test_motion_min_dim.c::test_float_motion_add_uv_chroma_guardmotion_add_uv at 4x4 and 3x3 now -EINVAL, 5x5 and 576x324 still succeed.
Netflix/vmaf#1580 Mostly already fixed by Research-0094's min-dim guards. Audit of all 15 motion extractor TUs: 12 guarded, 3 not — exactly the fork-added Metal ones, written after that sweep and shipped (feature_extractor.cpp:229/232/234). core/test/test_motion_min_dim.c::test_metal_motion_min_dim — host-side rejection, so it needs no Apple GPU; degrades to a no-op without HAVE_METAL.
Netflix/vmaf#1242 vmaf_model_feature_overload() returned -ENOMEM past the unconditional free (leaking the caller's dict); the collection wrapper discarded the copy error, leaked the partial copy, skipped remaining sub-models and could still return 0, and dereferenced *model_collection unchecked. <libvmaf/feature.h> and <libvmaf/model.h> documented opposite ownership rules — one reading is a latent CWE-415. core/test/test_model_feature_overload_ownership.c — pins the guard paths, the success path, and drives the merge-failure branch deterministically (a heap-allocated empty dict makes vmaf_dictionary_merge() return NULL, which is exactly the branch that leaked; LSan reports it in the ASan lane).
Netflix/vmaf#743 The CLI wrote UTF-8 braille + \033[K to stderr with a byte-oriented fprintf while nothing in the tree ever set the console code page or enabled VT — mojibake under cp437/cp1252, replacement boxes under cp936, and a literal ←[K on legacy conhost, every frame of every run. core/test/test_spinner.cpp — drives the selectors with the code pages a real conhost reports (437/1252/936/0), asserts the ASCII fallback and the VT gate, and pins the braille table's first/last entries byte-for-byte plus strlen == 6 for all 56 so POSIX output cannot drift.
Netflix/vmaf#1551 (retracts #1422) The MSVC __builtin_clz shim used __lzcnt, which emits F3 0F BD with no runtime gate; on x86-64 without ABM the prefix is ignored and it retires as BSR, returning the MSB index instead of the leading-zero count. Two of the four call sites are on the generic scalar path, so an MSVC build silently mis-normalised every VIF and ADM log2 — a 2048-LSB error, i.e. a factor of two in the VIF fixed point — with no fault and no CI signal (every hosted Windows runner has LZCNT). scripts/ci/check-msvc-clz-shim.sh (registered as a fast meson test) — fails on the pre-fix header with four findings, verified by stashing the file; also scans the rest of core/src. Plus core/test/test_compat_clz.c, which unit-tests the 31 - msb arithmetic the __lzcnt form got wrong.
Netflix/vmaf#1178 libvmaf.pc carried Libs.private: -pthread -lm and no C++ runtime, so linking the static archive failed with hundreds of undefined references to operator new / std::ios_base::ios_base(). More exposed here than upstream: the C++ symbols come from the fork's own converted TUs, not just vendored libsvm. ADR-0198's static FFmpeg reproducer had to add -lstdc++ by hand for exactly this reason. The libvmaf-build-matrix "Verify static pkgconfig" step no longer greps the flag list — it compiles and links a C consumer with the C driver against exactly what pkg-config --static --libs libvmaf reports, which is the test that reproduces the downstream FFmpeg failure.
Netflix/vmaf#1573 (b) nvcc fatbin includes were relative, which only resolve when the build dir is a direct child of core/; since ADR-0700 the documented layout put it elsewhere and every .cu failed with fatal error: cuda/integer_adm_cuda.h: No such file or directory. (c) Three shell-driven tool tests declared no depends, so a subset run built nothing and died with exit 127. Hunk (a) is already fixed in-tree — do not re-port. Reproduced on a freshly configured, uncompiled CUDA build dir; the include fix is exercised by the CUDA fatbin build that make test-netflix-golden triggers on this workstation.

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally. pre-commit run --files <48 changed files> exits 0. Every file this PR touches is at or below its clang-tidy baseline; the new files and both edited test files measure 0 warnings. See "Known follow-ups" for pre-existing ratchet drift on master that this PR does not cause.
  • Unit tests pass: meson test -C build --suite=fast -j 4110/110 OK.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. — no cross-backend run needed: the only GPU-side change is the three Metal init() dimension guards, which reject before any kernel dispatch and cannot move a score. No CUDA/SYCL/HIP kernel is touched. The CPU convolution change is proven bit-identical for every in-contract size by test_large_plane_bit_identical.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md). scripts/ci/check-copyright.sh exits 0.
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. — not a breaking change; see "Breaking changes / migration" for the two behaviour changes.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to docs/adr/_index_fragments/_order.txt. scripts/docs/concat-adr-index.sh --check exits 0.

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR with a row in the appropriate section.

Seven rows in Recently closed (one per fixed issue), nine in Open bugs (the confirmed-but-deferred set plus the Metal motion_v2 mirror off-by-one found while triaging Netflix/vmaf#1580), and six in Confirmed not-affected — including two claims from the reports themselves that are refuted here and should not be re-investigated (Netflix/vmaf#818's "pooling silently falls back to mean", and Netflix/vmaf#1494's nvd/adm_ref_display_height premise, which the existing guard at integer_adm.c:3509 already rejects).

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception. — no golden value changes.

CUDA_VISIBLE_DEVICES= make test-netflix-golden271 passed, 12 skipped, and the three canonical CPU pairs re-scored on the built binary:

src01_hrc00_576x324      vs src01_hrc01_576x324        VMAF mean = 76.66744
checkerboard ..._0_0     vs ..._1_0   (1-px shift)     VMAF mean = 35.070245
checkerboard ..._0_0     vs ..._10_0  (10-px shift)    VMAF mean = 7.985956

Cross-backend numerical results

No cross-backend run applies. The only device-side change is three Metal init() guards that return -EINVAL before any pipeline is created; no kernel, no numeric path.

<feature>                <cpu-vs-cuda-ULP>  <cpu-vs-sycl-ULP>  <cpu-vs-hip-ULP>
(not applicable — no GPU kernel touched)

Deep-dive deliverables (ADR-0108)

  • Research digestdocs/research/1166-upstream-issue-harvest-2026-09-03.md: the full triage table, per-candidate evidence, and the deferred set with its reasoning.
  • Decision matrixdocs/adr/1166-upstream-issue-harvest.md § Alternatives considered (five options: harvest-and-verify, ignore upstream, port blindly, report upstream only, one PR per issue).
  • AGENTS.md invariant notecore/src/feature/AGENTS.md (the iterative fold, the size <= 1 termination guard, the border clamp, the deliberate single-bounce divergence in the motion extractors, and the never-__lzcnt rule), core/tools/AGENTS.md (spinner table byte-stability and the WindowsConsoleGuard declaration-before-goto requirement), core/test/AGENTS.md (the ADR-1138 NULL carve-out and the run_tests() branch budget).
  • Reproducer / smoke-test command — pasted below under "Reproducer".
  • CHANGELOG fragmentchangelog.d/fixed/upstream-harvest.md; scripts/release/concat-changelog-fragments.sh --write run, --check exits 0.
  • Rebase notedocs/rebase-notes.md gained a section per touched upstream-mirror file, including the deliberate divergence from Keep motion filter mirror() in bounds for tiny frames Netflix/vmaf#1581 (upstream fixes mirror() so tiny frames can be scored; the fork errors out instead) and the DO-NOT-COPY marker against MSVC: Miscellaneous uncontroversial fixes Netflix/vmaf#1422.

Reproducer

# Build the way this PR was verified (matches the clang-tidy ratchet lane).
meson setup build core -Denable_cuda=false -Denable_sycl=false -Denable_dnn=disabled
meson compile -C build -j 8

# All the new regression tests, plus everything else in the fast suite.
meson test -C build --suite=fast -j 4          # 110/110 OK

# The two convolution defects in isolation. Stash
# core/src/feature/common/convolution{.c,_internal.h}, rebuild, and this
# fails at the first 5-tap case; restore and it passes.
./build/test/test_convolution_edge_small

# The MSVC clz shim guard. Stash core/src/feature/compat_builtin.h and this
# reports four findings and exits 1.
bash scripts/ci/check-msvc-clz-shim.sh

# The pkg-config static-link defect, end to end.
printf '#include <libvmaf/libvmaf.h>\nint main(void){VmafContext *c=0;VmafConfiguration f={0};return vmaf_init(&c,f);}\n' > /tmp/smoke.c
gcc /tmp/smoke.c -Icore/include build/src/libvmaf.a -pthread -lm -o /tmp/smoke   # pre-fix Libs.private: FAILS on operator new
PKG_CONFIG_PATH=$PWD/build/meson-private \
  gcc /tmp/smoke.c -Icore/include -Lbuild/src $(PKG_CONFIG_PATH=$PWD/build/meson-private pkg-config --static --libs libvmaf) -o /tmp/smoke   # post-fix: links

# Netflix golden gate.
CUDA_VISIBLE_DEVICES= make test-netflix-golden   # 271 passed, 12 skipped

Known follow-ups

Confirmed, deliberately not batched — each has a docs/state.md Open-bugs row with its evidence, and each needs its own PR because it either moves scores, changes CLI grammar, or needs hardware this workstation does not have:

Pre-existing clang-tidy ratchet drift on master, not caused by this PR. python3 scripts/ci/tidy-ratchet.py --lane cpu --build-dir build reports regressions in dict.cpp (+1), ort_backend.c (+2), feature_collector.cpp (+3), feature_extractor.cpp (+1), fex_ctx_vector.cpp (+1), log.cpp (+2) and test_ort_internals.c (+7), and stale-high slack in convolution.c (−2), float_motion.c (−5), integer_vif.c (−14) and vif_tools.c (−26). None of these files' warning counts changed with this PR — verified by measuring the same files with the changes stashed — and the flagged diagnostics are unrelated to anything here (e.g. log.cpp's two are assert()-could-be-static_assert at log.cpp:154-155). This PR additionally improves test_motion_min_dim.c (15 → 0) and test_float_vif_min_dim.c (8 → 0). I deliberately did not run tidy-ratchet.py --write, because --write overwrites the whole baseline with the local measurement and would silently raise the entries for the pre-existing regressions — the documented flow is to commit the measurement CI itself uploads. If the CI ratchet asks for a tighten, that artifact is the right source.

Breaking changes / migration

Not a breaking change (no ABI change, no removed symbol, no renamed flag), but two input-validation surfaces became stricter. Both convert previously undefined behaviour into a documented -EINVAL:

  1. float_vif now requires 16x16, not 9x9. The four-scale ladder halves the dimension per scale and re-convolves with that scale's Gaussian, so the real floor is max over the ladder of ((filter_width_s / 2) + 1) << s = 16 at the default vif_kernelscale. Input in 9..15 px previously passed the guard and read out of bounds at scale 3; those runs were reading uninitialised memory and their scores were not meaningful. The bound is derived from vif_kernelscale, so a non-default kernel scale moves it. Documented in docs/metrics/vif.md § Minimum frame size.
  2. float_motion with motion_add_uv=true now validates the chroma planes. In 4:2:0 that means 5x5 minimum (chroma 3x3) rather than 3x3; 4:2:2 needs 5x3, 4:4:4 is unchanged at 3x3. The error message names the failing plane. Documented in docs/metrics/motion.md § Input format constraints.

Also user-visible, but not a restriction: on Windows the CLI now switches the console to UTF-8 + VT for the duration of the run and restores the previous state on exit, falling back to an ASCII spinner when the console refuses. POSIX output is byte-identical. Documented in docs/usage/cli.md § Windows console output.

The VmafFeatureDictionary ownership contract is now written the same way in all three public headers. It describes what the implementation has always done, so no working caller changes behaviour — but a caller that followed <libvmaf/model.h>'s previous wording ("ownership transfers even on a non-zero return") and therefore never freed after an -EINVAL guard rejection was leaking, and one that followed <libvmaf/feature.h>'s wording literally was risking a double free on the non-guard paths. ADR-0806 is marked Superseded by ADR-1166.

🤖 Generated with Claude Code

no ffmpeg-patches update needed: the three public headers this PR touches
(core/include/libvmaf/feature.h, libvmaf.h, model.h) change
doc-comments only — the VmafFeatureDictionary ownership contract wording,
which previously stated opposite rules in two headers. Verified with
git diff -U0 origin/master..HEAD -- core/include/ filtered to non-comment
lines: zero changed lines outside comments, and zero VMAF_EXPORT,
signature, struct, enum or #define changes. No entry point is added,
renamed or removed, no LIBVMAFContext field changes, and no
check_pkg_config probe symbol moves, so nothing in ffmpeg-patches/
consumes anything new. The VmafContext / VmafModel mentions the gate
matched are the unchanged signatures those comments sit above.

@lusoris

lusoris commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Independent adversarial review — 20 findings

A multi-agent review attacked this PR's claims rather than confirming them, with a
separate refuter per finding that discarded 6 of the 20 it judged. What follows survived.
Everything is read-only analysis with line references; nothing was built.

[HIGH] core/src/feature/metal/float_motion_metal.mm:124

Metal motion min-dim guard of 3 is far too small for the kernels' fixed 20x20 halo tile — 4x4..9x9 frames still read deeply out of bounds

Trigger: Any Metal run of motion_metal / float_motion_metal / motion_v2_metal on a frame whose width or height is in [3,9] (and, for the two v1 extractors, also exactly 17). Concrete: a 4x4 8-bit YUV420p pair through --feature float_motion_metal. Guard passes (4 >= 3), then the kernel's phase-1 tile load computes sx = sy = skip_mirror(17, 4) = 2*(4-1)-17 = -11 and reads ref[-11*row_bytes - 11], i.e. 55 bytes before a 16-byte MTLBuffer.

Why: The threshold 3 is derived from the scalar CPU convolution, where the only out-of-range indices are i-2..i+2 around a real pixel, so the worst mirrored index is dim+1 and one bounce suffices (float_motion.c:285, integer_motion.c:272). The Metal kernels are not scalar: float_motion.metal:83-90, integer_motion.metal:83-90 and integer_motion_v2.metal:106-113 unconditionally cooperatively load a fixed TILE_W x TILE_H = 20x20 halo tile (n_elems = 400, wg_size = 256, dispatchThreadgroups with a uniform 16x16 threadgroup — float_motion_metal.mm:254-256), so tx/ty run 0..19 and the raw index passed to the single-bounce mirror runs from -2 up to (ceil(dim/16)-1)16+17, regardless of how small the frame is. skip_mirror (float_motion.metal:49-53, integer_motion.metal:48-52) returns 2(sup-1)-idx and mv2_mirror (integer_motion_v2.metal:51-56) returns 2sup-idx-1; neither loops, and neither result is clamped before the global read (float_motion.metal:90 and :187 row_ptr[sx], integer_motion.metal:90/:190, integer_motion_v2.metal:110-113 prev[gy*prev_stride+gx]). Solving for a non-negative mirrored index: v1 needs 2dim >= 16ceil(dim/16)+3, i.e. dim >= 10 for dim<=16 and dim=17 also fails; v2 needs 2dim >= 16*ceil(dim/16)+2, i.e. dim >= 9. So the guard leaves the whole band it was added to close: 3..9 (plus 17) for float_motion_metal.mm:124 and integer_motion_metal.mm:132, and 3..8 for integer_motion_v2_metal.mm:228. The PR's claim of parity with the twins is literally true — CUDA (float_motion/float_motion_score.cu:33-72, integer_motion) and HIP (integer_motion/motion_score.hip:46-108) use the same 20x20 tile with the same one-bounce mirror and the same wrong constant 3 — but that means the constant was copied rather than re-derived for a tiled kernel, and the OOB the ADR text says is now prevented is not prevented on any GPU backend.

[HIGH] core/tools/vmaf.cpp:1116

Stray } // namespace inside the _WIN32 block breaks the Windows build of the CLI

Trigger: Any build with _WIN32 defined — the required Build — Windows MinGW64 (CPU) leg (.github/workflows/libvmaf-build-matrix.yml:647, meson install -C core/build) compiles core/tools/vmaf.cpp with MinGW g++, which defines _WIN32. Also every MSVC/clang-cl build of the vmaf tool.

Why: The new Windows block runs from #ifdef _WIN32 at line 1058 to #endif /* _WIN32 */ at line 1117. Line 1116, immediately after the closing }; of class WindowsConsoleGuard, is } // namespace — but no namespace is open at that point. git show origin/fix/upstream-harvest-2026-09-03:core/tools/vmaf.cpp | grep -n namespace yields exactly three hits: 1116 } // namespace, 1119 namespace, 1191 } // namespace. The only opening namespace { is at 1119–1120, and it is outside the #ifdef; none of the three included headers (core/tools/cli_parse.h, vidinput.h, spinner.h) contains the token namespace at all. So on POSIX the braces balance (1119 opens, 1191 closes), but when _WIN32 is defined there are two closers for one opener: line 1116 is a bare } at file scope, which is a syntax error (expected declaration before '}' token / MSVC C2059). The fix for a Windows-only rendering bug therefore does not compile on Windows.

[HIGH] core/src/feature/float_motion.c:221

float_motion chroma buffers are allocated with floor(h/2) rows while the PR's new guard and picture.c use ceil((h+1)/2) — odd-height 4:2:0 + motion_add_uv overflows every chroma buffer by one row

Trigger: vmaf --pixel_format 420 --width 64 --height 65 --feature float_motion=motion_add_uv=1 (or the 5x5 case the PR's own new test asserts must succeed). Any YUV420P input with an ODD luma height and motion_add_uv=1. Golden fixtures (324, 1080) are even, so the gate does not see it.

Why: The PR adds motion_chroma_shifts (float_motion.c:299-318) and derives the guard's chroma dimensions as (h + ss_ver) >> ss_ver at line 346, explicitly citing picture.c's geometry — which is ceiling division (picture.c:148, pic->h[1] = (h + ss_ver) >> ss_ver, added by Research-0094 precisely to stop one-row under-allocation). But the untouched neighbour motion_chroma_heights at lines 216-233 still returns *h_u = h / 2 (floor), and init() at line 385/394 feeds that value to motion_plane_alloc, which sizes ref/tmp/blur[0..1] as float_stride * h_u. For odd h the two disagree by one row: h=65 -> guard validates a 33-row chroma plane and passes, allocation is 32 rows. extract() then calls picture_copy(..., channel=1) (picture_copy.cpp:88-94 loops i < src->h[1] = 33) and motion_blur_plane(s, &s->plane[c], ref_pic->w[c], ref_pic->h[c], ...) (float_motion.c:493) at 33 rows, so a full row (float_stride bytes) is written past the end of every chroma buffer, and motion_score_pair reads it back. The minimum trigger, h=5, is the exact configuration test_motion_min_dim.c:187-188 asserts init(5x5) must succeed (3x3 chroma) — the test stops at init()/close() and never reaches extract(), so the new regression test certifies the overflowing case as good. The PR's claim that float_motion now "validates every plane it will actually convolve" is true of the guard but not of the allocator it guards.

[MEDIUM] /home/kilian/dev/vmaf/core/src/feature/compat_builtin.h:75

Architecture guard excludes MSVC ARM64, where the shim is needed and _BitScanReverse does exist — the leg still cannot compile

Trigger: Build the fork with cl.exe targeting ARM64 (vcvarsall.bat amd64_arm64, a windows-11-arm runner, or a Windows-on-ARM host). _M_ARM64 is defined, __clang__ is not, _M_X64/_M_IX86 are not, so the whole block at lines 75-99 is skipped and no __builtin_clz exists. core/src/feature/integer_vif.h:148, :159 and core/src/feature/integer_adm.c:989 then fail with MSVC C2065 'undefined identifier'. There is no MSVC ARM64 leg in .github/workflows/build.yml or libvmaf-build-matrix.yml, so CI never sees it.

Why: The guard's stated purpose (compat_builtin.h:50-52 "__lzcnt and _BitScanReverse are x86-only, so an MSVC ARM64 leg previously failed to compile"; scripts/ci/check-msvc-clz-shim.sh:16-17 and :52-53 "an MSVC ARM64 leg (where neither intrinsic exists)"; docs/state.md:349 "neither intrinsic exists on MSVC ARM64") is factually wrong. MS Learn's _BitScanReverse Requirements table lists _BitScanReverse for "x86, ARM, x64, ARM64" and _BitScanReverse64 for "ARM64, x64", and the ARM64-intrinsics page lists both under "ARM64 support for intrinsics from other architectures". Only __lzcnt was x86-only. So the guard does not make ARM64 compile — it just moves the error from '__lzcnt undefined' to '__builtin_clz undefined' (MSVC provides no GCC __builtin_* clz of its own; that is the entire reason the shim exists, and the !defined(__clang__) term shows the author knows only clang-cl supplies it). The correct guard is _M_X64 || _M_IX86 || _M_ARM64 || _M_ARM, with the #if defined(_M_X64) at line 88 widened to _M_X64 || _M_ARM64 so ARM64 takes the _BitScanReverse64 path and _M_ARM/_M_IX86 take the 32-bit fallback. Note the check at scripts/ci/check-msvc-clz-shim.sh:54 actively locks the wrong guard in: a correct _M_ARM64-inclusive guard still passes it, but the script's own message tells maintainers the x86-only form is the required shape.

[MEDIUM] /home/kilian/dev/vmaf/.github/workflows/build.yml:469

test_compat_clz never runs on any MSVC leg, so no CI job ever executes the shim it was written to protect

Trigger: Break the shim body only — e.g. change compat_builtin.h:82 to return (int)idx; or drop the found test in vmaf_compat_clz32_from_msb's caller. Every CI job stays green: the Linux/macOS/MinGW legs that run meson test compile with GCC/Clang, where the _MSC_VER && !__clang__ block at line 75 is preprocessed away and __builtin_clz is the compiler builtin; the only MSVC job that runs any test executes a hardcoded list at build.yml:469-473 (test_dict test_ref test_log test_thread_pool test_cpu test_feature test_cli_parse test_model test_predict test_ciede test_cambi test_psnr test_luminance_tools test_feature_collector test_framesync) that does not include test_compat_clz; and the MSVC+CUDA / MSVC+oneAPI legs in libvmaf-build-matrix.yml run no tests at all ("No test step — windows-2025 has no GPU").

Why: core/test/test_compat_clz.c:132 asserts "On the MSVC CI legs this exercises the shim itself" — that is false for every leg in the repo. The only things CI actually verifies about the shim are (a) that it compiles on MSVC x64 (via ninja install, which builds the test targets) and (b) the textual grep in scripts/ci/check-msvc-clz-shim.sh. The runtime assertions in test_builtin_clz_matches_reference (lines 134-167) exercise the GCC/Clang builtin everywhere they run, never the MSVC bodies. Adding test_compat_clz to the build.yml:469 list is a one-word fix and would make the regression test actually regression-test.

[MEDIUM] core/src/feature/metal/float_vif_metal.mm:264

Same PR raises the CPU float_vif floor 9 -> 16 but leaves the Metal VIF twins at an effective floor of 8, where the scale-0 17-tap mirror indexes with a wrapped uint

Trigger: A 12x12 (any 8..15) frame with --feature float_vif_metal (identically integer_vif_metal). float_vif.c:212-218 now rejects it with -EINVAL (vif_get_min_dim(1.0) = 16), while the Metal guard only requires scale_w[3] = 12/2/2/2 = 1 != 0, so init succeeds. At scale 0 hfw=8, tile_ox = -8, tc runs 0..31 (float_vif.metal:178-190), so fvif_mirror(23, 12) = 2*12-23-2 = -1 and fvif_read_raw does plane[(uint)y * stride_bytes + (uint)x] (float_vif.metal:116-121) — the (uint) cast turns px = -1 into 0xFFFFFFFF, a ~4 GB offset read.

Why: The review brief asks which Metal extractors that convolve neighbours still lack an adequate guard. float_vif_metal.mm:264 and integer_vif_metal.mm:266 only check that the scale-3 dimension is non-zero, which admits any dim >= 8; the 17-tap scale-0 Gaussian needs dim >= 16 by exactly the derivation this PR adds in vif_tools.c::vif_get_min_dim, and the scale-3 3-tap pass at dim/8 = 1 is worse still (fvif_mirror(16,1) = -16). The PR fixed the CPU side of #1582 and left every GPU VIF twin behind — CUDA's float_vif_cuda.c init_fex_cuda has no dimension check at all — so after this PR a 12x12 input is rejected on CPU and faults on Metal. The three motion extractors were guarded; the two VIF extractors, which have a strictly larger footprint, were not.

[MEDIUM] core/tools/vmaf.cpp:1401

WindowsConsoleGuard is constructed before cli_parse, so --help/--version/usage errors exit() without restoring the console

Trigger: On Windows, vmaf --help, vmaf -v/--version, or any argument error (vmaf with no -r, a bad --backend, a bad --model string, etc.) run in cmd.exe/conhost. The console is left at code page 65001 with ENABLE_VIRTUAL_TERMINAL_PROCESSING set after the process exits, affecting every later program in that console window.

Why: const WindowsConsoleGuard console_guard; is declared at line 1401 and its constructor unconditionally calls SetConsoleOutputCP(CP_UTF8) and SetConsoleMode(stderr, mode|ENABLE_VIRTUAL_TERMINAL_PROCESSING) (lines 1084–1092). Only then, at line 1405, does main call cli_parse(argc, argv, &c). cli_parse never returns on a large fraction of invocations: core/tools/cli_parse.cpp:297 exit(reason ? 1 : 0); inside the [[noreturn]] usage() (reached from cli_parse.cpp lines 341, 425, 450, 469, 480, 510, 536, 543, 626, 657, 692, 912 (--help), 994, 1019, 1024, 1027, 1037, 1041, 1045, 1059, 1069) and cli_parse.cpp:922 exit(0); for -v/--version. C++ exit() does not destroy objects with automatic storage duration ([basic.start.term]), so ~WindowsConsoleGuard() — the only restore path — never runs. There is no atexit fallback: grep -n 'exit(\|atexit' core/tools/vmaf.cpp on this branch returns nothing at all. The guard is also not gated on istty (computed at line 1396 but unused here), so even vmaf --version 2>NUL from a script permanently mutates the console. The comments at lines 1399–1400 ("restore it on every exit path") and 1071–1073 are therefore wrong for these paths; only the goto cleanup spine and the normal return are actually covered.

[MEDIUM] core/include/libvmaf/feature.h:52

feature.h + docs/api tell callers to free the dictionary on an unknown feature name — for the two model overloads that is a double free

Trigger: Third-party caller follows the new contract literally: VmafFeatureDictionary *d = ...; if (vmaf_model_feature_overload(model, "VMAF_integer_feature_adm2_score", d)) {...} — i.e. passes a model-JSON feature name instead of the feature-extractor name "adm" (the same confusion the fork's own core/test/test_model_collection_api.c:274 makes, passing "VMAF_feature_adm2"). The call returns 0 with the dictionary already freed; feature.h:52-53 and docs/api/index.md:293-294 say to free it → free() of a freed pointer (CWE-415).

Why: core/src/model.c:247-253 skips every non-matching feature and falls through to the unconditional err |= vmaf_dictionary_free((VmafDictionary **)&opts_dict); at line 273, returning 0 — so an unknown feature_name CONSUMES the dictionary and never returns -EINVAL. feature.h:37-40 asserts the rules are "identical for @ref vmaf_use_feature, @ref vmaf_model_feature_overload and @ref vmaf_model_collection_feature_overload", then feature.h:42-46 puts "@p feature_name names no registered feature" in the caller-still-owns set, and feature.h:52-53 states it unconditionally: "free the dictionary yourself only when you passed a NULL argument or an unknown feature name". docs/api/index.md:281-294 repeats both. That carve-out is true ONLY for vmaf_use_feature (core/src/libvmaf.c:1618-1620 returns -EINVAL before touching the dict). model.h:169-178 correctly restricts the carve-out to NULL @p model / @p feature_name / @p opts_dict and says nothing about unknown names — so feature.h and model.h STILL disagree about one case, which is exactly the divergence class this PR set out to remove. The -EINVAL precondition in the feature.h bullet saves a careful reader, but the 'In practice' summary line is unconditional and is what a reader will copy.

[MEDIUM] core/src/libvmaf.c:1624

vmaf_use_feature does NOT consume the dictionary on the copy/-ENOMEM path, but libvmaf.h + feature.h now promise it does

Trigger: vmaf_use_feature(ctx, "psnr", opts) where vmaf_dictionary_copy() fails (strdup/malloc failure inside vmaf_dictionary_set — the only way it returns non-zero once opts_dict is non-NULL). Return value is -ENOMEM. A caller obeying the new libvmaf.h:278-280 ("On any other return — success or failure — the dictionary has already been released internally and the caller MUST NOT free it") never frees opts → the caller's dictionary and the partial copy are both leaked. Reproducible under an allocation-failure injector / ASan+LSan with malloc fault injection.

Why: libvmaf.c:1624-1626 is err = vmaf_dictionary_copy(&s, &d); if (err) return err; — it returns before line 1627's vmaf_dictionary_free(&s), so the caller's dictionary s (== opts_dict) is still allocated and unreachable, and the partially built copy d leaks too. This is the same defect shape the PR fixed in model.c:256-266, but libvmaf.c is not touched by the diff. The rewritten docs assert the opposite: feature.h:47-50 ("Every other path consumes it … including -ENOMEM from the merge/copy step"), libvmaf.h:273-281, docs/api/index.md:289-291 and the example comment at docs/api/index.md:301, plus docs/api/index.md:312-314 ("All three headers now state the rule above, and it matches what the implementation has always done") and model.h:175-176 ("This matches @ref vmaf_use_feature"). Under the OLD libvmaf.h wording ("Use vmaf_feature_dictionary_free() only in the case of failure") a caller handled this path correctly; the new wording turns it into a documented leak. The research digest (docs/research/1166-upstream-issue-harvest-2026-09-03.md:166-200) only audits model.c, never vmaf_use_feature's copy path, yet the header for that function was rewritten.

[MEDIUM] core/src/feature/float_motion.c:346

float_motion motion_add_uv guard validates ceil(h/2) chroma rows but motion_chroma_heights still allocates floor(h/2) — odd-height 4:2:0 overflows the chroma planes

Trigger: C API (not CLI): vmaf_picture_alloc(&pic, VMAF_PIX_FMT_YUV420P, 8, 6, 5) with --feature float_motion:motion_add_uv=1. Any YUV420P frame with ODD luma height >= 5. Verified arithmetic: guard sees chroma 3x3 and passes; allocation is 2 rows (64 B); picture_copy's last write ends at byte 76 — 12 B past the end of each of ref, tmp and blur[0..2] for both U and V. Even heights (6x6, 576x324) are unaffected.

Why: Line 346 computes ch = (h + ss_ver) >> ss_ver, i.e. the CEILING geometry, and the comment on line 344 explicitly cites picture.c:147-149 as the source. That is the right value — it matches ref_pic->h[1] which is what motion_copy_and_blur actually convolves at line 494. But the buffer for that plane is sized by motion_chroma_heights at line 221, which is unchanged by this PR and still uses FLOOR: *h_u = h / 2. Line 394 passes that h_u to motion_plane_alloc. For odd h the two disagree by one row, so the guard passing does NOT imply the convolution stays in bounds: picture_copy (picture_copy.cpp:88-92, loops i < src->h[channel]) writes ceil(h/2) rows into a floor(h/2)-row buffer, and motion_blur_plane (line 494) then reads p->ref and writes p->blur[idx] for the same out-of-range row. That is a heap-buffer-overflow WRITE, i.e. strictly worse than the OOB read this PR set out to fix. This directly falsifies claim (b) — 'float_motion now validates every plane it will convolve': it validates the correct dimensions but the plane that will be convolved is not the plane that was allocated. The new test codifies the unsafe case: test_motion_min_dim.c:188 asserts invoke_init_add_uv(fex, 5u, 5u) == 0, and 5x5 is precisely an odd-height 420 frame whose chroma allocation is one row short. (The allocator mismatch predates this PR; what is new is a guard and a test that both declare the case safe. Fix: make motion_chroma_heights use (h + 1) / 2 for YUV420P so it agrees with picture.c and with the new line 346.)

[MEDIUM] core/src/feature/common/convolution_avx.c:177

Border-bound clamp applied only to the scalar path; the AVX2/AVX-512 twins convolution.c dispatches to on x86 still start the trailing vertical border loop at a negative row and write out of bounds

Trigger: x86 build with AVX2 (i.e. every CI runner and the dev workstation), float_motion configured with motion_filter_size=1 (option table min=0, max=9, float_motion.c:161-171), on a frame of height 1 — e.g. vmaf_use_feature(ctx, "float_motion", opts_with_mfs_1) with an 8x1 YUV400P/YUV420P picture (vmaf_picture_alloc accepts any h >= 1, picture.c). motion_check_min_dim returns 0 without checking anything because effective_filter_size == 1 fails the if (effective_filter_size > 1u) gate, yet motion_blur_plane (float_motion.c:476-484) still convolves with filter_size = 5 (FILTER_5_NO_OP_s = {0,0,1,0,0}, motion_tools.h:29), so radius = 2 > height = 1.

Why: convolution_f32_c_s (convolution.c:114-120) returns early into convolution_f32_avx_s whenever VMAF_X86_CPU_FLAG_AVX2 is set, so on x86 the newly clamped convolution_x_c_s / convolution_y_c_s are never executed at all. convolution_f32_avx_s derives its own vertical split as int i_vec_end = height - radius; (convolution_avx.c:153) with no clamp — the exact defect (b) the PR fixes in convolution.c. With height=1, radius=2 this is -1, so the trailing border loop for (int i = i_vec_end; i < height; ++i) at convolution_avx.c:177 runs with i = -1 and executes tmp[(ptrdiff_t)i * tmp_pdt + j] = ..., writing one full row before the buffer. motion_plane_alloc (float_motion.c:246-251) allocates p->tmp as exactly float_stride * h bytes, and the AVX tmp_stride = vmaf_ceiln(width, 8) floats equals ALIGN_CEIL(w*4)/4, so that write lands squarely in the allocator header. The leading loop at convolution_avx.c:161 (for (int i = 0; i < radius; ++i)) symmetrically writes row 1 of a 1-row buffer. The same unclamped i_vec_end appears at convolution_avx.c:209 and 266 (_sq_s, _xy_s) and at convolution_avx512.c:194, 251, 308. The scalar path is genuinely fixed for this input (I traced h=1: borders_top clamps 2 -> 1, borders_bottom clamps -2 -> 1, both border loops stay in [0,1)), which is precisely why the SIMD gap is invisible in the new test — core/test/test_convolution_edge_small.c calls convolution_y_c_s / convolution_x_c_s directly (lines 159-160, 289-290) and never goes through convolution_f32_c_s, so it cannot reach the AVX path on any machine.

[MEDIUM] core/src/feature/common/convolution_avx.c:153

The #1582 border-bound clamp was added only to the scalar convolution; the AVX2/AVX-512 twins that convolution_f32_c_s actually dispatches to keep the identical unclamped negative bound

Trigger: vmaf --feature float_motion=motion_filter_size=1 on a frame whose luma (or, with motion_add_uv=1, chroma) plane is 1 row tall — e.g. --width 64 --height 1, or --width 2 --height 2 --pixel_format 420 --feature float_motion=motion_filter_size=1:motion_add_uv=1. Requires an AVX2-capable x86 CPU, i.e. the fork's own CI runners and dev workstation.

Why: convolution_f32_c_s (convolution.c:109-127) dispatches to convolution_f32_avx_s whenever VMAF_X86_CPU_FLAG_AVX2 is set and only falls back to the scalar convolution_y_c_s/convolution_x_c_s otherwise. The PR clamps the scalar bounds via the new convolution_clamp_borders (convolution.c:41-46, called at :55 and :84), but convolution_avx.c and convolution_avx512.c are byte-identical to master (git diff origin/master...HEAD -- core/src/feature/common/convolution_avx*.c is empty) and still compute int i_vec_end = height - radius; (avx.c:153, 209, 266; avx512.c:194, 251, 308) with no clamp. For height=1 and the 5-tap kernel (radius 2) i_vec_end = -1, so the trailing vertical loop for (int i = i_vec_end; i < height; ++i) at avx.c:177 writes tmp[-tmp_stride + j] — the same heap underflow WRITE the changelog says is fixed — and the leading loop at avx.c:160 writes row 1 of a buffer motion_plane_alloc sized at exactly float_stride * 1. The new regression test cannot see this: test_convolution_edge_small.c:159-160 and :289-290 call convolution_y_c_s/convolution_x_c_s directly, deliberately bypassing the SIMD dispatch, so the only path the fix covers is the only path the test exercises. docs/rebase-notes.md's convolution section likewise names only convolution.c and convolution_internal.h.

[MEDIUM] core/src/feature/float_motion.c:284

motion_check_min_dim skips the entire guard when motion_filter_size==1, while motion_blur_plane still convolves with a 5-tap kernel

Trigger: Any --feature float_motion=motion_filter_size=1 (alias mfs=1; the option's declared range is 0..9, float_motion.c:162-170) at any frame size, including 1x1 and 2x2, and with motion_add_uv=1 at any chroma size.

Why: motion_check_min_dim computes effective_filter_size = motion_filter_size and wraps the whole minimum-dimension test in if (effective_filter_size > 1u) (line 284), so mfs=1 returns 0 for every w/h. But motion_blur_plane (lines 476-485) only swaps the coefficients for mfs==1 — filter = FILTER_5_NO_OP_s — and leaves int filter_size = 5, so the convolution still runs with filter_width 5 and radius 2 and still dereferences taps at +-2 rows/columns. Since motion_check_min_dim is the single entry point of the new motion_check_min_dim_all_planes, mfs=1 also silently disables this PR's headline #1581 chroma-plane check. The comment above the function only accounts for motion_filter_size==0 (option not yet applied) and does not mention that ==1 is a total bypass. This is the enabling condition for the AVX finding above; on its own, post-fix, it is a robustness hole rather than a memory error, because convolution_reflect101 short-circuits at size<=1 and the scalar borders are now clamped. test_motion_min_dim.c contains no motion_filter_size coverage at all.

[MEDIUM] .github/workflows/libvmaf-build-matrix.yml:545

New static-link smoke uses bare cc while the archive is LTO-compiled by gcc-14 — GCC LTO bytecode is major-version-locked

Trigger: Both legs that reach this step: "Build — Ubuntu gcc Static (CPU)" (matrix lines 149-152) and "Build — Ubuntu CUDA Static" (161-166). Both set CC: ccache gcc-14 / CXX: ccache g++-14 in the job env: block (lines 652-654) and pass --default-library static with no -Db_lto=false. core/meson.build:11 puts b_lto=true in default_options, and only the two SYCL legs (workflow lines 176, 194) override it — so install/lib/libvmaf.a holds slim GCC-14 LTO objects (meson's GnuCompiler emits -flto=auto, never -ffat-lto-objects). The step then links with the literal command cc, i.e. /usr/bin/cc → the distro-default GCC. The workflow's own apt step says "gcc-14 is preinstalled on ubuntu-24.04" (line ~274) and still apt-installs it explicitly, i.e. gcc-14 is not the default; ubuntu-24.04's cc is gcc-13 (a roll to 26.04 makes it gcc-15 — either way ≠ 14).

Why: Line 545-546 hard-codes cc instead of $CC, which is already exported for the whole job. main() calls vmaf_init, so the linker pulls at least one LTO archive member out of libvmaf.a; gcc-13's liblto_plugin.so claims it and gcc-13's lto1 then hits GCC's LTO bytecode version check, which is fatal across compiler majors (lto1: fatal error: bytecode stream in file '...' generated with LTO version X.Y instead of the expected Z.W). The step is a plain run: (default bash -e), so this aborts the leg. The pre-existing three assertions on lines 526/527/531 are pure pkg-config | grep and never touched the compiler, so this failure mode is newly introduced by this hunk and is currently unproven: PR #1223 is a DRAFT (gh pr view 1223 --json statusCheckRollup returns []), and draft PRs on this repo skip the heavy legs, so no CI run has exercised it. Fix is one token — use $CC (or ${CC:-cc}); ccache gcc-14 word-splits correctly in this position. The same cc-based recipe was also copied into the user-facing doc at docs/development/build-flags.md:239, where it is only correct if the reader's cc happens to match the compiler that built the archive.

[MEDIUM] core/src/meson.build:2193

libc++ detection for Libs.private silently ignores -stdlib=libc++, the exact case its comment claims to handle

Trigger: CXX=clang++ meson setup build core -Dcpp_args=-stdlib=libc++ -Dcpp_link_args=-stdlib=libc++ --default-library static on Linux (equivalently CXXFLAGS=-stdlib=libc++). The build links libc++, but the generated libvmaf.pc gets Libs.private: ... -lstdc++. A downstream consumer following pkg-config --static --libs libvmaf then fails with undefined references to std::__1::* / libc++ operator new, i.e. the very failure Netflix/vmaf#1178 is about — just with the other STL.

Why: The inline comment at lines 2188-2190 asserts the probe beats upstream's compiler-id mapping because -Dcpp_args=-stdlib=libc++ "can flip either compiler" and _LIBCPP_VERSION detects the real STL. It cannot see that flag. cxx.get_define() runs the probe with mode=CompileCheckMode.PREPROCESS (mesonbuild/compilers/mixins/clike.py:629-631), and Compiler.build_wrapper_args injects coredata.get_external_args() only under if mode is CompileCheckMode.COMPILE: (mesonbuild/compilers/compilers.py:1588-1590), with elif mode is CompileCheckMode.LINK: for link args. PREPROCESS gets neither, so both the cpp_args built-in option and the CXXFLAGS env are dropped from the probe command line. With the flag gone, clang preprocesses against its default libstdc++, _LIBCPP_VERSION is undefined, the != '' test at line 2193 is false, and line 2196 appends -lstdc++. macOS/FreeBSD still work only because libc++ is the compiler default there and needs no flag. Related fragility in the same expression: <ciso646> was removed from C++20, and libstdc++'s copy #errors when _GLIBCXX_USE_DEPRECATED is 0 and #warnings otherwise at __cplusplus >= 202002L (/usr/include/c++/16/ciso646:44-52); get_define raises EnvironmentException('Could not get define ...') on a non-zero preprocess (clike.py:636-637), i.e. a hard configure abort, not a fallback. That is inert today only because PREPROCESS also drops -std=, so the probe always runs at the compiler's default gnu++17. Fixing the first half by passing args: would activate the second — the durable spelling is #include <cstddef> (or <version>) plus explicit args.

[LOW] /home/kilian/dev/vmaf/scripts/ci/check-msvc-clz-shim.sh:39

check-msvc-clz-shim.sh is textual and evadable by macro indirection, a different intrinsic spelling, or a file outside core/src

Trigger: Reintroduce LZCNT as #define VMAF_CLZ32 __lzcnt plus return (int)VMAF_CLZ32(x); inside compat_builtin.h. Check (1) at line 39 matches \b__lzcnt(64)?[[:space:]]*\( — the #define line has no ( after the token and the use site contains no __lzcnt token, so nothing matches; check (2) at line 47 is satisfied by the surviving comment block, which still says _BitScanReverse; check (3) at line 54 is satisfied by the unchanged guard line; check (4) at line 60 runs the same non-matching regex over core/src. The script prints "OK: MSVC clz shim uses _BitScanReverse and is architecture-guarded" and exits 0 with LZCNT back in the shipped MSVC binary.

Why: Three independent holes. (i) The regex requires the literal token __lzcnt/__lzcnt64 immediately followed by (, so any macro alias, a typedef'd function pointer, or a differently-spelled LZCNT-emitting intrinsic (_lzcnt_u32 / _lzcnt_u64, which contain no __lzcnt substring at all) passes. (ii) Check (2) greps the whole file including comments, so it proves only that the string appears somewhere, not that the implementation uses it — the header's own explanatory block at lines 45-52 satisfies it forever. (iii) Check (4)'s comment at line 59 says "Nothing else in the tree may reintroduce the intrinsic either", but line 60 scans only $ROOT/core/src; core/tools/, core/test/, core/include/ and ffmpeg-patches/ are unscanned, so the same intrinsic in core/tools/vmaf.cpp is invisible. A grep for the result — e.g. requiring the _BitScanReverse call to appear on a non-comment line inside the _MSC_VER block — would close (i)/(ii); widening line 60 to $ROOT closes (iii).

[LOW] core/test/test_vif_skip_scale0.c:97

Test comments still state the superseded ADR-0806 rule and cite the now-Superseded ADR without pointing at ADR-1166

Trigger: git grep 0806 after this PR: docs/adr/0806-feature-dictionary-ownership.md flips to "Superseded by ADR-1166" (that is the only change to it), but core/test/test_vif_skip_scale0.c:96-100 and core/test/test_integer_vif_cpu_cuda_parity.c:183-187 still carry ADR-0806's unconditional wording. A maintainer copying "takes ownership of opts and frees it internally, regardless of success or failure" to a call site whose feature name is unregistered or whose ctx is NULL leaks the dictionary — the exact case the new contract carves out.

Why: test_vif_skip_scale0.c:97-100 reads "vmaf_use_feature() takes ownership of opts and frees it internally, regardless of success or failure … See ADR-0806." That is the pre-fix contract; the new rule (feature.h:42-46) is that the argument-validation guards consume nothing. The code at both call sites happens to be correct ("vif"/"vif_cuda" are registered and the asserts require err == 0), so this is comment-only — but the PR's own claim (changelog.d/fixed/upstream-harvest.md:53-55, ADR-1166) is that the contract is now written identically everywhere, and these two in-tree comments are the residue that still asserts the old one.

[LOW] core/src/libvmaf.c:1633

vmaf_use_feature leaks its internal dictionary copy when the extractor context fails to parse options (pre-existing, not in the diff)

Trigger: vmaf --feature vif:vif_enhn_gain_limit=0.5 (or any out-of-range/unparseable option value; core/src/feature/integer_vif.c:79 sets .min = 1.0). vmaf_option_set returns -EINVAL → vmaf_fex_ctx_parse_options returns -EINVAL → vmaf_feature_extractor_context_create frees f/x/priv and returns -ENOMEM without releasing the opts_dict it was handed → the copy d (dict struct + entry array + key/val strdups) is leaked. No OOM needed; LeakSanitizer flags it on every run.

Why: core/src/feature/feature_extractor.cpp assigns f->opts_dict = opts_dict; and then, on the parse_options failure branch, takes goto free_x → free(x); free(f); return -ENOMEM — nothing ever calls vmaf_dictionary_free on opts_dict. Back in vmaf_use_feature, s (the caller's dictionary) was already freed at libvmaf.c:1627, so the public ownership contract is satisfied and the caller cannot recover the copy. This is an internal leak in the same ownership path the PR audited, and it also means the -ENOMEM return code is a lie about an -EINVAL condition. Pre-existing and untouched by this PR; flagged because the PR's premise is that this ownership area is now correct end to end.

[LOW] core/src/feature/float_vif.c:213

float_vif applies the ladder-derived minimum to the RAW input dimensions, which compute_vif never sees; with vif_prescale > 1 this rejects frames that are safe and that master scored

Trigger: --feature float_vif:vif_prescale=2.0 on a 12x12 frame (scaled to 24x24 — safe, dim_3 = 3 >= 2; master accepted and scored it, branch returns -EINVAL). Worse with a large kernelscale: vif_kernelscale=4.0 gives vif_min_dim = 56, so a 16x16 frame with vif_prescale=4.0 (scaled 64x64, entirely safe) is rejected. The option ranges permit this: vif_prescale .min=0.1/.max=4.0 and vif_kernelscale .min=0.1/.max=4.0 (float_vif.c:104-118).

Why: Line 323 hands compute_vif s->scaled_w/s->scaled_h, not w/h. The four-scale ladder therefore only ever constrains the SCALED dimensions, and the second guard at line 229 already enforces exactly that. Line 213 applies the same ladder-derived vif_min_dim to the raw w/h, which the ladder never touches. For vif_prescale <= 1 that is harmless (scaled <= raw), but for vif_prescale > 1 it rejects raw dimensions in [scaler_minimum, vif_min_dim) whose scaled counterparts clear the ladder. The only thing that actually consumes the raw dimensions is vif_scale_frame_s, whose own floor is much smaller — the interpolators fold through mirror() at vif_tools.c:581-584, a single-bounce fold that needs only dim >= 5 for the widest kernel (lanczos4, a=4, vif_tools.c:650/672-673). The block comment at lines 207-210 states the second guard exists to cover 'a prescale != 1.0 [that] moves the dimensions actually handed to compute_vif()', but the first guard makes that path unreachable for exactly the prescale > 1 half of the range. Master's raw floor of 9 had the same shape but a band of [1,8]; this PR widens it to [1,15] at the default kernelscale and up to [1,55] at kernelscale 4.0, so the behaviour change of claim (c) is larger than 'rejects below 16' when non-default options are in play.

[LOW] core/test/test_convolution_edge_small.c:273

test_large_plane_bit_identical is placed at a size where the clamp is provably inert and the fold provably cannot iterate, so it does not pin the contract boundary the bit-identity claim rests on

Trigger: Revert convolution_clamp_borders (convolution.c:41-47) to a no-op body, or change the size <= 1 short-circuit in convolution_reflect101 (convolution_internal.h:50) to size <= 0: test_large_plane_bit_identical still passes unchanged. Conversely there is no test anywhere in the file that compares a value produced at dim in [radius+1, filter_width-1] against single_bounce.

Why: enum { WIDTH = 24, HEIGHT = 24, STRIDE = 32 }; with the 5-tap kFilter5 puts the case in the dim >= filter_width regime. There borders_left = 2 and borders_right = 24 - 3 = 21, so neither branch of convolution_clamp_borders fires — the clamp is dead code for this input. And every tap index generated is in [-2, 25], which one bounce maps to [0, 23], so the while at convolution_internal.h:52 always exits on its first iteration. The test therefore compares the new code against single_bounce on inputs where the new code is single_bounce, executed through unchanged control flow. The sizes where the change actually alters control flow versus master — dim in [radius+1, filter_width-1], i.e. 3 and 4 for the 5-tap and 9..16 for the 17-tap kFilter17, where borders_hi < borders_lo and the clamp collapses the duplicated border bands — are exercised only by test_scalar_*_small_planes, whose assertions are plane_all_finite + poison_buf_guards_intact (lines 154-156): they detect escaped reads and writes but say nothing about the produced values. The contract boundary dim == radius + 1, where the single bounce lands exactly on index 0 and index dim-1 and any off-by-one in the fold would show up, has no value-level assertion at all. Adding WIDTH/HEIGHT = 3 and 4 (5-tap) and 9 (17-tap, with reference_convolve generalised past its hardcoded radius = 2 / k < 5 at lines 220-241) would make the test actually pin the claim.


🤖 Generated with Claude Code

@lusoris

lusoris commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Three HIGH findings fixed

The three HIGH items from the review above are fixed and pushed. Each was reproduced before being changed, and the new test fails on the pre-fix tree.

finding status
AVX2/AVX-512 border split unclamped (6 sites) — heap write fixed, clamp shared via convolution_internal.h
motion_filter_size=1 bypassed the min-dim guard, making the above reachable fixed, guard now mirrors motion_blur_plane
odd-height 4:2:0 chroma under-allocated by one row fixed, now uses picture.c's ceiling geometry
stray } // namespace breaking every Windows build removed

Evidence. Under -Db_sanitize=address, the pre-fix tree reports heap-buffer-overflow ... WRITE of size 4 in convolution_f32_avx_s, reached from vmaf_read_pictures, "0 bytes after 32-byte region" — the single-row buffer. Post-fix: no sanitizer reports, the guard returns -EINVAL, and the odd-height case scores cleanly.

New test core/test/test_motion_convolution_oob.c goes through the public API deliberately: test_motion_min_dim only calls init() and test_convolution_edge_small calls the scalar kernels directly, so neither could reach the dispatched SIMD path where these lived.

meson test --suite=fast: 111 Ok, 0 Fail. Netflix golden gate: 271 passed, 12 skipped, 0 failed.

Still open: the twelve MEDIUM and LOW findings from the review — the MSVC ARM64 clz guard, the VmafFeatureDictionary ownership inconsistency, the Metal 20x20 halo guard, the static-link smoke compiler mismatch, and the libc++ detection. This PR should not merge until those are triaged.

🤖 Generated with Claude Code

@lusoris
lusoris force-pushed the fix/upstream-harvest-2026-09-03 branch 2 times, most recently from 3fbc129 to 80910f1 Compare September 3, 2026 15:40
@lusoris
lusoris marked this pull request as ready for review September 3, 2026 15:40
@lusoris
lusoris force-pushed the fix/upstream-harvest-2026-09-03 branch from 80910f1 to 1c136d7 Compare September 3, 2026 15:41
/* maps that to -ENOMEM. Handing it a heap-allocated EMPTY dictionary */
/* reaches that branch deterministically, with no malloc-fail */
/* injection: pre-fix it `return`ed straight out and leaked the */
/* caller's dictionary (the defect Netflix/vmaf#1242 reports); */
@lusoris
lusoris force-pushed the fix/upstream-harvest-2026-09-03 branch from 455d767 to dc5fef5 Compare September 3, 2026 17:03
@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 3, 2026
@lusoris
lusoris force-pushed the fix/upstream-harvest-2026-09-03 branch 3 times, most recently from 37228e7 to fa9ef2c Compare September 3, 2026 19:48
@lusoris

lusoris commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

The two red legs here (Build — Linux (Intel LLVM, all backends), Build — macOS (Clang, CPU + Metal)) are not caused by this PR. Both fail test_output::test_vmaf_version, and this branch does not touch test_output.c, does not touch any version surface, and its only meson change is a CUDA include path. The test is byte-identical on master.

Root cause, reproduced exactly: core/include/meson.build passes --always to git describe, so a checkout that cannot reach a v*.*.* tag still exits 0 and prints a bare abbreviated object name, which becomes VMAF_VERSION verbatim. build.yml checks out at the actions/checkout default fetch-depth: 1 — no tags. The test only notices when that abbreviation contains no ASCII digit. This PR's merge commit is abafdfcc3c8ef40c369b4bb776c14188729ceadaabafdfc. No digit. Roughly one commit in a thousand.

It looked like this PR's fault because of a second gap: the CI impact planner selected no Intel LLVM job in any of the last eight master runs, so master was green on legs nobody ran, and the first PR broad enough to select them absorbed the blame.

Fix is in #1266 (drops --always, pins the explicit fallback, moves build.yml to fetch-depth: 0, adds a gate, and closes the Windows coverage gap that hid it). Once that lands, rebase this branch and the two legs should go green without any change here.

lusoris pushed a commit that referenced this pull request Sep 3, 2026
test_output::test_vmaf_version failed the Intel LLVM and macOS Clang legs
of build.yml on PR #1223, a PR that does not touch test_output.c, does not
touch any version surface, and whose only meson change is a CUDA include
path. The test is byte-identical on master. The failure is not that PR's.

Root cause. core/include/meson.build derives VMAF_VERSION from
`git describe --tags --long --match 'v*.*.*' --always`. With --always, git
exits 0 even when no matching tag is reachable and prints a bare
abbreviated object name, which meson substitutes into vcs_version.h
verbatim. build.yml checks out at the actions/checkout default fetch-depth
of 1, which fetches no tags, so every build on that workflow has been
stamping a commit abbreviation where a version belongs — `vmaf --version`,
the JSON and XML `version` field, and vmaf_version() all reported it.

That is silent until the abbreviation happens to contain no ASCII digit,
which is what test_vmaf_version asserts against. #1223's merge commit was
abafdfc, abbreviating to "abafdfc" — no
digit. Roughly one commit in a thousand, (6/16)^7, so the defect sat in
tree and then failed an unrelated PR. Reproduced exactly: a depth-1
tagless fetch of that commit run through mesonbuild's vcstagger yields
VMAF_VERSION="abafdfc" (test fails) before the change and "3.2.1" after.

Dropping --always makes git exit non-zero in precisely those cases, and
vcstagger.py substitutes the vcs_tag fallback on any exception from the
subprocess. The fallback is now spelled out rather than left implicit,
since it is the entire tagless path. When a tag is reachable the --long
form still embeds the commit, so no provenance is lost:
v3.1.0-2417-g1ee6ebde42. build.yml also moves to fetch-depth 0, matching
libvmaf-build-matrix.yml, so CI exercises the tagged path rather than the
fallback.

Two coverage gaps let this hide. The CI impact planner selected no Intel
LLVM job in any of the last eight master runs, so master's own build.yml
legs were not exercising the test; and the Windows job — the third leg —
runs an explicit whitelist that omitted test_output. Windows now runs it
(the test already carries a GetTempPathA/GetTempFileNameA path for that
platform). Its `for` loop also gained `|| exit /b 1`: GitHub runs
`shell: cmd` as `%ComSpec% /D /E:ON /V:OFF /S /C "CALL ..."`, so the step
result was the errorlevel of the last executable alone and a failure in
any earlier test was discarded. /V:OFF rules out !ERRORLEVEL!, so `||` is
the portable form.

scripts/ci/check-vcs-version-not-bare-sha.sh keeps --always out: it
brackets the vcs_tag call, strips comments so prose may discuss the flag,
and fails if --always returns, if the explicit fallback goes missing, or
if --match is dropped. Negative-tested on all three; positive-tested on
the restored file and on a comment mentioning the flag outside the call.
Upstream Netflix carries the --always form, so this file will conflict on
a sync — the gate turns a careless resolution into a build failure rather
than a silently wrong version. Wired into `make lint-sh`.

Verified: CPU build clean, fast suite 106 Ok / 0 Fail, test_output 13/13,
`vmaf --version` prints v3.1.0-2417-g1ee6ebde42, pre-commit clean on all
touched files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 3, 2026
test_output::test_vmaf_version failed the Intel LLVM and macOS Clang legs
of build.yml on PR #1223, a PR that does not touch test_output.c, does not
touch any version surface, and whose only meson change is a CUDA include
path. The test is byte-identical on master. The failure is not that PR's.

Root cause. core/include/meson.build derives VMAF_VERSION from
`git describe --tags --long --match 'v*.*.*' --always`. With --always, git
exits 0 even when no matching tag is reachable and prints a bare
abbreviated object name, which meson substitutes into vcs_version.h
verbatim. build.yml checks out at the actions/checkout default fetch-depth
of 1, which fetches no tags, so every build on that workflow has been
stamping a commit abbreviation where a version belongs — `vmaf --version`,
the JSON and XML `version` attribute, and vmaf_version() all reported it.

That is silent until the abbreviation happens to contain no ASCII digit,
which is what test_vmaf_version asserts against. #1223's merge commit was
abafdfc, abbreviating to "abafdfc" — no
digit. Roughly one commit in a thousand, (6/16)^7. Reproduced exactly: a
depth-1 tagless fetch of that commit run through mesonbuild's vcstagger
yields VMAF_VERSION="abafdfc" before this change and "3.2.1" after.

Master is not exempt, it has only been lucky. Its build.yml legs do run
and are green, but all 20 most recent master commits abbreviate with a
digit; the defect fires on whichever commit first abbreviates to all
letters. PR merge commits simply roll the dice more often.

Dropping --always makes git exit non-zero in precisely those cases, and
vcstagger.py substitutes the vcs_tag fallback on any exception from the
subprocess. The fallback is now spelled out rather than left implicit,
since it is the entire tagless path. When a tag is reachable the --long
form still embeds the commit, so no provenance is lost:
v3.1.0-2417-g1ee6ebde42. build.yml also moves to fetch-depth 0, matching
libvmaf-build-matrix.yml, so CI exercises the tagged path rather than the
fallback.

A real coverage gap kept the third leg quiet. The Windows job runs an
explicit whitelist that omitted test_output; it now runs it, and the test
already carries a GetTempPathA/GetTempFileNameA path for that platform.
That loop also gained `|| exit /b 1`: GitHub runs `shell: cmd` as
`%ComSpec% /D /E:ON /V:OFF /S /C "CALL ..."`, so the step result was the
errorlevel of the last executable alone and a failure in any earlier test
was discarded outright. /V:OFF rules out !ERRORLEVEL!, so `||` is the
portable check.

scripts/ci/check-vcs-version-not-bare-sha.sh keeps --always out: it
brackets the vcs_tag call, strips comments so prose may discuss the flag,
and fails if --always returns, if the explicit fallback goes missing, or
if --match is dropped. Negative-tested on all three; positive-tested on
the restored file and on a comment mentioning the flag outside the call.
Upstream Netflix carries the --always form (verified against
upstream/master:libvmaf/include/meson.build), so this file will conflict
on a sync — the gate turns a careless resolution into a build failure
rather than a silently wrong version. Wired into `make lint-sh`.

Verified: CPU build clean, fast suite 106 Ok / 0 Fail, test_output 13/13,
`vmaf --version` prints v3.1.0-2417-g1ee6ebde42, pre-commit clean on all
touched files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris

lusoris commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Correction to my comment above: the claim that the CI impact planner had skipped master's Intel LLVM legs was wrong — I conflated build.yml with libvmaf-build-matrix.yml. Master's build.yml does run the Intel LLVM and macOS legs, and they are green.

The root-cause finding is unchanged and, if anything, stronger: master passes only because all 20 most recent master commits abbreviate with a digit. The defect is latent on master, not absent from it — it fires on whichever commit first abbreviates to seven letters. A PR merge commit is a fresh object every time the PR or its base moves, so PRs simply roll the dice more often, which is why one surfaced it first.

This PR is still not the cause, and #1266 is still the fix.

lusoris added a commit that referenced this pull request Sep 3, 2026
…#1266)

* fix(build): stop VMAF_VERSION degrading to a bare commit abbreviation

test_output::test_vmaf_version failed the Intel LLVM and macOS Clang legs
of build.yml on PR #1223, a PR that does not touch test_output.c, does not
touch any version surface, and whose only meson change is a CUDA include
path. The test is byte-identical on master. The failure is not that PR's.

Root cause. core/include/meson.build derives VMAF_VERSION from
`git describe --tags --long --match 'v*.*.*' --always`. With --always, git
exits 0 even when no matching tag is reachable and prints a bare
abbreviated object name, which meson substitutes into vcs_version.h
verbatim. build.yml checks out at the actions/checkout default fetch-depth
of 1, which fetches no tags, so every build on that workflow has been
stamping a commit abbreviation where a version belongs — `vmaf --version`,
the JSON and XML `version` attribute, and vmaf_version() all reported it.

That is silent until the abbreviation happens to contain no ASCII digit,
which is what test_vmaf_version asserts against. #1223's merge commit was
abafdfc, abbreviating to "abafdfc" — no
digit. Roughly one commit in a thousand, (6/16)^7. Reproduced exactly: a
depth-1 tagless fetch of that commit run through mesonbuild's vcstagger
yields VMAF_VERSION="abafdfc" before this change and "3.2.1" after.

Master is not exempt, it has only been lucky. Its build.yml legs do run
and are green, but all 20 most recent master commits abbreviate with a
digit; the defect fires on whichever commit first abbreviates to all
letters. PR merge commits simply roll the dice more often.

Dropping --always makes git exit non-zero in precisely those cases, and
vcstagger.py substitutes the vcs_tag fallback on any exception from the
subprocess. The fallback is now spelled out rather than left implicit,
since it is the entire tagless path. When a tag is reachable the --long
form still embeds the commit, so no provenance is lost:
v3.1.0-2417-g1ee6ebde42. build.yml also moves to fetch-depth 0, matching
libvmaf-build-matrix.yml, so CI exercises the tagged path rather than the
fallback.

A real coverage gap kept the third leg quiet. The Windows job runs an
explicit whitelist that omitted test_output; it now runs it, and the test
already carries a GetTempPathA/GetTempFileNameA path for that platform.
That loop also gained `|| exit /b 1`: GitHub runs `shell: cmd` as
`%ComSpec% /D /E:ON /V:OFF /S /C "CALL ..."`, so the step result was the
errorlevel of the last executable alone and a failure in any earlier test
was discarded outright. /V:OFF rules out !ERRORLEVEL!, so `||` is the
portable check.

scripts/ci/check-vcs-version-not-bare-sha.sh keeps --always out: it
brackets the vcs_tag call, strips comments so prose may discuss the flag,
and fails if --always returns, if the explicit fallback goes missing, or
if --match is dropped. Negative-tested on all three; positive-tested on
the restored file and on a comment mentioning the flag outside the call.
Upstream Netflix carries the --always form (verified against
upstream/master:libvmaf/include/meson.build), so this file will conflict
on a sync — the gate turns a careless resolution into a build failure
rather than a silently wrong version. Wired into `make lint-sh`.

Verified: CPU build clean, fast suite 106 Ok / 0 Fail, test_output 13/13,
`vmaf --version` prints v3.1.0-2417-g1ee6ebde42, pre-commit clean on all
touched files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(state): record T-VCS-VERSION-BARE-SHA and the AGENTS.md invariant

Adds the docs/state.md row + Updated line for the version-string defect
(ADR-0165 / CLAUDE.md §12 r13) and the scripts/ci/AGENTS.md section
covering the three properties check-vcs-version-not-bare-sha.sh enforces
and why build.yml's fetch-depth: 0 is load-bearing.

The row records that master's build.yml legs do run and are green, and
that this is luck rather than coverage: every recent master commit
abbreviated with a digit, so the defect is latent on master rather than
absent from it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(changelog): render the Unreleased block from the new fragments

scripts/release/concat-changelog-fragments.sh --check gates on drift
between CHANGELOG.md and changelog.d/; the two fragments this PR adds
have to be rendered in the same commit range.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Lusoris <lusoris@pm.me>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Lusoris and others added 8 commits September 3, 2026 23:59
…ADR-1166)

Verify a batch of long-open Netflix/vmaf issues against this tree — the fork
diverged far enough (ADR-0700's `libvmaf/` -> `core/` rename, several C-to-C++
conversions, four fork-added GPU backends) that an upstream report is neither
automatically applicable nor automatically stale — and fix the subset that
still bites. The full triage table, including the ALREADY-FIXED and
NOT-APPLICABLE verdicts, is in
docs/research/1166-upstream-issue-harvest-2026-09-03.md.

Memory safety, all reachable from the public C API today:

* reported upstream as Netflix/vmaf#1582 (mirror half also Netflix/vmaf#1581):
  the reflect-101 fold in convolution_edge_s / _sq_s / _xy_s bounced an
  out-of-range tap exactly once, which only lands in range for
  size >= radius + 1; and convolution_x_c_s / convolution_y_c_s derived the
  trailing border bound as dim - (filter_width - radius), which goes negative
  for a plane narrower than the filter and starts the trailing loop at a
  negative index (heap underflow write). Two live paths reached those sizes:
  `--feature float_vif` on 9..15px frames (the guard admitted >= 9 but the
  four-scale ladder needs >= 16 — the binding constraint is scale 3), and
  `--feature float_motion` with motion_add_uv on a 4x4 YUV420P frame (the
  guard validated luma only while the blur runs at the 2x2 chroma dimensions).
  The fold is now iterative and bit-identical to the single bounce for every
  in-contract size; the borders are clamped; float_vif derives its floor from
  vif_get_min_dim(kernelscale); float_motion validates every plane it convolves.

* reported upstream as Netflix/vmaf#1580: the three fork-added Metal motion
  extractors were written after the Research-0094 sweep and never got the
  min-dim guard, so a 1- or 2-pixel-tall frame read out of bounds on device.

Correctness and contracts:

* reported upstream as Netflix/vmaf#1242: vmaf_model_feature_overload() leaked
  the caller's dictionary on the -ENOMEM path,
  vmaf_model_collection_feature_overload() swallowed the copy error and
  dereferenced *model_collection unchecked, and feature.h / model.h documented
  opposite ownership rules — one of the two readings a latent double free. All
  three public headers now state the implemented contract identically.
  Supersedes ADR-0806.

* reported upstream as Netflix/vmaf#1551, which retracts Netflix/vmaf#1422: the
  MSVC __builtin_clz shim used __lzcnt, which emits LZCNT with no runtime gate
  and silently retires as BSR on any x86-64 without ABM — wrong VIF and ADM
  log2 shifts, no fault, and invisible to CI because every hosted Windows
  runner has LZCNT. Now _BitScanReverse, with an architecture guard so an MSVC
  ARM64 leg compiles.

User-visible surfaces:

* reported upstream as Netflix/vmaf#743: the CLI wrote UTF-8 braille and a CSI
  erase to a Windows console it never configured, so the progress line was
  mojibake under every default code page. The console is switched to UTF-8 + VT
  for the run and restored on exit, with an ASCII fallback.

* reported upstream as Netflix/vmaf#1178: libvmaf.pc omitted the C++ runtime,
  so `pkg-config --static --libs libvmaf` produced a link line that fails with
  hundreds of undefined references — the reason ADR-0198's static FFmpeg
  reproducer had to add -lstdc++ by hand.

* reported upstream as Netflix/vmaf#1573: the nvcc fatbin include list used
  relative paths that stopped resolving at ADR-0700, and three shell-driven
  tool tests declared no `depends`, so a subset run built nothing and exited
  127.

Behaviour changes: float_vif now rejects frames below 16px in either dimension,
and float_motion with motion_add_uv rejects sub-minimum chroma planes. Both
convert previously undefined behaviour into a documented -EINVAL.

Regression tests: core/test/test_convolution_edge_small.c (NaN-poisoned guard
buffers; fails pre-fix), core/test/test_compat_clz.c,
core/test/test_model_feature_overload_ownership.c, core/test/test_spinner.cpp,
scripts/ci/check-msvc-clz-shim.sh (fails pre-fix), plus extended cases in
test_motion_min_dim.c and test_float_vif_min_dim.c, and a real static link in
the libvmaf-build-matrix pkg-config step.

Netflix golden scores unchanged: 76.66744 / 35.070245 / 7.985956
(271 passed, 12 skipped).

Confirmed but not batched, one docs/state.md row each: Netflix/vmaf#1564, #930,
off-by-one found while triaging Netflix/vmaf#1580.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…holes

Three HIGH findings from an independent adversarial review of this
harvest. Each was reproduced before being fixed, and the regression test
fails on the pre-fix tree.

1. The Netflix/vmaf#1582 border clamp landed only on the scalar path.
   convolution_f32_c_s dispatches straight into convolution_f32_avx_s
   whenever AVX2 is present — every CI runner and the dev workstation —
   so the clamp this PR added was dead code on x86. The AVX2 and AVX-512
   twins derive the same `height - radius` split at three sites each and
   kept it unclamped. For a plane shorter than the radius that value is
   negative, so the trailing border loop starts at a negative row and the
   leading one runs past the end. Both are heap WRITES, not reads. All
   six sites now share the scalar clamp, which moved into
   convolution_internal.h as a static inline.

2. motion_filter_size=1 bypassed the minimum-dimension guard entirely.
   motion_check_min_dim gated its check on `effective_filter_size > 1`,
   but motion_blur_plane keeps filter_size = 5 for that value and only
   swaps in the FILTER_5_NO_OP_s coefficients, so the radius is still 2.
   A 1-row plane therefore reached the convolution in (1) through a
   documented public option with range 0..9. The guard now mirrors
   motion_blur_plane exactly: 3 taps only for motion_filter_size == 3,
   otherwise 5.

3. Odd-height 4:2:0 chroma planes were under-allocated by one row.
   motion_chroma_heights used `h / 2` while picture.c and the guard both
   use the ceiling `(h + 1) >> 1`, so motion_copy_and_blur overran ref,
   tmp and every MOTION_BLUR_RING blur buffer for both U and V. Even
   heights were unaffected, which is why neither golden fixture caught
   it.

Also removes a stray `} // namespace` inside the _WIN32 block of
core/tools/vmaf.cpp that closed a namespace never opened. It broke every
Windows build and was invisible on Linux, where the preprocessor drops
the block. This PR is a draft and drafts run no CI here, so nothing had
compiled it. The file now has exactly one namespace opener and one
closer, neither inside any conditional.

New test core/test/test_motion_convolution_oob.c drives float_motion
through the public vmaf_read_pictures entry point, because no existing
test could reach the dispatched SIMD path: test_motion_min_dim only
calls init(), and test_convolution_edge_small calls the scalar kernels
directly.

Verified both ways. The new test fails on the pre-fix tree and passes
after. Under -Db_sanitize=address the pre-fix tree reports
"heap-buffer-overflow ... WRITE of size 4" in convolution_f32_avx_s
reached from vmaf_read_pictures, "0 bytes after 32-byte region" — the
single-row buffer. Post-fix: zero sanitizer reports, the guard returns
-EINVAL, and the odd-height case scores cleanly.

meson test --suite=fast: 111 Ok, 0 Fail. Netflix golden gate: 271
passed, 12 skipped, 0 failed.

The twelve MEDIUM and LOW findings from the same review are not
addressed here and remain open on the PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… two leaks

Fourth MEDIUM finding from the adversarial review of this harvest. The
Netflix/vmaf#1242 contract was still stated three different ways, and one
of them was a double free.

<libvmaf/feature.h> and docs/api/index.md both claimed that an unknown
feature_name never consumes the dictionary. That is true of
vmaf_use_feature, which resolves the name against the global extractor
registry and returns -EINVAL before touching it. It is NOT true of
vmaf_model_feature_overload, which matches feature_name against the
features of one particular model: a name that matches nothing there is
not an error, it is a successful no-op returning 0, and the tail
vmaf_dictionary_free consumes the dictionary anyway. A caller following
the old wording would free it a second time.

<libvmaf/libvmaf.h> already described vmaf_use_feature correctly.
<libvmaf/model.h> described the overloads correctly but then claimed its
rule "matches vmaf_use_feature", which is exactly the case where they
differ. All four surfaces now state the asymmetry explicitly and say why
it exists rather than papering over it.

vmaf_use_feature also leaked the caller's dictionary on two failure
paths: a failed vmaf_dictionary_copy returned without releasing the
source, and a failed vmaf_feature_extractor_context_create returned
without releasing the copy — that function frees only what it allocated
itself. Both leaked precisely when the documented contract told the
caller not to free, so nothing else could have released them.

Two cases added to test_model_feature_overload_ownership.c pin the
asymmetry from both sides: the model overload returning 0 and consuming
on an unknown name, and vmaf_use_feature returning -EINVAL and handing
the dictionary back. The suite is 8 tests and passes clean under
-Db_sanitize=address, which is where a regression would surface as a
double free rather than a silent contract violation.

meson test --suite=fast: 111 Ok, 0 Fail. Netflix golden gate: 271
passed, 12 skipped, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The MSVC shim's architecture test was `_M_X64 || _M_IX86`, justified in
both the header comment and scripts/ci/check-msvc-clz-shim.sh by the
claim that `_BitScanReverse` is x86-only. Per the MSVC intrinsics
reference that is wrong: `_BitScanReverse` is available on x86, ARM, x64
and ARM64, and only `_BitScanReverse64` is restricted (to x64 and ARM64).
`__lzcnt` is the x86-only one, and it is not used here.

The header is the sole definition of `__builtin_clz` for integer_adm.c
and integer_vif.h, which sit on the generic scalar path and are compiled
for every target, so MSVC ARM64 matched no branch and failed to compile
outright rather than falling back to anything. The fork runs no MSVC
ARM64 CI leg, so the break was latent.

The allowlist now enumerates every architecture MSVC targets, selects
`_BitScanReverse64` on x64/ARM64 and keeps the two-step 32-bit
reconstruction elsewhere. The gate now joins the guard's continuation
lines before matching (the guard legitimately spans several lines) and
asserts the ARM64 arm specifically, so the allowlist cannot be narrowed
again; both that narrowing and an `__lzcnt` reintroduction were
negative-tested against it. Header and gate comments corrected to the
documented architecture matrix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round-4 review findings; each checked against the code before acting,
and one did not hold up and is recorded as such.

Metal motion kernels — the round-3 guard was insufficient and the real
defect was in the kernel. integer_motion.metal, float_motion.metal and
integer_motion_v2.metal load a TILE_W x TILE_H = 20x20 threadgroup tile
at origin `bid * 16 - 2`, so the mirror helper receives indices up to
`16*bid + 17`, far outside the 5-tap neighbourhood it appears to serve,
and a single bounce only lands in range when `idx <= 2 * (sup - 1)`.
Enumerated over the real tile span, the single-bounce form read out of
bounds for every dimension in 1..9 AND for exactly 17 — at 17 the last
workgroup reaches idx 33 while 2*(17-1) = 32, folding to -1. A 3x3
floor closes neither 4..9 nor 17, so all three kernels now fold
iteratively, as the CPU scalar path already does in
convolution_internal.h. Verified over dims 1..299 across the full tile
span: always in range, always terminating, and bit-identical to the
single bounce wherever one bounce sufficed, so no in-contract score
moves. The host-side guard comments claimed the 3x3 floor was what kept
the kernel in bounds, which was wrong; corrected.

integer_motion_v2.metal was also the last backend still using the wrong
reflection convention: `2 * sup - idx - 1` repeats the boundary row
where reflect-101 skips it. CPU, CUDA (PR #120 / T7-15), SYCL and HIP
all carry `- 2`, and the SYCL fix records the old form as a systematic
~2.6e-3 motion drift vs CPU on every frame after the first. Metal now
matches (ADR-0214 places=4). The ADM kernels' `- 1` was checked and
deliberately left alone — ADM legitimately uses whole-sample
reflection, matching adm_tools.c::dwt2_src_indices_filt_s, CUDA's
calculate_indices() and the SYCL twin.

float_vif — all four GPU backends sat below the CPU floor. The CPU
requires vif_get_min_dim() = 16 at the default kernelscale (the binding
constraint is scale 3: max(9,10,12,16)). Metal checked only
`scale_w[FVIF_SCALES-1] == 0`, i.e. `w >> 3 == 0`, an effective floor
of 8; CUDA, HIP and SYCL had no dimension floor at all, halving to
scale 3 unchecked. All four now derive the floor from
vif_get_min_dim(), the CPU's own source of truth, so the 8..15px range
that walks the reflect-101 mirror out of the plane at scale 3 is
rejected uniformly. vif_tools.h gained an `extern "C"` guard — without
it the C++ (SYCL) and Objective-C++ (Metal) callers demand mangled
symbols against the C vif_tools.c. It was previously included only by
C translation units.

vmaf.cpp — `--help` and `--version` left the Windows console in UTF-8 +
VT mode. WindowsConsoleGuard was an automatic local whose comment
claimed it restored on every exit path. It did not: cli_parse
terminates via usage_exit(), which is [[noreturn]] and calls exit(),
and exit() does not destroy objects with automatic storage duration.
Objects with static storage duration ARE destroyed by exit()
([basic.start.term]), so the guard is now static and the restore runs
on the exit() paths, the `goto cleanup` spine and a normal return
alike. POSIX is unaffected (the block is #ifdef _WIN32).

check-msvc-clz-shim.sh was evadable by macro indirection: rules (1) and
(4) keyed on the call syntax `__lzcnt(`, so `#define LZ __lzcnt`
followed by `LZ(x)` reintroduced the instruction while still passing
the gate that exists to prevent exactly that. Both rules now match the
bare identifier, and rule (4) is scoped to source extensions because
core/src/feature/AGENTS.md legitimately discusses __lzcnt in prose.
Negative-tested: macro indirection, a narrowed architecture allowlist,
and a direct __lzcnt reintroduction all fail the gate.

libvmaf-build-matrix.yml — the static-link smoke test linked with bare
`cc` while the matrix builds with `ccache gcc-14` / `ccache clang-22`,
so it exercised a toolchain the archive was not produced with; now
${CC:-cc}. The accompanying LTO concern does not apply: b_lto is
meson-default false here and explicitly false on the SYCL/CUDA legs, so
the archive holds plain objects rather than LTO IR.

NOT a defect — the Libs.private libc++ detection. The review held that
keying on _LIBCPP_VERSION ignores an explicit -stdlib=libc++. Tested
against the installed meson: a probe project reading
cxx.get_define('FOO') under -Dcpp_args=-DFOO=42 reports 42, so compiler
checks do observe the project's cpp_args and the _LIBCPP_VERSION probe
therefore sees -stdlib=libc++ exactly as its comment claims. No change.

Verified: CPU build + fast suite 111 Ok / 0 Fail; CUDA lane rc=0 with
float_vif_cuda.c.o built; SYCL lane rc=0 under icpx with
float_vif_sycl.o built and no undefined vif_get_min_dim, confirming the
extern "C" linkage resolves. clang-tidy exit=0 on both files CI's
changed-files job globs (core/tools/vmaf.cpp,
core/src/feature/vif_tools.h); .mm and .metal are not in that glob and
cuda/ hip/ sycl/ are excluded by path. The Metal kernels are not
buildable on Linux — CI's macOS legs compile them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The whole-tree ratchet exited 2 on this branch — four files were ABOVE
their baseline, which ADR-1142 treats as the PR's own regression to fix
in code, never to baseline away:

  core/src/feature/common/convolution_avx.c            0 -> 1
  core/src/feature/common/convolution_avx512.c         0 -> 1
  core/test/test_model_feature_overload_ownership.c    0 -> 1
  core/test/test_motion_convolution_oob.c              0 -> 11

convolution_avx.c / convolution_avx512.c —
readability-function-size on convolution_f32_avx{,512}_xy_s: "62 lines
including whitespace and comments (threshold 60)". The threshold counts
comments, and the clamp this PR added carried a five-line rationale
block duplicated at all six call sites while the same explanation
already lives on convolution_clamp_borders() in
convolution_internal.h. Replaced with a three-line pointer to that
definition at every site: the explanation is not lost, it is no longer
copied six times, and both functions drop back under the threshold. No
code changed.

test_model_feature_overload_ownership.c — readability-function-size on
run_tests. Each mu_run_test expands to several statements, and the two
cases added for the ownership asymmetry took it to eight, crossing
StatementThreshold 120. Split into run_guard_tests() and
run_consumption_tests(), grouped the way core/test/test_iqa_helpers.c
and test_cli_parse.c already group theirs.

test_motion_convolution_oob.c — eleven modernize-use-nullptr. This is a
C translation unit, and ADR-1138 keeps NULL in C TUs because MSVC's
documented /std:clatest C23 feature set has no `nullptr` while the
required Windows build compiles it with cl.exe. Wrapped in
NOLINTBEGIN/NOLINTEND(modernize-use-nullptr) with the ADR-1138
citation inline, matching the pattern test_model.c and test_output.c
already use.

Verified: clang-tidy reports 0 warnings on all four files, the CPU fast
suite is 112 Ok / 0 Fail, and both new tests pass individually. The
three stale-high entries the same run reported (convolution.c 2 -> 0,
test_float_vif_min_dim.c 8 -> 0, test_motion_min_dim.c 15 -> 0) are
left for CI's next measurement to be committed as the tightened
baseline, since the previous measurement was taken with these
regressions still present and so is not a usable baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…gate on Windows

Two platform failures on this branch, neither reproducible on Linux x86-64.

macOS arm64 — test_convolution_edge_small::test_large_plane_bit_identical
failed with "iterative fold moved an in-contract result". The fold was
NOT the cause. It is integer-only, so it cannot vary by platform; the
failure was floating-point contraction. The test's reference_convolve()
reads the file-scope `kFilter5`, which the compiler can constant-fold
and vectorize, while the library kernel receives an opaque
`const float *filter`. On any target with FMA in its baseline — every
arm64 — clang contracts `accum += filter[k] * src[...]` to an fma in one
and not necessarily the other, so the last bit legitimately differs.
x86-64 agreed only because FMA is not in its baseline. Two
separately-compiled float accumulations are not a portable bit-identity
claim.

Rather than loosen the invariant, this states it where it actually
lives. test_fold_matches_single_bounce_exactly asserts the real claim
directly and exhaustively: for every size 2..64 and every in-contract
index, convolution_reflect101() must return exactly what a single bounce
returns, and out of contract it must still land inside the plane. That
is integer-only and platform-independent, and it is a stronger statement
than the float comparison ever made. The end-to-end 24x24 cross-check is
kept but compared within 8 ULP, with the contraction reasoning recorded
on it; 8 ULP is far below anything score-visible while a genuine fold
divergence changes which sample is read and moves results by O(1e-2).
The now-unused bit-identity helpers are removed.

Windows MinGW64 — check_msvc_clz_shim failed, and my first reading of it
was wrong: it is unrelated to the rule changes in this PR. meson invokes
the script through its shebang interpreter, and on the MinGW64 runner
`bash` resolves to Windows' own WSL bash.exe, which has no installed
distribution. The leg printed "Windows Subsystem for Linux has no
installed distributions" and exited 1 before the script ever ran. The
gate is a static source-content check, so it is now registered on
non-Windows hosts only — Linux and macOS both run it in the fast suite
(macOS passes it today) and the lint lane runs it as well, so no
coverage is lost.

Also replaced rule (4)'s `grep -vF "$HDR"` self-exclusion with grep's
own --exclude on the basename. This is a robustness cleanup, not the
Windows fix: comparing grep's walked path against a separately
constructed absolute path is fragile, and the basename form has no path
dependency. Negative-tested that macro indirection, a narrowed
architecture allowlist, and an __lzcnt reintroduction in another file all
still fail the gate, and that a clean tree passes.

Verified: fast suite 112 Ok / 0 Fail, test_convolution_edge_small passes
with the new exhaustive case, clang-tidy 0 warnings on the changed test,
and the gate is still registered and passing on this Linux host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Windows MSVC + oneAPI SYCL leg failed to compile this file:

  test_motion_convolution_oob.c(70,22): error: call to undeclared
  function 'strtok_r'; ISO C99 and later do not support implicit
  function declarations
  test_motion_convolution_oob.c(70,16): error: incompatible integer to
  pointer conversion initializing 'char *' with an expression of type
  'int'

strtok_r is POSIX; the MSVC runtime ships strtok_s instead, so the call
went undeclared and its int return was then assigned to a char *. Plain
strtok is on the fork's banned-function list (docs/principles.md S1.2
rule 30), so neither variant is available here.

The option string this test parses is a fixed "k=v:k=v" form under its
own control, so it now splits with strchr in a small loop: portable
everywhere, no reentrancy question, and no banned call. Behaviour is
identical for every input the test uses.

Verified: test_motion_convolution_oob passes, the fast suite is green,
and clang-tidy reports 0 warnings on the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/upstream-harvest-2026-09-03 branch from fa9ef2c to 1a01ff4 Compare September 3, 2026 21:59
@lusoris
lusoris merged commit 71da046 into master Sep 4, 2026
72 of 73 checks passed
@lusoris
lusoris deleted the fix/upstream-harvest-2026-09-03 branch September 4, 2026 05:01
lusoris pushed a commit that referenced this pull request Sep 5, 2026
…servability

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry
lusoris pushed a commit that referenced this pull request Sep 5, 2026
…servability

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry
lusoris pushed a commit that referenced this pull request Sep 5, 2026
…servability

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…servability

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…servability

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…servability

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry
lusoris added a commit that referenced this pull request Sep 6, 2026
…servability (#1294)

- Correct integer_motion_v2.metal header comment to document reflect-101
  mirror padding matching CPU, CUDA, SYCL, and HIP twins
- Set mu_skipped = 1 on -ENODEV in test_metal_motion_v2_parity.c to exit
  77 on systems without Metal devices, and log stdout on active hardware
- Configure test_metal_motion_v2_parity with protocol 'exitcode',
  should_fail false, and verbose true in core/test/meson.build
- Add ADR-1176 closing out the Metal motion_v2 mirror fix landed in #1223,
  linking from ADR-1166 and docs/adr/README.md
- Move T-METAL-MOTION-V2-MIRROR-OFF-BY-ONE-2026-09-03 to Recently closed
  in docs/state.md with HTML marker at line 337
- Add mv2_mirror reflect-101 invariant to core/src/feature/metal/AGENTS.md
- Add changelog fragment and rebase notes entry

Co-authored-by: Lusoris <lusoris@pm.me>
@lusoris lusoris added the type:bug Something isn't working label Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants