From aa28e32be826cacf8580ecc3c5a62558e1880d2a Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 21:55:17 +0200 Subject: [PATCH 01/15] docs(adr): keep the fork 64-bit only and retire the resurrected i686 lane (ADR-1258) ADR-0728 removed the i686 lane; the libvmaf/ -> core/ rename merge brought it back the same day and it ran compile-only since. ADR-1258 upholds ADR-0728: the lane and preflight's m32 stage go, and the x86-64-only intrinsics get 32-bit-safe forms as hygiene. ADR-0151 is marked superseded by ADR-0728. --- docs/adr/0151-i686-ci-netflix-1481.md | 2 +- .../1258-keep-64-bit-only-retire-i686-lane.md | 97 +++++++++++++++++++ docs/adr/README.md | 3 +- .../0151-i686-ci-netflix-1481.md | 2 +- .../1258-keep-64-bit-only-retire-i686-lane.md | 1 + docs/adr/_index_fragments/_order.txt | 1 + docs/adr/by-tag/build.md | 3 +- docs/adr/by-tag/ci.md | 3 +- docs/adr/by-tag/fork-local.md | 2 +- docs/adr/by-tag/index.md | 8 +- docs/adr/by-tag/netflix-upstream.md | 3 +- docs/adr/by-tag/x86.md | 3 +- mkdocs.yml | 1 + 13 files changed, 117 insertions(+), 12 deletions(-) create mode 100644 docs/adr/1258-keep-64-bit-only-retire-i686-lane.md create mode 100644 docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md diff --git a/docs/adr/0151-i686-ci-netflix-1481.md b/docs/adr/0151-i686-ci-netflix-1481.md index 00dda81c1..b9e6dd1f7 100644 --- a/docs/adr/0151-i686-ci-netflix-1481.md +++ b/docs/adr/0151-i686-ci-netflix-1481.md @@ -1,6 +1,6 @@ # ADR-0151: i686 build-only CI job — reproduce Netflix #1481 -- **Status**: Accepted +- **Status**: Superseded by [ADR-0728](0728-native-build-sunset.md) - **Date**: 2026-04-24 - **Deciders**: Lusoris - **Tags**: ci, build, x86, netflix-upstream diff --git a/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md b/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md new file mode 100644 index 000000000..5060765c2 --- /dev/null +++ b/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md @@ -0,0 +1,97 @@ + +# ADR-1258: Keep the fork 64-bit only; retire the resurrected i686 lane + +- **Status**: Accepted +- **Date**: 2026-09-18 +- **Deciders**: Lusoris +- **Tags**: build, ci, x86, netflix-upstream, fork-local + +## Context + +ADR-0728 (Accepted, 2026-05-28) removed the `Build — Ubuntu i686 gcc (CPU, +no-asm)` lane from `.github/workflows/libvmaf-build-matrix.yml` (commit +`9aa008e70`, #1564), and `docs/development/deprecations.md` records that +"32-bit x86 is unsupported; the fork is 64-bit only". The same day, +`384d97d03`, the squash-merge of the `libvmaf/` → `core/` rename that +"merge-resolves 42 master commits", brought the row back. No ADR asked for +it. It has run on every PR since, compile-only with `-Denable_asm=false` as +ADR-0151 had set it up, and was later renamed `Ubuntu i686 gcc`. ADR-1234 +then built preflight's `m32` stage on the assumption that the lane was +deliberate. + +Checking an i686 build with asm enabled on 2026-09-18 showed what that lane +was and was not covering: + +- Asm builds failed on three x86-64-only intrinsics: `_mm_extract_epi64` in + `adm_avx2.c` and `adm_avx512.c`, and `_mm_cvtsi128_si64` in `psnr_avx2.c`. + That is Netflix#1481's cause, never fixed because the lane pinned the + workaround. +- The lane never ran a test. Run natively, 2 tests fail without asm and 7 + with it. GCC and Clang use the x87 FPU on 32-bit x86, and its 80-bit + intermediates make scalar float code round differently from x86-64 and + from the SIMD kernels. On the Netflix golden pair, 11 of 15 metrics differ + from x86-64 by up to 5.1e-5. With `-msse2 -mfpmath=sse` all tests pass and + every feature metric matches x86-64. + +So the lane looked like 32-bit coverage while testing none of it. The +maintainer was first asked to make 32-bit a tested platform with an SSE2 +floor, and chose that. When ADR-0728's decision and the accidental +resurrection were then put in front of them, they chose to keep the fork +64-bit only (see References). + +## Decision + +We keep ADR-0728's decision that the fork is 64-bit only, and remove what +contradicts it: + +- The `Ubuntu i686 gcc` matrix row, its dependency step and its + `matrix.i686` conditions are removed from `libvmaf-build-matrix.yml`. +- `scripts/dev/preflight.sh` drops its `m32` stage, which only existed to + mirror that lane. This changes the stage list ADR-1234 chose; the rest of + ADR-1234 stands. +- The three x86-64-only intrinsic calls are replaced with 32-bit-safe forms + (`extract_epi64_128()` beside the existing `extract_epi64()` fallback, and + `_mm_storel_epi64`). They cost nothing on x86-64 and keep the x86 sources + portable, but they are hygiene, not a support promise. +- No 32-bit float policy is set. With nothing building 32-bit, an SSE2 floor + would be a rule nobody checks. +- `build-aux/i686-linux-gnu.ini` stays, unreferenced by CI, because historical + documents link to it. + +ADR-0151's status becomes Superseded by ADR-0728, which retired its lane +without recording that. + +## Alternatives considered + +| Option | Pros | Cons | Why not chosen | +| --- | --- | --- | --- | +| Keep 64-bit only; retire the lane and `m32` (chosen) | Matches ADR-0728, the container-first posture (ADR-0686) and what is shipped; no CI or preflight time on an unsupported target | 32-bit portability regressions go unnoticed | — | +| Support 32-bit again: SSE2 floor, asm on, tests in CI | Real 32-bit coverage; scores match x86-64 | Reverses ADR-0728 for a target the project ships nothing for; one more lane to keep green | The maintainer chose 64-bit only once ADR-0728 was on the table | +| Keep the resurrected lane as it was (compile-only, no asm) | No change | Keeps an accidental lane that implies support it does not test, and contradicts ADR-0728 | Coverage that tests nothing is misleading | +| Remove the lane but keep `m32` | Still catches 32-bit width assumptions | Enforces a portability property the project does not promise | Same reason as the lane | + +## Consequences + +- **Positive**: CI, preflight and the docs agree with ADR-0728 again. The + x86 SIMD sources no longer carry Netflix#1481's cause. +- **Negative**: nothing notices if a change breaks 32-bit compilation or + 32-bit scores. Anyone building 32-bit x86 themselves is on their own, + including x87 score drift unless they build with `-msse2 -mfpmath=sse`. +- **Neutral / follow-ups**: the other ADR-0728 removals that the same merge + resurrected are audited separately; their records are corrected where a + later ADR re-adopted them. + +## References + +- [ADR-0728](0728-native-build-sunset.md) (upheld), + [ADR-0151](0151-i686-ci-netflix-1481.md) (superseded by ADR-0728), + [ADR-1234](1234-local-preflight-gate.md) (stage list amended), + [ADR-0686](0686-vmafx-rebrand-aggressive-modernization.md). +- Commits `9aa008e70` (#1564, lane removed) and `384d97d03` (layout rename + merge, lane resurrected). +- Netflix/vmaf issue [#1481](https://github.com/Netflix/vmaf/issues/1481). +- Popup, 2026-09-18, first round, question 1: "Require SSE2 everywhere + (Recommended)"; question 2: "Default options + run tests (Recommended)". + Both were asked without ADR-0728's decision in view. +- Popup, 2026-09-18, second round, question 1 (32-bit x86, with ADR-0728 and + the resurrection shown): "Stay 64-bit only (ADR-0728)". diff --git a/docs/adr/README.md b/docs/adr/README.md index 0923c1b42..9ee45be7f 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -183,7 +183,7 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-0148](0148-iqa-rename-and-cleanup.md) | Rename the IQA-derived reserved-identifier surface (`_iqa_*`, `struct _kernel`, `_ssim_int`, `_map_reduce`, `_map`, `_reduce`, `_context`, `_ms_ssim_map`, `_ssim_map`, `_ms_ssim_reduce`, `_ssim_reduce`, `_alloc_buffers`, `_free_buffers`, header guards `_CONVOLVE_H_` / `_DECIMATE_H_` / `_SSIM_TOOLS_H_` / `__VMAF_MS_SSIM_DECIMATE_H__`) to non-reserved `iqa_*` / `ms_ssim_*` / `_INCLUDED` spellings across `core/src/feature/{iqa,*}` (21 files). Sweeps the ADR-0141 touched-file lint cascade that surfaced (~40 pre-existing warnings: misc-use-internal-linkage → `static` or cross-TU NOLINT; widening multiplications → size_t casts; multi-decl splits; function-size refactors of `calc_ssim` / `compute_ssim` / `compute_ms_ssim` / `run_gauss_tests`; unused-parameter `(void)` casts; scoped NOLINTBEGIN/END for analyzer false positives on kernel-offset bounds and test-helper malloc leaks). Bit-identical VMAF score on Netflix golden pair (scalar vs SIMD, with `--feature float_ssim --feature float_ms_ssim`). Closes backlog item T7-6. | Accepted | lint, cleanup, refactor, iqa, touched-file-rule | | [ADR-0149](0149-port-netflix-1376-fifo-semaphore.md) | Port Netflix upstream PR #1376 ("Fix fifo hangs") into the Python harness under `python/vmaf/core/executor.py` + `python/vmaf/core/raw_extractor.py`. Replaces the 1-second `os.path.exists()` polling loop in `_open_{work,proc}files_in_fifo_mode` with `multiprocessing.Semaphore(0)` signalled by the child processes after `os.mkfifo(...)`; parent acquires with 5-second soft-timeout warn then blocks indefinitely. Applied to both the base `Executor` class hierarchy and the `ExternalVmafExecutor`-style subclass (single-process variant). Fork carve-outs: skip upstream's `__version__ = "3.0.0" → "4.0.0"` bump (fork tracks its own versioning per ADR-0025); drop now-unused `from time import sleep` from both files (ADR-0141 ruff F401). Closes backlog item T4-7. | Accepted | upstream-port, python, concurrency, fifo | | [ADR-0150](0150-port-netflix-1472-cuda-windows.md) | Port Netflix upstream PR #1472 ("cuda: enable CUDA feature extraction on Windows (MSYS2/MinGW)") two-commit series: source-portability guards in CUDA headers + `.cu` files (drop `` from `cuda/common.h`, DEVICE_CODE guards on `` vs `` in `cuda_helper.cuh` + `picture.h`, `#ifndef DEVICE_CODE` around `feature_collector.h` in 5 ADM `.cu` files), and meson build plumbing (`vswhere`-based `cl.exe` discovery without `PATH` pollution, Windows SDK + MSVC include path injection via `-I` flags to nvcc, CUDA version detection via `nvcc --version` instead of `meson.get_compiler('cuda')`). Fork-specific conflict resolutions: keep positional (not `#ifndef __CUDACC__`) initializers in `integer_adm.h`; keep `pthread_dependency` on `cuda_static_lib` (ring_buffer.c still uses pthread directly); merge fork's gencode coverage block (ADR-0122) with upstream's new nvcc-detect block. Drive-by: rename reserved `__VMAF_SRC_*_H__` header guards to `VMAF_SRC_*_INCLUDED`. Linux CPU build 32/32 + Linux CUDA build 35/35 pass. Closes backlog item T4-2. | Accepted | upstream-port, cuda, windows, mingw, build | -| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Accepted | ci, build, x86, netflix-upstream | +| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Superseded by [ADR-0728](0728-native-build-sunset.md) | ci, build, x86, netflix-upstream | | [ADR-0152](0152-vmaf-read-pictures-monotonic-index.md) | `vmaf_read_pictures` now rejects non-monotonic indices with `-EINVAL`. Addresses Netflix upstream issue #910: `integer_motion` / motion2 / motion3 extractors keep sliding-window state keyed by `index % N`, so submitting frames out of order or with duplicate indices silently corrupts their ring-buffers (symptom: missing `integer_motion2_score` on last frame when submission order doesn't match frame order). Enforced via new `last_index` + `have_last_index` fields on `VmafContext`, checked inside the existing `read_pictures_validate_and_prep` helper from ADR-0146. Net visible behaviour change: duplicates and out-of-order indices now return `-EINVAL` instead of producing silent-wrong-answer. 3-subtest reducer in `test_read_pictures_monotonic.c` verified to fail pre-fix and pass post-fix. Zero impact on in-tree callers (vmaf CLI + test suite already iterate strictly increasing). Closes backlog item T1-2. | Accepted | api, correctness, motion, netflix-upstream | | [ADR-0153](0153-float-ms-ssim-min-dim-netflix-1414.md) | `float_ms_ssim` init rejects input below 176×176 with `-EINVAL` + clear error message. Addresses Netflix upstream issue #1414: the 5-level 11-tap MS-SSIM pyramid walks off the kernel footprint at a mid-level scale for inputs below 176×176 (QCIF and smaller), producing a confusing mid-run `error: scale below 1x1!` cascade. Fix checks `w < GAUSSIAN_LEN << (SCALES - 1)` at init time in `float_ms_ssim.c:init`, derived from the existing filter constants so it stays in sync. Extracted `ms_ssim_init_simd_dispatch` helper to keep the function under the ADR-0141 size budget. 3-subtest reducer in `test_float_ms_ssim_min_dim.c` covers 5 boundary rejections (including exact-under cases like 175×176 and 176×175) and 2 accept cases (176×176 exact + 576×324). Verified fail-without-fix + pass-with-fix. Visible behaviour: init now fails immediately instead of mid-stream; zero impact on inputs ≥176×176. Closes backlog item T1-4. | Accepted | correctness, ms-ssim, netflix-upstream | | [ADR-0154](0154-score-pooled-eagain-netflix-755.md) | `vmaf_score_pooled` (via `vmaf_feature_collector_get_score` + the inline `vmaf_feature_vector_get_score` fast-path) now returns `-EAGAIN` when a requested feature index is valid but not yet written (transient — e.g. motion2 waiting for the next frame or flush), distinguishing it from `-EINVAL` which remains for programmer errors (bad pointer, out-of-range, unknown feature). Addresses Netflix upstream issue #755: downstream integrations that want per-frame streaming VMAF output can now distinguish 'retry after next read or flush' from 'abort'. Inline-helper return was previously `-1`; now `-EINVAL` (structural) or `-EAGAIN` (pending). Drive-by: rename reserved `__VMAF_FEATURE_COLLECTOR_H__` header guard to `VMAF_FEATURE_COLLECTOR_INCLUDED` (ADR-0141 touched-file rule). 4-subtest reducer in `test_score_pooled_eagain.c` verified to fail pre-fix, pass post-fix. Closes backlog item T1-1. | Accepted | api, correctness, motion, netflix-upstream | @@ -1072,4 +1072,5 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-1250](1250-eupl-fork-relicense.md) | Licence fork-authored code under EUPL-1.2 and leave every file that carries someone else's code on the terms it already has, with that code's copyright notice restored. 946 files declared `SPDX-License-Identifier: BSD-3-Clause-Plus-Patent`, which is not a real identifier, while fork code otherwise spread across five licences plus 223 prose-only grants and 147 files with no notice at all. Which files move is decided by provenance and applied by `scripts/dev/relicense_fork_files.py`: no counterpart path or name upstream, no copyright naming anyone else, no licence the fork has not used for its own work, no outside contributor in its history, and — recorded in a reviewed `relicense_provenance.toml` — it does not carry ported or copied code. That moved 1,522 files and left 269 on their terms as carriers of Netflix, NVIDIA, libjxl, Xiph or IQA code, plus 526 more the earlier vetoes catch. The 123 dual `... OR MIT` Go files become plain EUPL-1.2, since an MIT option would defeat the reciprocity. Because `libvmaf.so` links fork and inherited code together, the shipped library is effectively copyleft from here on: a breaking change for permissive consumers, who can use Netflix upstream instead. | Proposed | license, compliance, process, docs, breaking-change | | [ADR-1249](1249-praetor-governance-adoption.md) | Adopt the praetor / HISS-16 governance system before `1.0.0-rc.1` as scaffolding and a ratchet only: declarations, the six compiled agent-context projections, generated policy artefacts and one required CI gate land now, while the five pre-migration epic tasks (#1444–#1448) wait until after the release and no product code is refactored for them. `AGENTS.md` becomes a harness of about 250 lines in the engine's internal register, which `compile-context --verify` and `audit` lint from praetor `e4b35cb` on; its hard rules and rebase-sensitive invariants move to `docs/development/` pages that the harness imports, and the rules only the hand-written `CLAUDE.md` carried move with them. The reviewer personas gain canonical sources in `.agents/agents/`. Lefthook owns the `pre-commit` and `pre-push` hooks and delegates both to the pre-commit framework, while the ADR-1241 dispatchers keep `commit-msg` and `pre-rebase`, superseding only ADR-0924's conclusion that a second hook manager is unacceptable. `sync --remote` stays unused: the engine hardcodes `main` while this fork's default branch is `master`. | Proposed | ci, process, agents, tooling, governance, docs, workspace | | [ADR-1256](1256-cambi-spatial-mask-simd-dispatch.md) | CAMBI's spatial-mask row kernels (`compute_dp_row`, `compute_mask_row`, from upstream `86da14d03`) are dispatched per ISA only where they measurably beat scalar: dp row on AVX2 / AVX-512 / NEON, mask row on AVX2 / AVX-512. Upstream's AVX2 dp row is 0.64–0.74x of scalar under Clang and icx, so the fork keeps a single add on the loop-carried chain (1.8–2.3x AVX2, 2.6–3.2x AVX-512 at 1080p on Zen 5); the AVX2 mask row compares with a 2^31 bias so it is exact for every input. `compute_mask_row_neon` stays parity-tested but undispatched because GCC and Clang already compile the scalar loop to the same NEON code. Scores are byte-identical. | Accepted | simd, avx2, avx512, neon, cambi, perf, upstream-port, fork-local | +| [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0728 decided. The `Ubuntu i686 gcc` lane it removed had come back by accident in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151 is marked superseded by ADR-0728. | Accepted | build, ci, x86, netflix-upstream, fork-local | | [ADR-1247](1247-scorecard-exact-head-gates.md) | Bind Scorecard gates to their measured source and scope | Accepted | ci, security, supply-chain | diff --git a/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md b/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md index 8c9adf53f..d2a4e62e8 100644 --- a/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md +++ b/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md @@ -1 +1 @@ -| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Accepted | ci, build, x86, netflix-upstream | +| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Superseded by [ADR-0728](0728-native-build-sunset.md) | ci, build, x86, netflix-upstream | diff --git a/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md b/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md new file mode 100644 index 000000000..3f7bb7fd4 --- /dev/null +++ b/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md @@ -0,0 +1 @@ +| [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0728 decided. The `Ubuntu i686 gcc` lane it removed had come back by accident in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151 is marked superseded by ADR-0728. | Accepted | build, ci, x86, netflix-upstream, fork-local | diff --git a/docs/adr/_index_fragments/_order.txt b/docs/adr/_index_fragments/_order.txt index c8a635432..17984973c 100644 --- a/docs/adr/_index_fragments/_order.txt +++ b/docs/adr/_index_fragments/_order.txt @@ -981,3 +981,4 @@ 1250-eupl-fork-relicense 1249-praetor-governance-adoption 1256-cambi-spatial-mask-simd-dispatch +1258-keep-64-bit-only-retire-i686-lane diff --git a/docs/adr/by-tag/build.md b/docs/adr/by-tag/build.md index b8ed4e6e6..935c55e1d 100644 --- a/docs/adr/by-tag/build.md +++ b/docs/adr/by-tag/build.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -162 ADR(s) carry this tag. +163 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -169,3 +169,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1253](../1253-scalar-fma-not-fused-on-msvcrt.md) | Scalar references compute their fused multiply-add themselves | | [ADR-1254](../1254-win64-cannot-realign-the-stack.md) | Wide vector register pressure is a Win64 correctness constraint, not a performance one | | [ADR-1255](../1255-spdx-residual-identifier-correction.md) | Correct the SPDX identifiers PR #1457 does not reach, to the licence this repository already declares | +| [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | diff --git a/docs/adr/by-tag/ci.md b/docs/adr/by-tag/ci.md index 99008a9c4..4dc0790e8 100644 --- a/docs/adr/by-tag/ci.md +++ b/docs/adr/by-tag/ci.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -192 ADR(s) carry this tag. +193 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -199,3 +199,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1251](../1251-renovate-draft-automerge-deadlock.md) | Renovate opens automerge-eligible and security bumps ready for review | | [ADR-1252](../1252-solo-maintainer-declared-bypass.md) | Declare the single maintainer's bypass actor | | [ADR-1254](../1254-win64-cannot-realign-the-stack.md) | Wide vector register pressure is a Win64 correctness constraint, not a performance one | +| [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | diff --git a/docs/adr/by-tag/fork-local.md b/docs/adr/by-tag/fork-local.md index d8b606876..f92cb700e 100644 --- a/docs/adr/by-tag/fork-local.md +++ b/docs/adr/by-tag/fork-local.md @@ -411,4 +411,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1244](../1244-merge-train-ownership-and-validation.md) | Guard merge-train ownership and exact-head validation | | [ADR-1251](../1251-renovate-draft-automerge-deadlock.md) | Renovate opens automerge-eligible and security bumps ready for review | | [ADR-1252](../1252-solo-maintainer-declared-bypass.md) | Declare the single maintainer's bypass actor | -| [ADR-1256](../1256-cambi-spatial-mask-simd-dispatch.md) | Dispatch CAMBI's spatial-mask row SIMD kernels only where they measurably beat scalar | +| [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | diff --git a/docs/adr/by-tag/index.md b/docs/adr/by-tag/index.md index 91bbe810c..eb39708d2 100644 --- a/docs/adr/by-tag/index.md +++ b/docs/adr/by-tag/index.md @@ -72,7 +72,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [bug](bug.md) | 15 | | [bug-fix](bug-fix.md) | 8 | | [bugfix](bugfix.md) | 24 | -| [build](build.md) | 162 | +| [build](build.md) | 163 | | [bvi-dvc](bvi-dvc.md) | 2 | | [c](c.md) | 9 | | [c++](c++.md) | 12 | @@ -93,7 +93,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [chart](chart.md) | 2 | | [chroma](chroma.md) | 3 | | [chug](chug.md) | 11 | -| [ci](ci.md) | 192 | +| [ci](ci.md) | 193 | | [ci-gate](ci-gate.md) | 1 | | [ciede](ciede.md) | 2 | | [clang-tidy](clang-tidy.md) | 4 | @@ -351,7 +351,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [netflix-compat](netflix-compat.md) | 1 | | [netflix-golden](netflix-golden.md) | 2 | | [netflix-public](netflix-public.md) | 2 | -| [netflix-upstream](netflix-upstream.md) | 8 | +| [netflix-upstream](netflix-upstream.md) | 9 | | [networkpolicy](networkpolicy.md) | 2 | | [nightly](nightly.md) | 1 | | [nn-vc](nn-vc.md) | 1 | @@ -597,6 +597,6 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [windows](windows.md) | 14 | | [workspace](workspace.md) | 27 | | [x264](x264.md) | 1 | -| [x86](x86.md) | 4 | +| [x86](x86.md) | 5 | | [yuv-rgb](yuv-rgb.md) | 1 | | [zero-copy](zero-copy.md) | 4 | diff --git a/docs/adr/by-tag/netflix-upstream.md b/docs/adr/by-tag/netflix-upstream.md index 97882d82f..b284bfea7 100644 --- a/docs/adr/by-tag/netflix-upstream.md +++ b/docs/adr/by-tag/netflix-upstream.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -8 ADR(s) carry this tag. +9 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -15,3 +15,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-0156](../0156-cuda-graceful-error-propagation-netflix-1420.md) | CUDA backend: graceful error propagation (Netflix#1420) | | [ADR-0157](../0157-cuda-preallocation-leak-netflix-1300.md) | CUDA preallocation memory leak fix + `vmaf_cuda_state_free` public API (Netflix#1300) | | [ADR-0158](../0158-netflix-1486-motion-updates-verified-present.md) | Netflix#1486 "Port motion updates" — verified present in fork | +| [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | diff --git a/docs/adr/by-tag/x86.md b/docs/adr/by-tag/x86.md index aa1252f0d..88e864dd5 100644 --- a/docs/adr/by-tag/x86.md +++ b/docs/adr/by-tag/x86.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -4 ADR(s) carry this tag. +5 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -11,3 +11,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-0179](../0179-float-moment-simd.md) | float\_moment SIMD parity (AVX2 + NEON) | | [ADR-0784](../0784-integer-ssim-avx2.md) | AVX2 SIMD path for integer SSIM horizontal moment accumulation | | [ADR-1237](../1237-perf-pass-1245.md) | CAMBI Anti-Dithering AVX2 Vectorization, SpEED SIMD QR Dispatch, and Threaded GPU Flush Alignment | +| [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | diff --git a/mkdocs.yml b/mkdocs.yml index f226f03f3..4ee4c48fe 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1381,6 +1381,7 @@ nav: - "ADR-1254: win64-cannot-realign-the-stack": adr/1254-win64-cannot-realign-the-stack.md - "ADR-1255: spdx-residual-identifier-correction": adr/1255-spdx-residual-identifier-correction.md - "ADR-1256: cambi-spatial-mask-simd-dispatch": adr/1256-cambi-spatial-mask-simd-dispatch.md + - "ADR-1258: keep-64-bit-only-retire-i686-lane": adr/1258-keep-64-bit-only-retire-i686-lane.md - "By tag": - Overview: adr/by-tag/index.md - "aarch64": adr/by-tag/aarch64.md From e417726a65cea0786e49527ad06d2c8d2112bde4 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 21:55:57 +0200 Subject: [PATCH 02/15] fix(simd): stop using x86-64-only intrinsics in the AVX2 and AVX-512 sources adm_avx2.c and adm_avx512.c called _mm_extract_epi64 on 12 lines each, next to an extract_epi64() fallback that covered only the 256-bit form, and psnr_avx2.c called _mm_cvtsi128_si64. GCC declares both only on x86-64, which is Netflix#1481: no 32-bit x86 build with asm. extract_epi64_128() maps to _mm_extract_epi64 on x86-64 and to a 32-bit fallback elsewhere; the PSNR sum is read with _mm_storel_epi64. x86-64 code is unchanged. The fork stays 64-bit only (ADR-1258); this keeps the sources portable. --- core/src/feature/x86/adm_avx2.c | 34 ++++++++++++++++++++----------- core/src/feature/x86/adm_avx512.c | 34 ++++++++++++++++++++----------- core/src/feature/x86/psnr_avx2.c | 4 +++- 3 files changed, 47 insertions(+), 25 deletions(-) diff --git a/core/src/feature/x86/adm_avx2.c b/core/src/feature/x86/adm_avx2.c index 6ae7f5d71..ff6d7dcf2 100644 --- a/core/src/feature/x86/adm_avx2.c +++ b/core/src/feature/x86/adm_avx2.c @@ -1490,6 +1490,7 @@ static inline __m256i sra_epi64(__m256i a, __m256i mask) #if defined(__x86_64__) || defined(_M_X64) || defined(_M_AMD64) #define extract_epi64 _mm256_extract_epi64 +#define extract_epi64_128 _mm_extract_epi64 #else static inline int64_t extract_epi64(__m256i a, const int index) { @@ -1517,6 +1518,15 @@ static inline int64_t extract_epi64(__m256i a, const int index) return 0; } } + +/* 128-bit twin of the fallback above: _mm_extract_epi64 is x86-64 only too. */ +static inline int64_t extract_epi64_128(__m128i a, const int index) +{ + if (index == 0) { + return ((uint64_t)_mm_extract_epi32(a, 1) << 32) | (unsigned)_mm_extract_epi32(a, 0); + } + return ((uint64_t)_mm_extract_epi32(a, 3) << 32) | (unsigned)_mm_extract_epi32(a, 2); +} #endif // No lzcnt in avx2 @@ -2530,19 +2540,19 @@ float adm_cm_avx2(AdmBuffer *buf, int w, int h, int src_stride, int csf_a_stride __m128i r2_h = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_h_lo_256), _mm256_extracti128_si256(accum_inner_h_lo_256, 1)); int64_t res_h = - (int64_t)_mm_extract_epi64(r2_h, 0) + (int64_t)_mm_extract_epi64(r2_h, 1); + (int64_t)extract_epi64_128(r2_h, 0) + (int64_t)extract_epi64_128(r2_h, 1); accum_inner_v_lo_256 = _mm256_add_epi64(accum_inner_v_lo_256, accum_inner_v_hi_256); __m128i r2_v = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_v_lo_256), _mm256_extracti128_si256(accum_inner_v_lo_256, 1)); int64_t res_v = - (int64_t)_mm_extract_epi64(r2_v, 0) + (int64_t)_mm_extract_epi64(r2_v, 1); + (int64_t)extract_epi64_128(r2_v, 0) + (int64_t)extract_epi64_128(r2_v, 1); accum_inner_d_lo_256 = _mm256_add_epi64(accum_inner_d_lo_256, accum_inner_d_hi_256); __m128i r2_d = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_d_lo_256), _mm256_extracti128_si256(accum_inner_d_lo_256, 1)); int64_t res_d = - (int64_t)_mm_extract_epi64(r2_d, 0) + (int64_t)_mm_extract_epi64(r2_d, 1); + (int64_t)extract_epi64_128(r2_d, 0) + (int64_t)extract_epi64_128(r2_d, 1); for (j = end_col_mod6; j < end_col; ++j) { xh = src->band_h[i * src_stride + j] * i_rfactor[0]; @@ -3026,17 +3036,17 @@ float i4_adm_cm_avx2(AdmBuffer *buf, int w, int h, int src_stride, int csf_a_str __m128i r2_h = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_h_256), _mm256_extracti128_si256(accum_inner_h_256, 1)); int64_t res_h = - (int64_t)_mm_extract_epi64(r2_h, 0) + (int64_t)_mm_extract_epi64(r2_h, 1); + (int64_t)extract_epi64_128(r2_h, 0) + (int64_t)extract_epi64_128(r2_h, 1); __m128i r2_v = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_v_256), _mm256_extracti128_si256(accum_inner_v_256, 1)); int64_t res_v = - (int64_t)_mm_extract_epi64(r2_v, 0) + (int64_t)_mm_extract_epi64(r2_v, 1); + (int64_t)extract_epi64_128(r2_v, 0) + (int64_t)extract_epi64_128(r2_v, 1); __m128i r2_d = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_d_256), _mm256_extracti128_si256(accum_inner_d_256, 1)); int64_t res_d = - (int64_t)_mm_extract_epi64(r2_d, 0) + (int64_t)_mm_extract_epi64(r2_d, 1); + (int64_t)extract_epi64_128(r2_d, 0) + (int64_t)extract_epi64_128(r2_d, 1); for (j = end_col_mod2; j < end_col; ++j) { xh = (int32_t)((((int64_t)src->band_h[i * src_stride + j] * rfactor[0]) + @@ -4221,17 +4231,17 @@ float adm_csf_den_scale_avx2(const adm_dwt_band_t *src, int w, int h, int src_st accum_inner_h_lo = _mm256_add_epi64(accum_inner_h_lo, accum_inner_h_hi); __m128i h_r2 = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_h_lo), _mm256_extracti128_si256(accum_inner_h_lo, 1)); - uint64_t h_r1 = (uint64_t)_mm_extract_epi64(h_r2, 0) + (uint64_t)_mm_extract_epi64(h_r2, 1); + uint64_t h_r1 = (uint64_t)extract_epi64_128(h_r2, 0) + (uint64_t)extract_epi64_128(h_r2, 1); accum_inner_v_lo = _mm256_add_epi64(accum_inner_v_lo, accum_inner_v_hi); __m128i v_r2 = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_v_lo), _mm256_extracti128_si256(accum_inner_v_lo, 1)); - uint64_t v_r1 = (uint64_t)_mm_extract_epi64(v_r2, 0) + (uint64_t)_mm_extract_epi64(v_r2, 1); + uint64_t v_r1 = (uint64_t)extract_epi64_128(v_r2, 0) + (uint64_t)extract_epi64_128(v_r2, 1); accum_inner_d_lo = _mm256_add_epi64(accum_inner_d_lo, accum_inner_d_hi); __m128i d_r2 = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_d_lo), _mm256_extracti128_si256(accum_inner_d_lo, 1)); - uint64_t d_r1 = (uint64_t)_mm_extract_epi64(d_r2, 0) + (uint64_t)_mm_extract_epi64(d_r2, 1); + uint64_t d_r1 = (uint64_t)extract_epi64_128(d_r2, 0) + (uint64_t)extract_epi64_128(d_r2, 1); for (int j = right_mod_8; j < right; ++j) { uint16_t h_abs = (uint16_t)abs(src_h[j]); @@ -4788,15 +4798,15 @@ float adm_csf_den_s123_avx2(const i4_adm_dwt_band_t *src, int scale, int w, int } __m128i h_r2 = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_h_256), _mm256_extracti128_si256(accum_inner_h_256, 1)); - uint64_t h_r1 = (uint64_t)_mm_extract_epi64(h_r2, 0) + (uint64_t)_mm_extract_epi64(h_r2, 1); + uint64_t h_r1 = (uint64_t)extract_epi64_128(h_r2, 0) + (uint64_t)extract_epi64_128(h_r2, 1); __m128i d_r2 = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_d_256), _mm256_extracti128_si256(accum_inner_d_256, 1)); - uint64_t d_r1 = (uint64_t)_mm_extract_epi64(d_r2, 0) + (uint64_t)_mm_extract_epi64(d_r2, 1); + uint64_t d_r1 = (uint64_t)extract_epi64_128(d_r2, 0) + (uint64_t)extract_epi64_128(d_r2, 1); __m128i v_r2 = _mm_add_epi64(_mm256_castsi256_si128(accum_inner_v_256), _mm256_extracti128_si256(accum_inner_v_256, 1)); - uint64_t v_r1 = (uint64_t)_mm_extract_epi64(v_r2, 0) + (uint64_t)_mm_extract_epi64(v_r2, 1); + uint64_t v_r1 = (uint64_t)extract_epi64_128(v_r2, 0) + (uint64_t)extract_epi64_128(v_r2, 1); for (int j = right_mod_4; j < right; ++j) { uint32_t h_abs = (uint32_t)abs(src_h[j]); diff --git a/core/src/feature/x86/adm_avx512.c b/core/src/feature/x86/adm_avx512.c index b7f355dd1..a3b9fbae8 100644 --- a/core/src/feature/x86/adm_avx512.c +++ b/core/src/feature/x86/adm_avx512.c @@ -1161,6 +1161,7 @@ static inline uint16_t get_best15_from32(uint32_t temp, int *x) #if defined(__x86_64__) || defined(_M_X64) || defined(_M_AMD64) #define extract_epi64 _mm256_extract_epi64 +#define extract_epi64_128 _mm_extract_epi64 #else static inline int64_t extract_epi64(__m256i a, const int index) { @@ -1188,6 +1189,15 @@ static inline int64_t extract_epi64(__m256i a, const int index) return 0; } } + +/* 128-bit twin of the fallback above: _mm_extract_epi64 is x86-64 only too. */ +static inline int64_t extract_epi64_128(__m128i a, const int index) +{ + if (index == 0) { + return ((uint64_t)_mm_extract_epi32(a, 1) << 32) | (unsigned)_mm_extract_epi32(a, 0); + } + return ((uint64_t)_mm_extract_epi32(a, 3) << 32) | (unsigned)_mm_extract_epi32(a, 2); +} #endif // NOLINTNEXTLINE(readability-function-size) — bit-exactness invariant: splitting would perturb register allocation + reduction order vs scalar (ADR-0138/0139, ADR-0141) @@ -2133,7 +2143,7 @@ float adm_cm_avx512(AdmBuffer *buf, int w, int h, int src_stride, int csf_a_stri __m128i r2_h = _mm_add_epi64(_mm256_castsi256_si128(r4_h), _mm256_extracti128_si256(r4_h, 1)); int64_t res_h = - (int64_t)_mm_extract_epi64(r2_h, 0) + (int64_t)_mm_extract_epi64(r2_h, 1); + (int64_t)extract_epi64_128(r2_h, 0) + (int64_t)extract_epi64_128(r2_h, 1); accum_inner_v_lo_512 = _mm512_add_epi64(accum_inner_v_lo_512, accum_inner_v_hi_512); __m256i r4_v = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_v_lo_512), @@ -2141,7 +2151,7 @@ float adm_cm_avx512(AdmBuffer *buf, int w, int h, int src_stride, int csf_a_stri __m128i r2_v = _mm_add_epi64(_mm256_castsi256_si128(r4_v), _mm256_extracti128_si256(r4_v, 1)); int64_t res_v = - (int64_t)_mm_extract_epi64(r2_v, 0) + (int64_t)_mm_extract_epi64(r2_v, 1); + (int64_t)extract_epi64_128(r2_v, 0) + (int64_t)extract_epi64_128(r2_v, 1); accum_inner_d_lo_512 = _mm512_add_epi64(accum_inner_d_lo_512, accum_inner_d_hi_512); __m256i r4_d = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_d_lo_512), @@ -2149,7 +2159,7 @@ float adm_cm_avx512(AdmBuffer *buf, int w, int h, int src_stride, int csf_a_stri __m128i r2_d = _mm_add_epi64(_mm256_castsi256_si128(r4_d), _mm256_extracti128_si256(r4_d, 1)); int64_t res_d = - (int64_t)_mm_extract_epi64(r2_d, 0) + (int64_t)_mm_extract_epi64(r2_d, 1); + (int64_t)extract_epi64_128(r2_d, 0) + (int64_t)extract_epi64_128(r2_d, 1); for (j = end_col_mod14; j < end_col; ++j) { xh = src->band_h[i * src_stride + j] * i_rfactor[0]; @@ -2611,21 +2621,21 @@ float i4_adm_cm_avx512(AdmBuffer *buf, int w, int h, int src_stride, int csf_a_s __m128i r2_h = _mm_add_epi64(_mm256_castsi256_si128(r4_h), _mm256_extracti128_si256(r4_h, 1)); int64_t res_h = - (int64_t)_mm_extract_epi64(r2_h, 0) + (int64_t)_mm_extract_epi64(r2_h, 1); + (int64_t)extract_epi64_128(r2_h, 0) + (int64_t)extract_epi64_128(r2_h, 1); __m256i r4_v = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_v_512), _mm512_extracti64x4_epi64(accum_inner_v_512, 1)); __m128i r2_v = _mm_add_epi64(_mm256_castsi256_si128(r4_v), _mm256_extracti128_si256(r4_v, 1)); int64_t res_v = - (int64_t)_mm_extract_epi64(r2_v, 0) + (int64_t)_mm_extract_epi64(r2_v, 1); + (int64_t)extract_epi64_128(r2_v, 0) + (int64_t)extract_epi64_128(r2_v, 1); __m256i r4_d = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_d_512), _mm512_extracti64x4_epi64(accum_inner_d_512, 1)); __m128i r2_d = _mm_add_epi64(_mm256_castsi256_si128(r4_d), _mm256_extracti128_si256(r4_d, 1)); int64_t res_d = - (int64_t)_mm_extract_epi64(r2_d, 0) + (int64_t)_mm_extract_epi64(r2_d, 1); + (int64_t)extract_epi64_128(r2_d, 0) + (int64_t)extract_epi64_128(r2_d, 1); for (j = end_col_mod6; j < end_col; ++j) { xh = (int32_t)((((int64_t)src->band_h[i * src_stride + j] * rfactor[0]) + @@ -4546,21 +4556,21 @@ float adm_csf_den_scale_avx512(const adm_dwt_band_t *src, int w, int h, int src_ _mm512_extracti64x4_epi64(accum_inner_h_lo, 1)); __m128i h_r2 = _mm_add_epi64(_mm256_castsi256_si128(h_r4), _mm256_extracti64x2_epi64(h_r4, 1)); - uint64_t h_r1 = (uint64_t)_mm_extract_epi64(h_r2, 0) + (uint64_t)_mm_extract_epi64(h_r2, 1); + uint64_t h_r1 = (uint64_t)extract_epi64_128(h_r2, 0) + (uint64_t)extract_epi64_128(h_r2, 1); accum_inner_v_lo = _mm512_add_epi64(accum_inner_v_lo, accum_inner_v_hi); __m256i v_r4 = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_v_lo), _mm512_extracti64x4_epi64(accum_inner_v_lo, 1)); __m128i v_r2 = _mm_add_epi64(_mm256_castsi256_si128(v_r4), _mm256_extracti64x2_epi64(v_r4, 1)); - uint64_t v_r1 = (uint64_t)_mm_extract_epi64(v_r2, 0) + (uint64_t)_mm_extract_epi64(v_r2, 1); + uint64_t v_r1 = (uint64_t)extract_epi64_128(v_r2, 0) + (uint64_t)extract_epi64_128(v_r2, 1); accum_inner_d_lo = _mm512_add_epi64(accum_inner_d_lo, accum_inner_d_hi); __m256i d_r4 = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_d_lo), _mm512_extracti64x4_epi64(accum_inner_d_lo, 1)); __m128i d_r2 = _mm_add_epi64(_mm256_castsi256_si128(d_r4), _mm256_extracti64x2_epi64(d_r4, 1)); - uint64_t d_r1 = (uint64_t)_mm_extract_epi64(d_r2, 0) + (uint64_t)_mm_extract_epi64(d_r2, 1); + uint64_t d_r1 = (uint64_t)extract_epi64_128(d_r2, 0) + (uint64_t)extract_epi64_128(d_r2, 1); for (int j = right_mod_16; j < right; ++j) { uint16_t h_abs = (uint16_t)abs(src_h[j]); @@ -4707,19 +4717,19 @@ float adm_csf_den_s123_avx512(const i4_adm_dwt_band_t *src, int scale, int w, in _mm512_extracti64x4_epi64(accum_inner_h_512, 1)); __m128i h_r2 = _mm_add_epi64(_mm256_castsi256_si128(h_r4), _mm256_extracti64x2_epi64(h_r4, 1)); - uint64_t h_r1 = (uint64_t)_mm_extract_epi64(h_r2, 0) + (uint64_t)_mm_extract_epi64(h_r2, 1); + uint64_t h_r1 = (uint64_t)extract_epi64_128(h_r2, 0) + (uint64_t)extract_epi64_128(h_r2, 1); __m256i d_r4 = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_d_512), _mm512_extracti64x4_epi64(accum_inner_d_512, 1)); __m128i d_r2 = _mm_add_epi64(_mm256_castsi256_si128(d_r4), _mm256_extracti64x2_epi64(d_r4, 1)); - uint64_t d_r1 = (uint64_t)_mm_extract_epi64(d_r2, 0) + (uint64_t)_mm_extract_epi64(d_r2, 1); + uint64_t d_r1 = (uint64_t)extract_epi64_128(d_r2, 0) + (uint64_t)extract_epi64_128(d_r2, 1); __m256i v_r4 = _mm256_add_epi64(_mm512_castsi512_si256(accum_inner_v_512), _mm512_extracti64x4_epi64(accum_inner_v_512, 1)); __m128i v_r2 = _mm_add_epi64(_mm256_castsi256_si128(v_r4), _mm256_extracti64x2_epi64(v_r4, 1)); - uint64_t v_r1 = (uint64_t)_mm_extract_epi64(v_r2, 0) + (uint64_t)_mm_extract_epi64(v_r2, 1); + uint64_t v_r1 = (uint64_t)extract_epi64_128(v_r2, 0) + (uint64_t)extract_epi64_128(v_r2, 1); for (int j = right_mod_8; j < right; ++j) { uint32_t h_abs = (uint32_t)abs(src_h[j]); diff --git a/core/src/feature/x86/psnr_avx2.c b/core/src/feature/x86/psnr_avx2.c index 53db944d7..932ee8655 100644 --- a/core/src/feature/x86/psnr_avx2.c +++ b/core/src/feature/x86/psnr_avx2.c @@ -107,7 +107,9 @@ uint64_t psnr_sse_line_16_avx2(const uint16_t *ref, const uint16_t *dis, unsigne __m128i thi = _mm256_extracti128_si256(total, 1); __m128i t128 = _mm_add_epi64(tlo, thi); t128 = _mm_add_epi64(t128, _mm_shuffle_epi32(t128, 0x4E)); - uint64_t result = (uint64_t)_mm_cvtsi128_si64(t128); + /* _mm_storel_epi64, not _mm_cvtsi128_si64: the latter is x86-64 only. */ + uint64_t result = 0; + _mm_storel_epi64((__m128i *)&result, t128); /* Scalar tail — use unsigned abs-diff to avoid signed-int overflow UB * (65535^2 > INT32_MAX); mirrors sse_line_16_c in integer_psnr.c. */ From ea0224c3de7c1d3ffb059895f4258a2c327165c7 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:42:32 +0200 Subject: [PATCH 03/15] chore(standards): re-record the HISS baseline for moved ADM function lines The 32-bit-safe extract_epi64_128() helper in adm_avx2.c and adm_avx512.c moves 19 baselined HISS-04 functions down by 10 lines. Their sizes are unchanged and the total stays at 1627 infractions, so only the recorded line numbers change. Recorded with the pinned engine (~/.cache/vmafx-praetor/praetorctl baseline --record, e4b35cb3c7fe) in a fresh clone of the rebased branch, per the ADR-1249 rebase note. --- .standards-baseline.json | 160 ++++++++++++++++++++++++--------------- 1 file changed, 100 insertions(+), 60 deletions(-) diff --git a/.standards-baseline.json b/.standards-baseline.json index a2b842736..8f77e967d 100644 --- a/.standards-baseline.json +++ b/.standards-baseline.json @@ -1,9 +1,9 @@ { "version": 1, - "generated_at": "2026-09-19T09:52:52Z", + "generated_at": "2026-09-18T20:40:41Z", "repository": "", "commit_sha": "", - "total_infractions": 1622, + "total_infractions": 1627, "infractions": [ { "rule_id": "HISS-04", @@ -2581,6 +2581,14 @@ "message": "Function '{' (212 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", "fingerprint": "core/src/feature/arm64/adm_neon.c:64:HISS-04" }, + { + "rule_id": "HISS-04", + "file_path": "core/src/feature/arm64/cambi_neon.c", + "line_number": 125, + "symbol": "{", + "message": "Function '{' (86 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", + "fingerprint": "core/src/feature/arm64/cambi_neon.c:125:HISS-04" + }, { "rule_id": "HISS-04", "file_path": "core/src/feature/arm64/float_adm_dwt2_neon.c", @@ -2709,6 +2717,38 @@ "message": "Function '{' (114 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", "fingerprint": "core/src/feature/arm64/ssimulacra2_sve2.c:673:HISS-04" }, + { + "rule_id": "HISS-04", + "file_path": "core/src/feature/arm64/vif_neon.c", + "line_number": 292, + "symbol": "vif_subsample_rd_8_neon", + "message": "Function 'vif_subsample_rd_8_neon' (132 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", + "fingerprint": "core/src/feature/arm64/vif_neon.c:292:HISS-04" + }, + { + "rule_id": "HISS-04", + "file_path": "core/src/feature/arm64/vif_neon.c", + "line_number": 427, + "symbol": "{", + "message": "Function '{' (137 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", + "fingerprint": "core/src/feature/arm64/vif_neon.c:427:HISS-04" + }, + { + "rule_id": "HISS-04", + "file_path": "core/src/feature/arm64/vif_neon.c", + "line_number": 566, + "symbol": "vif_statistic_8_neon", + "message": "Function 'vif_statistic_8_neon' (326 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", + "fingerprint": "core/src/feature/arm64/vif_neon.c:566:HISS-04" + }, + { + "rule_id": "HISS-04", + "file_path": "core/src/feature/arm64/vif_neon.c", + "line_number": 895, + "symbol": "{", + "message": "Function '{' (326 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", + "fingerprint": "core/src/feature/arm64/vif_neon.c:895:HISS-04" + }, { "rule_id": "HISS-04", "file_path": "core/src/feature/brisque_math.h", @@ -2720,53 +2760,53 @@ { "rule_id": "HISS-04", "file_path": "core/src/feature/cambi.c", - "line_number": 196, + "line_number": 189, "symbol": "static const VmafOption options[] = {", "message": "Function 'static const VmafOption options[] = {' (184 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/cambi.c:196:HISS-04" + "fingerprint": "core/src/feature/cambi.c:189:HISS-04" }, { "rule_id": "HISS-02", "file_path": "core/src/feature/cambi.c", - "line_number": 440, + "line_number": 433, "message": "Legacy unbounded loop in native code", - "fingerprint": "core/src/feature/cambi.c:440:HISS-02" + "fingerprint": "core/src/feature/cambi.c:433:HISS-02" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 857, + "line_number": 834, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:857:HISS-01" + "fingerprint": "core/src/feature/cambi.c:834:HISS-01" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 863, + "line_number": 840, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:863:HISS-01" + "fingerprint": "core/src/feature/cambi.c:840:HISS-01" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 870, + "line_number": 847, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:870:HISS-01" + "fingerprint": "core/src/feature/cambi.c:847:HISS-01" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 874, + "line_number": 851, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:874:HISS-01" + "fingerprint": "core/src/feature/cambi.c:851:HISS-01" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/cambi.h", - "line_number": 49, + "line_number": 40, "symbol": "static const float reciprocal_lut[CAMBI_RECIPROCAL_LUT_SIZE] = {0.0f,", "message": "Function 'static const float reciprocal_lut[CAMBI_RECIPROCAL_LUT_SIZE] = {0.0f,' (4226 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/cambi.h:49:HISS-04" + "fingerprint": "core/src/feature/cambi.h:40:HISS-04" }, { "rule_id": "HISS-04", @@ -7345,74 +7385,74 @@ { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 1526, + "line_number": 1536, "symbol": "{", "message": "Function '{' (747 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:1526:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:1536:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 2298, + "line_number": 2308, "symbol": "{", "message": "Function '{' (483 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:2298:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:2308:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 2787, + "line_number": 2797, "symbol": "{", "message": "Function '{' (551 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:2787:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:2797:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 3342, + "line_number": 3352, "symbol": "{", "message": "Function '{' (242 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:3342:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:3352:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 3589, + "line_number": 3599, "symbol": "{", "message": "Function '{' (532 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:3589:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:3599:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 4127, + "line_number": 4137, "symbol": "{", "message": "Function '{' (155 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:4127:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:4137:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 4287, + "line_number": 4297, "symbol": "{", "message": "Function '{' (179 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:4287:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:4297:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 4471, + "line_number": 4481, "symbol": "{", "message": "Function '{' (223 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:4471:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:4481:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx2.c", - "line_number": 4700, + "line_number": 4710, "symbol": "{", "message": "Function '{' (151 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx2.c:4700:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx2.c:4710:HISS-04" }, { "rule_id": "HISS-04", @@ -7441,82 +7481,82 @@ { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 1196, + "line_number": 1206, "symbol": "{", "message": "Function '{' (682 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:1196:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:1206:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 1903, + "line_number": 1913, "symbol": "{", "message": "Function '{' (476 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:1903:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:1913:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 2385, + "line_number": 2395, "symbol": "{", "message": "Function '{' (542 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:2385:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:2395:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 2953, + "line_number": 2963, "symbol": "{", "message": "Function '{' (492 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:2953:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:2963:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 3449, + "line_number": 3459, "symbol": "{", "message": "Function '{' (242 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:3449:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:3459:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 3716, + "line_number": 3726, "symbol": "{", "message": "Function '{' (319 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:3716:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:3726:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 4040, + "line_number": 4050, "symbol": "{", "message": "Function '{' (173 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:4040:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:4050:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 4218, + "line_number": 4228, "symbol": "{", "message": "Function '{' (226 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:4218:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:4228:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 4450, + "line_number": 4460, "symbol": "{", "message": "Function '{' (161 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:4450:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:4460:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/x86/adm_avx512.c", - "line_number": 4617, + "line_number": 4627, "symbol": "{", "message": "Function '{' (157 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/x86/adm_avx512.c:4617:HISS-04" + "fingerprint": "core/src/feature/x86/adm_avx512.c:4627:HISS-04" }, { "rule_id": "HISS-04", @@ -9737,10 +9777,10 @@ { "rule_id": "HISS-04", "file_path": "core/test/test_psnr_hvs_simd.c", - "line_number": 164, + "line_number": 157, "symbol": "{", "message": "Function '{' (115 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/test/test_psnr_hvs_simd.c:164:HISS-04" + "fingerprint": "core/test/test_psnr_hvs_simd.c:157:HISS-04" }, { "rule_id": "HISS-04", @@ -9769,18 +9809,18 @@ { "rule_id": "HISS-04", "file_path": "core/test/test_ssimulacra2_simd.c", - "line_number": 768, + "line_number": 758, "symbol": "{", "message": "Function '{' (76 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/test/test_ssimulacra2_simd.c:768:HISS-04" + "fingerprint": "core/test/test_ssimulacra2_simd.c:758:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/test/test_ssimulacra2_simd.c", - "line_number": 874, + "line_number": 864, "symbol": "test_ptlr_one", "message": "Function 'test_ptlr_one' (104 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/test/test_ssimulacra2_simd.c:874:HISS-04" + "fingerprint": "core/test/test_ssimulacra2_simd.c:864:HISS-04" }, { "rule_id": "HISS-04", From cb9401b31cf113b4bd03103e40625820c7f1e0f7 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 21:56:02 +0200 Subject: [PATCH 04/15] ci: remove the resurrected i686 lane and preflight's m32 stage ADR-0691 retired 32-bit x86; the libvmaf/ -> core/ rename merge (384d97d03) restored the lane the same day. It ran compile-only with -Denable_asm=false and no tests. Per ADR-1258 the fork stays 64-bit only: the matrix row, its dependency step and its matrix.i686 conditions go, and preflight drops the m32 stage that mirrored the lane. AGENTS.md (recompiled into CLAUDE.md and the other agent context files), the preflight page and the ADM page no longer describe 32-bit as covered. The shellcheck findings actionlint reports in this workflow are fixed on the way: quoted redirections and substitutions, and pkg-config flags read into arrays instead of relying on word splitting. --- .codex/rules.md | 2 +- .cursor/rules/hiss-invariants.mdc | 2 +- .gemini/GEMINI.md | 2 +- .github/copilot-instructions.md | 2 +- .github/workflows/libvmaf-build-matrix.yml | 68 +++++++--------------- .windsurfrules | 2 +- AGENTS.md | 2 +- CLAUDE.md | 2 +- core/src/meson.build | 2 +- docs/development/preflight.md | 25 ++------ docs/metrics/features.md | 20 +++---- scripts/dev/preflight.sh | 59 +------------------ 12 files changed, 44 insertions(+), 144 deletions(-) diff --git a/.codex/rules.md b/.codex/rules.md index 4826b33f1..174cd2497 100644 --- a/.codex/rules.md +++ b/.codex/rules.md @@ -136,7 +136,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/.cursor/rules/hiss-invariants.mdc b/.cursor/rules/hiss-invariants.mdc index bfade9e94..1bbd806ac 100644 --- a/.cursor/rules/hiss-invariants.mdc +++ b/.cursor/rules/hiss-invariants.mdc @@ -142,7 +142,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/.gemini/GEMINI.md b/.gemini/GEMINI.md index 4826b33f1..174cd2497 100644 --- a/.gemini/GEMINI.md +++ b/.gemini/GEMINI.md @@ -136,7 +136,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 4826b33f1..174cd2497 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -136,7 +136,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/.github/workflows/libvmaf-build-matrix.yml b/.github/workflows/libvmaf-build-matrix.yml index 2332343cf..fd0c8419c 100644 --- a/.github/workflows/libvmaf-build-matrix.yml +++ b/.github/workflows/libvmaf-build-matrix.yml @@ -62,22 +62,6 @@ jobs: CXX: ccache clang++-22 name: Ubuntu ARM clang - # --- i686 (32-bit x86) build — reproduces Netflix#1481 --- - # Verifies libvmaf still compiles on 32-bit x86 with - # -Denable_asm=false. The documented workaround for the - # `_mm256_extract_epi64` implicit-declaration failure on - # 32-bit x86 is to disable the asm path; this job pins - # that contract so any future 32-bit-only regression trips - # the matrix. Tests are skipped (meson cross-build marks - # them SKIP 77 — host can run i686 binaries natively but - # meson doesn't know that). See ADR-0151. - - os: ubuntu-latest - CC: ccache gcc-14 - CXX: ccache g++-14 - i686: true - meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false - name: Ubuntu i686 gcc - # --- DNN-enabled builds (ADR-0120) --- # Build with -Denable_dnn=enabled across compiler/OS combinations # to ensure the ORT C-API surface compiles on every CPU leg, not @@ -273,7 +257,7 @@ jobs: pip install meson - name: Install dependencies (ubuntu) - if: ${{ (startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.i686 && !matrix.hip) && steps.impact.outputs.c_core == 'true' }} + if: ${{ (startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip) && steps.impact.outputs.c_core == 'true' }} run: | sudo apt-get update sudo -E apt-get -yq install ccache ninja-build nasm libomp-dev pkg-config @@ -297,14 +281,6 @@ jobs: *clang) sudo -E apt-get -yq install clang ;; esac - - name: Install dependencies (ubuntu i686) - if: ${{ (matrix.i686) && steps.impact.outputs.c_core == 'true' }} - run: | - sudo dpkg --add-architecture i386 - sudo apt-get update - sudo -E apt-get -yq install ccache ninja-build pkg-config \ - gcc g++ gcc-multilib g++-multilib - - name: Install dependencies (mac) if: ${{ (matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true' }} env: @@ -378,7 +354,7 @@ jobs: # to preserve the rest of the DNN suite for macOS coverage. brew install -q onnxruntime ORT_PREFIX=$(brew --prefix onnxruntime) - echo "PKG_CONFIG_PATH=${ORT_PREFIX}/lib/pkgconfig" >> $GITHUB_ENV + echo "PKG_CONFIG_PATH=${ORT_PREFIX}/lib/pkgconfig" >> "$GITHUB_ENV" # ---------- Intel oneAPI (SYCL) ---------- # Pinned to oneAPI 2025.3 (icpx 2025.3.x) and Level Zero v1.28.0. @@ -415,10 +391,10 @@ jobs: git clone --depth=1 --branch "v${LEVEL_ZERO_VERSION}" \ https://github.com/oneapi-src/level-zero.git /tmp/level-zero cmake -S /tmp/level-zero -B /tmp/level-zero/build -DCMAKE_INSTALL_PREFIX=/usr - cmake --build /tmp/level-zero/build -j$(nproc) + cmake --build /tmp/level-zero/build -j"$(nproc)" sudo cmake --install /tmp/level-zero/build rm -rf /tmp/level-zero - echo "/opt/intel/oneapi/compiler/latest/bin" >> $GITHUB_PATH + echo "/opt/intel/oneapi/compiler/latest/bin" >> "$GITHUB_PATH" echo "/opt/intel/oneapi/compiler/latest/lib" | sudo tee /etc/ld.so.conf.d/oneapi.conf sudo ldconfig @@ -525,7 +501,7 @@ jobs: source /opt/intel/oneapi/setvars.sh fi meson setup core core/build --buildtype release \ - --prefix $PWD/install \ + --prefix "$PWD/install" \ -Denable_float=true \ ${{ matrix.meson_extra }} @@ -552,7 +528,7 @@ jobs: run: | export PKG_CONFIG_PATH=$PWD/install/lib/pkgconfig:$PWD/install/lib/x86_64-linux-gnu/pkgconfig echo "=== libvmaf.pc ===" - cat $PWD/install/lib/pkgconfig/libvmaf.pc || cat $PWD/install/lib/x86_64-linux-gnu/pkgconfig/libvmaf.pc + cat "$PWD/install/lib/pkgconfig/libvmaf.pc" || cat "$PWD/install/lib/x86_64-linux-gnu/pkgconfig/libvmaf.pc" echo "" echo "=== pkg-config --static --libs libvmaf ===" pkg-config --static --libs libvmaf @@ -585,8 +561,12 @@ jobs: # SYCL/CUDA legs, so the archive holds plain objects rather than LTO # IR — a plugin mismatch is not in play, but matching the compiler # still costs nothing.) - ${CC:-cc} /tmp/pc_static_smoke.c $(pkg-config --cflags libvmaf) \ - $(pkg-config --static --libs libvmaf) -o /tmp/pc_static_smoke + # pkg-config prints space-separated flags; read them into arrays + # rather than relying on unquoted word splitting. + read -ra pc_cflags <<< "$(pkg-config --cflags libvmaf)" + read -ra pc_libs <<< "$(pkg-config --static --libs libvmaf)" + ${CC:-cc} /tmp/pc_static_smoke.c "${pc_cflags[@]}" "${pc_libs[@]}" \ + -o /tmp/pc_static_smoke echo "static link OK" # DNN legs: run the dedicated dnn suite first so its logs are easy @@ -600,14 +580,11 @@ jobs: # user's $HOME so a bare `sudo meson` reports "command not found". # Sudo is needed because the preceding `sudo ninja install` step # left build artifacts owned by root. - run: sudo $(which meson) test -C core/build --suite=dnn --print-errorlogs + run: sudo "$(which meson)" test -C core/build --suite=dnn --print-errorlogs - # CPU-only: full test suite + tox (skipped for i686 cross-build - # because meson marks cross-compiled tests SKIP 77 regardless of - # whether the host can actually run them; the build itself is - # what Netflix#1481 needs to pin). + # CPU-only: full test suite + tox. - name: Run tests - if: ${{ (!matrix.sycl && !matrix.cuda && !matrix.hip && !matrix.i686) && steps.impact.outputs.c_core == 'true' }} + if: ${{ (!matrix.sycl && !matrix.cuda && !matrix.hip) && steps.impact.outputs.c_core == 'true' }} run: sudo ninja -vC core/build test # ---------- SSH debug session (macOS × workflow_dispatch only) ---------- @@ -650,7 +627,7 @@ jobs: # Insulates the tox lanes from the recurring HTTP-502s on # raw.githubusercontent.com when fetching python/test/resource/ # fixtures via download_reactively (compat/python-vmaf/config.py). - if: ${{ ((startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip && !matrix.i686) || matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true' }} + if: ${{ ((startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip) || matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true' }} uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: python/test/resource @@ -666,20 +643,20 @@ jobs: # never repaired and the test fails with "no frames decoded". The # success()-gated save below stops new poisoning; this heals caches # already poisoned, which otherwise need a manual cache deletion. - if: ${{ ((startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip && !matrix.i686) || matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true' }} + if: ${{ ((startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip) || matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true' }} run: bash scripts/ci/prune-corrupt-fixtures.sh python/test/resource - name: Run tox tests (ubuntu) - if: ${{ (startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip && !matrix.i686) && steps.impact.outputs.c_core == 'true' }} + if: ${{ (startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip) && steps.impact.outputs.c_core == 'true' }} run: | - mkdir -p ~/.ccache && sudo chown -R $(whoami) ~/.ccache + mkdir -p ~/.ccache && sudo chown -R "$(whoami)" ~/.ccache pip install tox tox -c python - name: Run tox tests (mac) if: ${{ (matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true' }} run: | - mkdir -p ~/.ccache && sudo chown -R $(whoami) ~/.ccache + mkdir -p ~/.ccache && sudo chown -R "$(whoami)" ~/.ccache pip install tox PATH=$(brew --prefix)/opt/llvm/bin:$PATH \ CC=$(brew --prefix)/opt/llvm/bin/clang \ @@ -689,7 +666,7 @@ jobs: - name: Save Netflix vmaf_resource fixtures # success() only: a cancelled or failing run must never publish a # partial fixture tree under this key. Placed after both tox legs. - if: ${{ success() && (((startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip && !matrix.i686) || matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true') && steps.fixtures.outputs.cache-hit != 'true' }} + if: ${{ success() && (((startsWith(matrix.os, 'ubuntu') && !matrix.sycl && !matrix.cuda && !matrix.hip) || matrix.os == 'macos-latest') && steps.impact.outputs.c_core == 'true') && steps.fixtures.outputs.cache-hit != 'true' }} uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: python/test/resource @@ -745,9 +722,6 @@ jobs: fail-fast: false matrix: include: - # - msystem: MINGW32 - # MINGW_PACKAGE_PREFIX: mingw-w64-i686 - # CFLAGS: -msse2 -mfpmath=sse -mstackrealign # required-aggregator - name: Windows MinGW64 msystem: MINGW64 diff --git a/.windsurfrules b/.windsurfrules index 4826b33f1..174cd2497 100644 --- a/.windsurfrules +++ b/.windsurfrules @@ -136,7 +136,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/AGENTS.md b/AGENTS.md index 92b2bfc32..14ff6c886 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -133,7 +133,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/CLAUDE.md b/CLAUDE.md index 4826b33f1..174cd2497 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -136,7 +136,7 @@ make test-netflix-golden # Netflix CPU golden-data gate (see §8) ## 4. Lint / format ```text -make preflight # CI compiler matrix locally: clang, 32-bit, sanitizers, MSVC-hostile constructs, tidy, cppcheck +make preflight # CI compiler matrix locally: clang, sanitizers, MSVC-hostile constructs, tidy, cppcheck make lint # configured native + Python, shell, Markdown, Go and docs checks make format # clang-format + black + ruff (writes) make format-check # dry-run (CI / pre-commit) diff --git a/core/src/meson.build b/core/src/meson.build index 8832ec33f..19f1c80bc 100644 --- a/core/src/meson.build +++ b/core/src/meson.build @@ -78,7 +78,7 @@ endif is_avx512_enabled = get_option('enable_avx512') == true is_nvtx_enabled = get_option('enable_nvtx') == true # AVX-512 is a pure assembly path and requires enable_asm=true to have any -# effect. When enable_asm=false (e.g. i686 cross-compile or --no-asm builds) +# effect. When enable_asm=false (a --no-asm build) # silently disabling would produce a confusing build; warn and force-disable # instead of hard-erroring, so no-asm consumer builds configure cleanly. if is_avx512_enabled and not is_asm_enabled diff --git a/docs/development/preflight.md b/docs/development/preflight.md index be8305123..082ed8645 100644 --- a/docs/development/preflight.md +++ b/docs/development/preflight.md @@ -5,7 +5,7 @@ ```bash make preflight # everything, on the files you changed scripts/dev/preflight.sh --full # everything, on the whole tree -scripts/dev/preflight.sh --stage m32 # just one stage +scripts/dev/preflight.sh --stage clang # just one stage scripts/dev/preflight.sh --list # which CI context each stage mirrors ``` @@ -19,7 +19,7 @@ afternoon. All three were green under local gcc: | What was written | Locally | In CI | | --- | --- | --- | -| `static_assert(UINT_MAX <= SIZE_MAX/2/sizeof(ptr))` | fine on 64-bit | `Ubuntu i686 gcc` cannot compile it — the claim is false on 32-bit | +| `static_assert(UINT_MAX <= SIZE_MAX/2/sizeof(ptr))` | fine on 64-bit | the i686 lane of the time could not compile it (that lane is retired: the fork is 64-bit only, ADR-0728 / ADR-1258) | | `__declspec(align((x)))` | gcc never compiles the MSVC branch | `Windows MSVC+CUDA` → C2059 on every use; `align()` needs a literal | | `__attribute__(noinline)` | gcc accepts the single paren | `Ubuntu clang`, `clang+DNN` and four Sanitizer lanes fail to compile | | `nullptr` in a `.c` file | gcc and clang accept it under `-std=c23` | `Windows MSVC+CUDA` → C2065 at every site; ADR-1138 keeps C TUs on `NULL` | @@ -43,15 +43,16 @@ for about three hours. | --- | --- | --- | | `gcc` | Ubuntu gcc(+DNN) | the baseline build and fast tests | | `clang` | Ubuntu clang(+DNN) | clang-only syntax; gcc is far more permissive about attributes and extensions | -| `m32` | Ubuntu i686 gcc | assumptions about the width of `size_t` / `ptrdiff_t` / pointers | | `msvcism` | Windows MSVC+CUDA / +SYCL | constructs MSVC rejects, checked statically so no MSVC is needed | | `sanitizers` | Sanitizers (address) / (undefined) | UB the plain build hides | | `tidy` | Tidy Changed | clang-tidy on the touched files, using the workflow's own exclusion list | | `cppcheck` | Cppcheck | cppcheck's findings | A stage whose toolchain is missing is **skipped with a notice**, not failed, so -the script is still useful on a partially provisioned machine. For the 32-bit -stage you want `gcc-multilib`. +the script is still useful on a partially provisioned machine. + +There is no 32-bit stage. The fork is 64-bit only (ADR-0728), and the +`m32` stage went with the i686 lane it mirrored (ADR-1258). ## Behaviours worth knowing @@ -67,20 +68,6 @@ the link. The repo's own `fuzz.yml` pairs the same two options. Without it the stage fails on every branch, including ones that change no code at all — and a stage that cries wolf is a stage people learn to ignore. -**`m32` needs a configured build.** It compiles `-fsyntax-only` with -`-I build/src` so the generated `config.h` resolves. Without that the compile -aborts at the first `#include` and the stage silently becomes a no-op — run the -`gcc` stage first, or just use `make preflight`, which orders them correctly. - -**`m32` sweeps only what the i686 lane builds.** It takes the changed files -that the `gcc` stage's CPU build compiles (its `compile_commands.json`), so GPU -sources and the per-backend GPU tests are left out, and it skips the `x86/`, -`arm64/` and `arm/` trees because that lane configures `-Denable_asm=false`. -Under `-m32` those files report missing intrinsics and headers, not width -assumptions. An error about a missing header (`No such file or directory`, or -clang's `file not found`) is ignored for the same reason. The `gcc` stage -reuses `build/` only when it holds a configured build (`build/build.ninja`). - ## What it does not cover `Windows MinGW64`, the Windows MSVC lanes proper, and `Ubuntu HIP` have no diff --git a/docs/metrics/features.md b/docs/metrics/features.md index d82cf49a0..85a7fe6d3 100644 --- a/docs/metrics/features.md +++ b/docs/metrics/features.md @@ -454,19 +454,13 @@ Netflix ADM. **Backends** — `adm`: AVX2, AVX-512, NEON, CUDA, SYCL, HIP, Metal. `float_adm`: AVX2, AVX-512, NEON, CUDA, SYCL, HIP, Metal. -**32-bit (i686) portability** — the integer ADM SSE2 path uses -`_mm_extract_epi64`, an intrinsic that is unavailable on 32-bit x86 toolchains. -Ports of upstream Netflix commits -[`8a289703`](https://github.com/Netflix/vmaf/commit/8a289703) and -[`1b6c3886`](https://github.com/Netflix/vmaf/commit/1b6c3886) add a portable -scalar fallback for the 64-bit lane extraction and lift the 32-bit gating -guard around the AVX/AVX-512 ADM dispatch table. Together they let the -integer ADM scalar path build and run on i686, which is what the dedicated -i686 CI lane introduced in -[ADR-0151](../adr/0151-i686-ci-netflix-1481.md) (T4-8) is there to exercise; without -the fallbacks the lane could only link, not actually score frames through -ADM. Functional behaviour on x86-64 and aarch64 is unchanged — the fallback -is selected at preprocess time only when `__x86_64__` is undefined. +**32-bit x86** — the fork is 64-bit only (ADR-0728, ADR-1258). The ADM x86 +sources still extract 64-bit lanes through 32-bit-safe helpers +(`extract_epi64()`, `extract_epi64_128()`), ported from upstream Netflix +commits [`8a289703`](https://github.com/Netflix/vmaf/commit/8a289703) and +[`1b6c3886`](https://github.com/Netflix/vmaf/commit/1b6c3886) and completed +in ADR-1258, so they compile for 32-bit x86. Nothing builds or tests 32-bit in +CI, so that is portability hygiene, not a supported configuration. **Reference** — Li S., Zhang F., Ma L., Ngan K., "Image Quality Assessment by Separately Evaluating Detail Losses and Additive Impairments," IEEE diff --git a/scripts/dev/preflight.sh b/scripts/dev/preflight.sh index 27365cfb2..d2f367ef2 100755 --- a/scripts/dev/preflight.sh +++ b/scripts/dev/preflight.sh @@ -11,7 +11,8 @@ # one PR at a time can be in flight: # # * a `static_assert` on `UINT_MAX <= SIZE_MAX/2/sizeof(ptr)` — true on LP64, -# FALSE on 32-bit, so `Ubuntu i686 gcc` would not compile the file; +# FALSE on 32-bit, so the i686 lane of the time would not compile the file +# (that lane is retired: the fork is 64-bit only, ADR-0728 / ADR-1258); # * `#define ALIGNED(x) __declspec(align((x)))` — MSVC needs a literal there, # so `Windows MSVC+CUDA` failed C2059 on every use; # * `__attribute__(noinline)` (a paren accidentally stripped) — gcc accepted @@ -124,7 +125,6 @@ stage mirrors CI context catches ----- ------------------ ------- gcc Ubuntu gcc(+DNN) the baseline build clang Ubuntu clang(+DNN) clang-only syntax, e.g. __attribute__(x) -m32 Ubuntu i686 gcc 32-bit-only static_assert / size assumptions msvcism Windows MSVC+CUDA / +SYCL constructs MSVC rejects (no MSVC needed) sanitizers Sanitizers (a/t/ub) UB and races the plain build hides tidy Tidy Changed clang-tidy on the files this branch touches @@ -175,61 +175,6 @@ if want clang; then fi fi -# ------------------------------------------------------------ 32-bit syntax -- -# A full 32-bit build needs multilib libraries; -fsyntax-only needs only the -# compiler, and that is enough for the failure mode this exists for: an -# assumption about the width of size_t / ptrdiff_t / pointers. -if want m32; then - say "32-bit syntax sweep" "mirrors: Ubuntu i686 gcc" - if ! echo 'int main(void){return 0;}' | gcc -m32 -x c - -o /tmp/preflight-m32probe 2>/dev/null; then - skipped m32 "no 32-bit gcc support (install gcc-multilib)" - elif [ ! -f build/src/config.h ]; then - skipped m32 "build/src/config.h missing — run the gcc stage first" - else - m32_fail=0 - # Sweep only what a CPU build compiles: the gcc stage's compile database - # lists it. That leaves out GPU sources and the per-backend GPU tests, - # which only build with a backend enabled. - cpu_units=$(python3 -c 'import json,os,sys -for e in json.load(open("build/compile_commands.json")): - print(os.path.relpath(os.path.join(e["directory"], e["file"])))' 2>/dev/null) - while IFS= read -r f; do - [ -f "$f" ] || continue - case "$f" in *.h | *.hpp) continue ;; esac - printf '%s\n' "$cpu_units" | grep -Fxq "$f" || continue - # The i686 lane also configures -Denable_asm=false, so it never compiles - # the ISA-specific trees; under -m32 they report intrinsics that do not - # exist there, not width assumptions. - case "$f" in */x86/* | */arm64/* | */arm/*) continue ;; esac - if [ "${f%.cpp}" != "$f" ]; then - std=(-std=c++23 -x c++) - else - std=(-std=c2x -x c) - fi - # -I build/src supplies the generated config.h. Without it the compile - # aborts at the first #include and never reaches the code under test, - # which silently turned this stage into a no-op. - gcc -m32 -fsyntax-only "${std[@]}" -D_GNU_SOURCE \ - -I core/include -I core/src -I core/src/feature -I core/tools -I core/test \ - -I build/src -I build \ - "$f" 2>/tmp/preflight-m32.err - # Missing generated headers (config.h) and absent intrinsics are expected - # outside a configured build; width assumptions are not. - if grep -qE 'error:' /tmp/preflight-m32.err && - ! grep -qE "config\.h|file not found|No such file or directory|Datei oder Verzeichnis" /tmp/preflight-m32.err; then - printf ' %s\n' "$f" - grep -m2 -E 'error:' /tmp/preflight-m32.err | sed 's/^/ /' - m32_fail=1 - fi - done < <(changed_sources) - if [ "$m32_fail" -eq 0 ]; then - ok m32 - else - bad m32 - fi - fi -fi - # --------------------------------------------------------------- MSVC-isms -- # No MSVC on Linux, but its rejections are a small, well-known set and they are # greppable. Each pattern below cost a real CI round-trip at least once. From 6d88f9a215cbdb434e38de1749197ad1ae10344b Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 21:56:19 +0200 Subject: [PATCH 05/15] docs: record the retired i686 lane and the x86-64-only intrinsic fix State rows T-CI-I686-LANE-RESURRECTED-2026-09-18 and T-X86-64-ONLY-INTRINSICS-2026-09-18 (closed), changelog fragments, a rebase note (no i686 row; keep extract_epi64_128() and the _mm_storel_epi64 read) and an x86 AGENTS invariant. --- CHANGELOG.md | 14 ++++++++++++++ changelog.d/fixed/x86-64-only-intrinsics.md | 5 +++++ changelog.d/removed/ci-i686-lane-retired-again.md | 5 +++++ core/src/feature/x86/AGENTS.md | 7 +++++++ docs/rebase-notes.md | 12 ++++++++++++ docs/state.md | 2 ++ 6 files changed, 45 insertions(+) create mode 100644 changelog.d/fixed/x86-64-only-intrinsics.md create mode 100644 changelog.d/removed/ci-i686-lane-retired-again.md diff --git a/CHANGELOG.md b/CHANGELOG.md index daa68995b..3fcaf36dc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16523,6 +16523,13 @@ No build/test/runtime impact — the `enable_vulkan` meson option was already re `integer_cambi_hip.c` updated to reflect the removal. +- **CI: the 32-bit x86 (`Ubuntu i686 gcc`) lane is removed again.** ADR-0728 + retired 32-bit x86 in May, but a merge the same day brought the lane back, + and it kept running compile-only with no tests. The fork stays 64-bit only + (ADR-1258). `scripts/dev/preflight.sh` drops its `m32` stage, which only + mirrored that lane. + + - **core: delete dead C translation units `core/src/gpu_picture_pool.c` and `core/src/opt.c`** — both translation units were superseded by modern C++23 implementations (`core/src/gpu_picture_pool.cpp` under ADR-0768, `core/src/opt.cpp` under ADR-0761) compiled into `libvmaf`. Their lingering inclusion in isolated test targets (`test_integer_ssim_simd` and `test_motion_avx512_parity`) in `core/test/meson.build` was updated to link against the standard C++23 libraries and objects (`gpu_picture_pool.cpp`, `log_cpp23_test_objects`, `wave8_opt_only_objects`), collapsing the twin pairs from 6 to 4 in `twin-drift-check.sh`. - **test: rescue orphaned `core/test/test_gpu_picture_pool_partial_init.c`** — wired into `core/test/meson.build` under the fast test suite, testing `vmaf_gpu_picture_pool_init` error-unwind paths against `gpu_picture_pool.cpp`. - **docs: fix stale pre-rename `libvmaf/` and `python/vmaf/` paths** — updated path references across `core/tools/meson.build`, `core/tools/compat/win32/getopt.{c,h}`, `core/tools/vmaf_roi_core.h`, `testdata/bench_all.sh`, and `docs/usage/{bd-rate,matlab,python}.md` to point to `core/` and `compat/python-vmaf/` (ADR-0700). @@ -29208,6 +29215,13 @@ Restores the VK-1 + VK-2 perf fix originally landed in PR #879. wrappers are global symbols. +- **The AVX2 and AVX-512 sources no longer use x86-64-only intrinsics.** + `_mm_extract_epi64` in the ADM kernels and `_mm_cvtsi128_si64` in the PSNR + kernel were the cause of Netflix#1481 (no 32-bit x86 build with asm). They + now go through 32-bit-safe forms. Nothing changes on x86-64, and 32-bit x86 + remains unsupported. + + - **`y4m_convert_411_422jpeg` 1-byte heap-buffer-overflow on 4:1:1 streams whose destination chroma row reduces to a single pixel (`dst_c_w == 1`).** The Daala-derived 4:1:1 → 4:2:2-jpeg diff --git a/changelog.d/fixed/x86-64-only-intrinsics.md b/changelog.d/fixed/x86-64-only-intrinsics.md new file mode 100644 index 000000000..006f3b1fe --- /dev/null +++ b/changelog.d/fixed/x86-64-only-intrinsics.md @@ -0,0 +1,5 @@ +- **The AVX2 and AVX-512 sources no longer use x86-64-only intrinsics.** + `_mm_extract_epi64` in the ADM kernels and `_mm_cvtsi128_si64` in the PSNR + kernel were the cause of Netflix#1481 (no 32-bit x86 build with asm). They + now go through 32-bit-safe forms. Nothing changes on x86-64, and 32-bit x86 + remains unsupported. diff --git a/changelog.d/removed/ci-i686-lane-retired-again.md b/changelog.d/removed/ci-i686-lane-retired-again.md new file mode 100644 index 000000000..7323fc149 --- /dev/null +++ b/changelog.d/removed/ci-i686-lane-retired-again.md @@ -0,0 +1,5 @@ +- **CI: the 32-bit x86 (`Ubuntu i686 gcc`) lane is removed again.** ADR-0728 + retired 32-bit x86 in May, but a merge the same day brought the lane back, + and it kept running compile-only with no tests. The fork stays 64-bit only + (ADR-1258). `scripts/dev/preflight.sh` drops its `m32` stage, which only + mirrored that lane. diff --git a/core/src/feature/x86/AGENTS.md b/core/src/feature/x86/AGENTS.md index 95a8eb077..b04586d5d 100644 --- a/core/src/feature/x86/AGENTS.md +++ b/core/src/feature/x86/AGENTS.md @@ -148,6 +148,13 @@ Skill scaffolds: verbatim — they lose bit-exact contract that ADR-0138 / 0139 / 0142 / 0143 froze. +- **No x86-64-only intrinsics.** `_mm_extract_epi64`, + `_mm256_extract_epi64`, `_mm_cvtsi128_si64` and the like go through + `extract_epi64_128()` / `extract_epi64()` (ADM) or a store + (`_mm_storel_epi64`). The fork is 64-bit only (ADR-1258), but these files + stay 32-bit clean because they are upstream-mirror code and that is + Netflix#1481's fix (T-X86-64-ONLY-INTRINSICS-2026-09-18). + ## Governing ADRs See [../AGENTS.md §Governing ADRs](../AGENTS.md) for full list. diff --git a/docs/rebase-notes.md b/docs/rebase-notes.md index cc6437d38..a76bb35f3 100644 --- a/docs/rebase-notes.md +++ b/docs/rebase-notes.md @@ -29,6 +29,18 @@ mask row") is ported, with three differences a sync must keep: upstream counterpart. The same branch refactors `calculate_c_values_row_neon` into a per-pixel helper (touched-file lint, bit-exact under `test_cambi_simd`). See [Research-2062](research/2062-cambi-spatial-mask-simd.md). +## ci/retire-i686-lane — the fork stays 64-bit only; x86 SIMD sources use no x86-64-only intrinsics (2026-09-18) + +- `.github/workflows/libvmaf-build-matrix.yml`: there is no i686 row + (ADR-0728, ADR-1258). Upstream Netflix/vmaf has its own 32-bit cross build + (`f6d6dde1`); do not port it. A merge that restores `i686: true` rows is + wrong: that is how the lane came back in `384d97d03`. +- `core/src/feature/x86/adm_avx2.c`, `adm_avx512.c`: 64-bit lane extraction + from an `__m128i` goes through `extract_epi64_128()`, defined beside + `extract_epi64()`. Upstream calls `_mm_extract_epi64` directly; keep the fork + form. +- `core/src/feature/x86/psnr_avx2.c`, `psnr_sse_line_16_avx2()`: the final + 64-bit sum is read with `_mm_storel_epi64`, not `_mm_cvtsi128_si64`. ## Canonical envtest installer (2026-09-08) diff --git a/docs/state.md b/docs/state.md index e89efaf17..02024a920 100644 --- a/docs/state.md +++ b/docs/state.md @@ -457,6 +457,8 @@ landed fix yet._ | **T-CI-FORMATTER-PINS-DRIFT-2026-09-19** | **`make lint-tools` installed an older ruff than the pre-commit hook ran.** The `Makefile` states that its `RUFF_VERSION` / `BLACK_VERSION` are identical to `.pre-commit-config.yaml`, but nothing enforced it and Renovate only managed the hook side: the hook reached ruff 0.16.8 while the `Makefile` installed 0.16.5 and one install hint still named 0.15.17, so the local gate and the hook could disagree about what counts as a violation. **FIXED**: pins aligned, install hints use the variables, `scripts/ci/check-workflow-versions.py` gained `check_formatter_pins` (fails on a version mismatch or a literal `ruff==` / `black==` in a recipe; fixture `scripts/ci/tests/test_formatter_pins_single_source.py`), and `renovate.json` gained two regex managers plus a rule that groups the `Makefile` pins with the pre-commit hooks. The same PR holds `onnx` below 1.23 in Renovate (see T-AI-ONNX-IR14-UNLOADABLE-2026-09-18) and adds `.devcontainer/**` to `ignorePaths`, because a digest bump edited the generated `Dockerfile.praetor` and the governance audit then reported the devcontainer as out of sync. | — | `integration/deps-20260919` | 2026-09-19 | closed | | **T-AI-ONNX-IR14-UNLOADABLE-2026-09-18** | **The Tiny AI test job failed on master once `onnx` 1.23.0 was released.** `ai/pyproject.toml` allowed `onnx>=1.22.0,<2.0`, so CI resolved 1.23.0, whose `helper.make_model()` writes IR version 14 by default. The pinned `onnxruntime` 1.30.0 loads at most IR 13, so every test that builds a model with onnx and runs it in ORT failed with `Unsupported model IR version: 14, max supported IR version: 13` (21 tests across `test_cross_backend`, `test_learned_filter_audit`, `test_profile`, `test_bisect_model_quality` and the PTQ round trip). The same mismatch would make any model built in-repo with onnx 1.23 unloadable by the runtime the fork ships. Reproduced in isolation: the same Relu model is IR 14 and rejected with onnx 1.23.0, IR 13 and accepted with 1.22.0. **FIXED**: `onnx>=1.22.0,<1.23` in `ai/pyproject.toml` and `tools/ensemble-training-kit/pyproject.toml`, with a comment tying the cap to onnxruntime's IR limit. | — | `fix/onnx-ir-cap` | 2026-09-18 | closed | | **T-NO-ASM-SIMD-TEST-WARNINGS-2026-09-18** | **A `-Denable_asm=false` build emitted ~30 `-Wunused-function` / `-Wunused-const-variable` / `-Wunused-variable` warnings in SIMD parity-test files.** Ten files under `core/test/` (`test_vif_simd.c`, `test_ssimulacra2_simd.c`, `test_speed_simd.c`, `test_psnr_hvs_simd.c`, `test_ms_ssim_decimate.c`, `test_motion_v2_simd.c`, `test_iqa_convolve.c`, `test_integer_ssim_simd.c`, `test_cambi_simd.c`, `test_cambi.c`) defined scalar-reference helpers, fixture builders, and lookup tables/variables unconditionally while every caller sat under an ISA guard (`ARCH_X86`, `HAVE_AVX512`, `ARCH_AARCH64`, or the `ARCH_X86 \|\| ARCH_AARCH64` union `run_tests()` already uses). Measured on an i686 build; reproduces on x86-64 with asm off too. **FIXED**: each helper moved under the exact union of ISA conditions its callers use, walking the full dependency chain so the warning does not relocate to a `pick_*` dispatcher or `ref_*` scalar reference one level down (`test_ssimulacra2_simd.c` needed one guard spanning its whole pick/ref/test helper section for this reason). Nothing deleted, no `(void)`-cast or `[[maybe_unused]]` suppression. Verified zero warnings and an identical test count on `-Denable_asm=false` x86-64, default x86-64, and an aarch64 cross build under `qemu-aarch64-static`. The aarch64 build's two `vif_neon.c` `-Wunused-but-set-variable` warnings (dead `i_dst_stride` counters in the VIF statistic kernels) are fixed on the same branch, and `vif_neon.c` is split into helpers that clear its 36 clang-tidy findings with NEON output byte-identical under qemu (68 of 68 runs). | none | `fix/no-asm-test-warnings` | 2026-09-18 | closed | +| **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for four months after ADR-0728 removed it.** ADR-0728 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The other ADR-0728 removals the same merge restored are audited separately. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0728](adr/0728-native-build-sunset.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | +| **T-X86-64-ONLY-INTRINSICS-2026-09-18** | **The x86 SIMD sources did not compile for 32-bit x86 with asm enabled** (the cause of [Netflix#1481](https://github.com/Netflix/vmaf/issues/1481)). `adm_avx2.c` and `adm_avx512.c` called `_mm_extract_epi64` on 12 lines each, beside an `extract_epi64()` fallback that only covered the 256-bit form, and `psnr_avx2.c` called `_mm_cvtsi128_si64`. GCC declares both only on x86-64. **FIXED**: `extract_epi64_128()` (`_mm_extract_epi64` on x86-64, a 32-bit fallback otherwise) and `_mm_storel_epi64`. An i686 build with asm then compiles warning-free. x86-64 code is unchanged (fast suite 137/137; clang-tidy and cppcheck clean). 32-bit stays unsupported (ADR-0728, ADR-1258), so this is portability hygiene. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-PREFLIGHT-M32-FALSE-FAILURES-2026-09-18** | **`scripts/dev/preflight.sh`'s 32-bit sweep failed on code the i686 lane never compiles.** It swept every changed C file with `gcc -m32`, including `arm64/`, `x86/` SIMD and GPU trees, which the `Ubuntu i686 gcc` lane skips because it configures `-Denable_asm=false` and no GPU backend. Its missing-header filter also knew only clang's `file not found` and the German gcc text, while the script runs under `LC_ALL=C`, so gcc's `No such file or directory` (e.g. `arm_neon.h`) counted as a failure. Seen on the `fix/adm-cm-simd-bitexact` branch. **FIXED**: the stage skips those trees and accepts the English message. The underlying i686+asm gap stays as Netflix#1481 / ADR-0151 describe it. | none | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | | **T-EXPORTED-SYMBOLS-ASAN-GNU-LD-2026-09-18** | **`check_exported_symbols` failed in a GNU-ld ASan build.** GNU ld exports the linker-defined `__start_asan_globals` / `__stop_asan_globals` bounds of ASan's metadata section from `libvmaf.so`; lld, which the CI sanitizer lane uses, hides them. They are sanitizer runtime artefacts, not API. **FIXED**: the checker treats `__start_` / `__stop_` bounds of the ASan, HWASan and SanitizerCoverage sections as runtime-owned; any other section bound still fails. Also cleared the file's three ruff findings. | [ADR-0379](adr/0379-libvmaf-symbol-visibility.md) | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | | **T-CUDA-EXTERN-C-CHECK-VACUOUS-2026-09-18** | **`scripts/dev/check-cuda-extern-c.sh` (ADR-0747) never checked a kernel.** Its name regex expected `cuModuleGetFunction(&fn, "name")`, but every call in the tree passes the module first (`&fn, module, "name"`), so it collected no names; with none, `${#KERNEL_NAMES[@]}` on an empty associative array under `set -u` aborted the script with exit 1. The verification recorded in T-CUDA-EXTERN-C-SWEEP-0747-2026-05-28 (`exits 0`) could therefore not have held. Found by the ADR-1142 cleanup of `integer_adm_cuda.c`. **FIXED**: the check is rewritten; it takes the kernel name from every `cuModuleGetFunction` call including split ones, blanks comments and strings before counting braces, and lists by name the looked-up kernels it cannot locate as a literal `__global__` definition (23 macro-generated ones) instead of passing them silently. It now reports 48 of 71 located, 0 unwrapped, and fails with the kernel names when an `extern "C"` block is removed. | [ADR-0747](adr/0747-cuda-extern-c-sweep.md) | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | From ff1d79d582236243c1f20d7ad7bfe27716db5916 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 21:56:21 +0200 Subject: [PATCH 06/15] chore(adr): drop the claim stubs committed next to ADR-1223 and ADR-1224 #1384 committed both .md.stub reservation files alongside the finished ADRs. A stub is a working-tree marker that the real ADR replaces (ADR-0535); these two only kept numbers 1223 and 1224 looking claimed twice. --- ...223-cuda-ampere-architecture-floor.md.stub | 36 ------------------- docs/adr/1224-cuda-tile-not-adopted.md.stub | 36 ------------------- 2 files changed, 72 deletions(-) delete mode 100644 docs/adr/1223-cuda-ampere-architecture-floor.md.stub delete mode 100644 docs/adr/1224-cuda-tile-not-adopted.md.stub diff --git a/docs/adr/1223-cuda-ampere-architecture-floor.md.stub b/docs/adr/1223-cuda-ampere-architecture-floor.md.stub deleted file mode 100644 index f466be78a..000000000 --- a/docs/adr/1223-cuda-ampere-architecture-floor.md.stub +++ /dev/null @@ -1,36 +0,0 @@ - - - - -# ADR-1223: - -- **Status**: Proposed -- **Date**: 2026-09-07 -- **Deciders**: -- **Tags**: - -## Context - - - -## Decision - - - -## Alternatives considered - -| Option | Pros | Cons | Why not chosen | -|---|---|---|---| -| | | | | - -## Consequences - -- **Positive**: -- **Negative**: -- **Neutral / follow-ups**: - -## References - -- See [ADR-0535](0535-adr-atomic-allocator.md) for the original allocator design. -- See [ADR-0628](0628-adr-allocator-remote-aware.md) for the remote-aware extension. -- Source: .> diff --git a/docs/adr/1224-cuda-tile-not-adopted.md.stub b/docs/adr/1224-cuda-tile-not-adopted.md.stub deleted file mode 100644 index 9c9e43cf5..000000000 --- a/docs/adr/1224-cuda-tile-not-adopted.md.stub +++ /dev/null @@ -1,36 +0,0 @@ - - - - -# ADR-1224: - -- **Status**: Proposed -- **Date**: 2026-09-07 -- **Deciders**: -- **Tags**: - -## Context - - - -## Decision - - - -## Alternatives considered - -| Option | Pros | Cons | Why not chosen | -|---|---|---|---| -| | | | | - -## Consequences - -- **Positive**: -- **Negative**: -- **Neutral / follow-ups**: - -## References - -- See [ADR-0535](0535-adr-atomic-allocator.md) for the original allocator design. -- See [ADR-0628](0628-adr-allocator-remote-aware.md) for the remote-aware extension. -- Source: .> From dc5a18abc87f7b9cff44ef5e09eb51e32a43cf90 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:10:22 +0200 Subject: [PATCH 07/15] docs(adr): attribute the i686 lane removal to ADR-0691, not ADR-0728 9aa008e70 (#1564), the commit that removed the i686 lane, implemented ADR-0691. ADR-0728 (bfd4c436b, #52) landed after the rename merge had already restored the lane and changed only its own ADR, a changelog fragment and deprecations.md. ADR-1258, its index row, the state row, the changelog fragment and the rebase note now say so, and the preflight page, preflight.sh and the ADM page cite ADR-1258 for the 64-bit-only rule. ADR-0151 is marked superseded by ADR-1258 instead of ADR-0728: ADR-0691's removal was undone the same day, so the lane ADR-0151 added ran until ADR-1258 removed it, and ADR-1258 is the record that is in force. --- CHANGELOG.md | 2 +- .../removed/ci-i686-lane-retired-again.md | 2 +- docs/adr/0151-i686-ci-netflix-1481.md | 2 +- .../1258-keep-64-bit-only-retire-i686-lane.md | 64 ++++++++++--------- docs/adr/README.md | 3 +- .../0151-i686-ci-netflix-1481.md | 2 +- .../1258-keep-64-bit-only-retire-i686-lane.md | 2 +- docs/development/preflight.md | 6 +- docs/metrics/features.md | 2 +- docs/rebase-notes.md | 2 +- docs/state.md | 1 + scripts/dev/preflight.sh | 2 +- 12 files changed, 49 insertions(+), 41 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3fcaf36dc..56acb7476 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16523,7 +16523,7 @@ No build/test/runtime impact — the `enable_vulkan` meson option was already re `integer_cambi_hip.c` updated to reflect the removal. -- **CI: the 32-bit x86 (`Ubuntu i686 gcc`) lane is removed again.** ADR-0728 +- **CI: the 32-bit x86 (`Ubuntu i686 gcc`) lane is removed again.** ADR-0691 retired 32-bit x86 in May, but a merge the same day brought the lane back, and it kept running compile-only with no tests. The fork stays 64-bit only (ADR-1258). `scripts/dev/preflight.sh` drops its `m32` stage, which only diff --git a/changelog.d/removed/ci-i686-lane-retired-again.md b/changelog.d/removed/ci-i686-lane-retired-again.md index 7323fc149..d88d9dc37 100644 --- a/changelog.d/removed/ci-i686-lane-retired-again.md +++ b/changelog.d/removed/ci-i686-lane-retired-again.md @@ -1,4 +1,4 @@ -- **CI: the 32-bit x86 (`Ubuntu i686 gcc`) lane is removed again.** ADR-0728 +- **CI: the 32-bit x86 (`Ubuntu i686 gcc`) lane is removed again.** ADR-0691 retired 32-bit x86 in May, but a merge the same day brought the lane back, and it kept running compile-only with no tests. The fork stays 64-bit only (ADR-1258). `scripts/dev/preflight.sh` drops its `m32` stage, which only diff --git a/docs/adr/0151-i686-ci-netflix-1481.md b/docs/adr/0151-i686-ci-netflix-1481.md index b9e6dd1f7..0401e42b4 100644 --- a/docs/adr/0151-i686-ci-netflix-1481.md +++ b/docs/adr/0151-i686-ci-netflix-1481.md @@ -1,6 +1,6 @@ # ADR-0151: i686 build-only CI job — reproduce Netflix #1481 -- **Status**: Superseded by [ADR-0728](0728-native-build-sunset.md) +- **Status**: Superseded by [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) - **Date**: 2026-04-24 - **Deciders**: Lusoris - **Tags**: ci, build, x86, netflix-upstream diff --git a/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md b/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md index 5060765c2..98b916a50 100644 --- a/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md +++ b/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md @@ -1,23 +1,25 @@ # ADR-1258: Keep the fork 64-bit only; retire the resurrected i686 lane -- **Status**: Accepted +- **Status**: Accepted, Supersedes [ADR-0151](0151-i686-ci-netflix-1481.md) - **Date**: 2026-09-18 - **Deciders**: Lusoris - **Tags**: build, ci, x86, netflix-upstream, fork-local ## Context -ADR-0728 (Accepted, 2026-05-28) removed the `Build — Ubuntu i686 gcc (CPU, -no-asm)` lane from `.github/workflows/libvmaf-build-matrix.yml` (commit -`9aa008e70`, #1564), and `docs/development/deprecations.md` records that -"32-bit x86 is unsupported; the fork is 64-bit only". The same day, -`384d97d03`, the squash-merge of the `libvmaf/` → `core/` rename that -"merge-resolves 42 master commits", brought the row back. No ADR asked for -it. It has run on every PR since, compile-only with `-Denable_asm=false` as -ADR-0151 had set it up, and was later renamed `Ubuntu i686 gcc`. ADR-1234 -then built preflight's `m32` stage on the assumption that the lane was -deliberate. +ADR-0691 (Accepted, 2026-05-28) decided the fork is 64-bit only and removed +the `Build — Ubuntu i686 gcc (CPU, no-asm)` lane from +`.github/workflows/libvmaf-build-matrix.yml` (commit `9aa008e70`, #1564). The +same day, `384d97d03`, the squash-merge of the `libvmaf/` → `core/` rename +that "merge-resolves 42 master commits", brought the row back. No ADR asked +for it. ADR-0728 (`bfd4c436b`, #52) landed after that merge and restated the +removal in `docs/development/deprecations.md` ("32-bit x86 is unsupported; the +fork is 64-bit only"), but it changed no workflow, so the restored row went +unnoticed. It has run on every PR since, compile-only with +`-Denable_asm=false` as ADR-0151 had set it up, and was later renamed +`Ubuntu i686 gcc`. ADR-1234 then built preflight's `m32` stage on the +assumption that the lane was deliberate. Checking an i686 build with asm enabled on 2026-09-18 showed what that lane was and was not covering: @@ -35,14 +37,16 @@ was and was not covering: So the lane looked like 32-bit coverage while testing none of it. The maintainer was first asked to make 32-bit a tested platform with an SSE2 -floor, and chose that. When ADR-0728's decision and the accidental -resurrection were then put in front of them, they chose to keep the fork -64-bit only (see References). +floor, and chose that. When the recorded 64-bit-only decision and the +accidental resurrection were then put in front of them, they chose to keep +the fork 64-bit only (see References). The popup named that decision +ADR-0728, because the deprecation notice cites it; the decision is +ADR-0691's. ## Decision -We keep ADR-0728's decision that the fork is 64-bit only, and remove what -contradicts it: +We keep ADR-0691's decision that the fork is 64-bit only, which ADR-0728 +restated, and remove what contradicts it: - The `Ubuntu i686 gcc` matrix row, its dependency step and its `matrix.i686` conditions are removed from `libvmaf-build-matrix.yml`. @@ -58,37 +62,39 @@ contradicts it: - `build-aux/i686-linux-gnu.ini` stays, unreferenced by CI, because historical documents link to it. -ADR-0151's status becomes Superseded by ADR-0728, which retired its lane -without recording that. +ADR-0151's status becomes Superseded by ADR-1258. ADR-0691 decided to drop +its lane first, without marking it superseded, but that removal was undone +the same day; the lane ADR-0151 set up ran until this ADR removed it. ## Alternatives considered | Option | Pros | Cons | Why not chosen | | --- | --- | --- | --- | -| Keep 64-bit only; retire the lane and `m32` (chosen) | Matches ADR-0728, the container-first posture (ADR-0686) and what is shipped; no CI or preflight time on an unsupported target | 32-bit portability regressions go unnoticed | — | -| Support 32-bit again: SSE2 floor, asm on, tests in CI | Real 32-bit coverage; scores match x86-64 | Reverses ADR-0728 for a target the project ships nothing for; one more lane to keep green | The maintainer chose 64-bit only once ADR-0728 was on the table | -| Keep the resurrected lane as it was (compile-only, no asm) | No change | Keeps an accidental lane that implies support it does not test, and contradicts ADR-0728 | Coverage that tests nothing is misleading | +| Keep 64-bit only; retire the lane and `m32` (chosen) | Matches ADR-0691, the container-first posture (ADR-0686) and what is shipped; no CI or preflight time on an unsupported target | 32-bit portability regressions go unnoticed | — | +| Support 32-bit again: SSE2 floor, asm on, tests in CI | Real 32-bit coverage; scores match x86-64 | Reverses ADR-0691 for a target the project ships nothing for; one more lane to keep green | The maintainer chose 64-bit only once the recorded decision was on the table | +| Keep the resurrected lane as it was (compile-only, no asm) | No change | Keeps an accidental lane that implies support it does not test, and contradicts ADR-0691 | Coverage that tests nothing is misleading | | Remove the lane but keep `m32` | Still catches 32-bit width assumptions | Enforces a portability property the project does not promise | Same reason as the lane | ## Consequences -- **Positive**: CI, preflight and the docs agree with ADR-0728 again. The +- **Positive**: CI, preflight and the docs agree with ADR-0691 again. The x86 SIMD sources no longer carry Netflix#1481's cause. - **Negative**: nothing notices if a change breaks 32-bit compilation or 32-bit scores. Anyone building 32-bit x86 themselves is on their own, including x87 score drift unless they build with `-msse2 -mfpmath=sse`. -- **Neutral / follow-ups**: the other ADR-0728 removals that the same merge - resurrected are audited separately; their records are corrected where a - later ADR re-adopted them. +- **Neutral / follow-ups**: the other lanes ADR-0691 and ADR-0689 removed, + which the same merge restored, and the lane removals ADR-0728 describes but + never made, are recorded in ADR-1259. ## References -- [ADR-0728](0728-native-build-sunset.md) (upheld), - [ADR-0151](0151-i686-ci-netflix-1481.md) (superseded by ADR-0728), +- [ADR-0691](0691-vmafx-drop-legacy-build-paths.md) (64-bit-only decision + upheld), [ADR-0728](0728-native-build-sunset.md) (restated it in docs only), + [ADR-0151](0151-i686-ci-netflix-1481.md) (superseded by this ADR), [ADR-1234](1234-local-preflight-gate.md) (stage list amended), [ADR-0686](0686-vmafx-rebrand-aggressive-modernization.md). -- Commits `9aa008e70` (#1564, lane removed) and `384d97d03` (layout rename - merge, lane resurrected). +- Commits `9aa008e70` (#1564, ADR-0691, lane removed), `384d97d03` (layout + rename merge, lane resurrected) and `bfd4c436b` (#52, ADR-0728, docs only). - Netflix/vmaf issue [#1481](https://github.com/Netflix/vmaf/issues/1481). - Popup, 2026-09-18, first round, question 1: "Require SSE2 everywhere (Recommended)"; question 2: "Default options + run tests (Recommended)". diff --git a/docs/adr/README.md b/docs/adr/README.md index 9ee45be7f..e9fb98714 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -183,7 +183,7 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-0148](0148-iqa-rename-and-cleanup.md) | Rename the IQA-derived reserved-identifier surface (`_iqa_*`, `struct _kernel`, `_ssim_int`, `_map_reduce`, `_map`, `_reduce`, `_context`, `_ms_ssim_map`, `_ssim_map`, `_ms_ssim_reduce`, `_ssim_reduce`, `_alloc_buffers`, `_free_buffers`, header guards `_CONVOLVE_H_` / `_DECIMATE_H_` / `_SSIM_TOOLS_H_` / `__VMAF_MS_SSIM_DECIMATE_H__`) to non-reserved `iqa_*` / `ms_ssim_*` / `_INCLUDED` spellings across `core/src/feature/{iqa,*}` (21 files). Sweeps the ADR-0141 touched-file lint cascade that surfaced (~40 pre-existing warnings: misc-use-internal-linkage → `static` or cross-TU NOLINT; widening multiplications → size_t casts; multi-decl splits; function-size refactors of `calc_ssim` / `compute_ssim` / `compute_ms_ssim` / `run_gauss_tests`; unused-parameter `(void)` casts; scoped NOLINTBEGIN/END for analyzer false positives on kernel-offset bounds and test-helper malloc leaks). Bit-identical VMAF score on Netflix golden pair (scalar vs SIMD, with `--feature float_ssim --feature float_ms_ssim`). Closes backlog item T7-6. | Accepted | lint, cleanup, refactor, iqa, touched-file-rule | | [ADR-0149](0149-port-netflix-1376-fifo-semaphore.md) | Port Netflix upstream PR #1376 ("Fix fifo hangs") into the Python harness under `python/vmaf/core/executor.py` + `python/vmaf/core/raw_extractor.py`. Replaces the 1-second `os.path.exists()` polling loop in `_open_{work,proc}files_in_fifo_mode` with `multiprocessing.Semaphore(0)` signalled by the child processes after `os.mkfifo(...)`; parent acquires with 5-second soft-timeout warn then blocks indefinitely. Applied to both the base `Executor` class hierarchy and the `ExternalVmafExecutor`-style subclass (single-process variant). Fork carve-outs: skip upstream's `__version__ = "3.0.0" → "4.0.0"` bump (fork tracks its own versioning per ADR-0025); drop now-unused `from time import sleep` from both files (ADR-0141 ruff F401). Closes backlog item T4-7. | Accepted | upstream-port, python, concurrency, fifo | | [ADR-0150](0150-port-netflix-1472-cuda-windows.md) | Port Netflix upstream PR #1472 ("cuda: enable CUDA feature extraction on Windows (MSYS2/MinGW)") two-commit series: source-portability guards in CUDA headers + `.cu` files (drop `` from `cuda/common.h`, DEVICE_CODE guards on `` vs `` in `cuda_helper.cuh` + `picture.h`, `#ifndef DEVICE_CODE` around `feature_collector.h` in 5 ADM `.cu` files), and meson build plumbing (`vswhere`-based `cl.exe` discovery without `PATH` pollution, Windows SDK + MSVC include path injection via `-I` flags to nvcc, CUDA version detection via `nvcc --version` instead of `meson.get_compiler('cuda')`). Fork-specific conflict resolutions: keep positional (not `#ifndef __CUDACC__`) initializers in `integer_adm.h`; keep `pthread_dependency` on `cuda_static_lib` (ring_buffer.c still uses pthread directly); merge fork's gencode coverage block (ADR-0122) with upstream's new nvcc-detect block. Drive-by: rename reserved `__VMAF_SRC_*_H__` header guards to `VMAF_SRC_*_INCLUDED`. Linux CPU build 32/32 + Linux CUDA build 35/35 pass. Closes backlog item T4-2. | Accepted | upstream-port, cuda, windows, mingw, build | -| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Superseded by [ADR-0728](0728-native-build-sunset.md) | ci, build, x86, netflix-upstream | +| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Superseded by [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | ci, build, x86, netflix-upstream | | [ADR-0152](0152-vmaf-read-pictures-monotonic-index.md) | `vmaf_read_pictures` now rejects non-monotonic indices with `-EINVAL`. Addresses Netflix upstream issue #910: `integer_motion` / motion2 / motion3 extractors keep sliding-window state keyed by `index % N`, so submitting frames out of order or with duplicate indices silently corrupts their ring-buffers (symptom: missing `integer_motion2_score` on last frame when submission order doesn't match frame order). Enforced via new `last_index` + `have_last_index` fields on `VmafContext`, checked inside the existing `read_pictures_validate_and_prep` helper from ADR-0146. Net visible behaviour change: duplicates and out-of-order indices now return `-EINVAL` instead of producing silent-wrong-answer. 3-subtest reducer in `test_read_pictures_monotonic.c` verified to fail pre-fix and pass post-fix. Zero impact on in-tree callers (vmaf CLI + test suite already iterate strictly increasing). Closes backlog item T1-2. | Accepted | api, correctness, motion, netflix-upstream | | [ADR-0153](0153-float-ms-ssim-min-dim-netflix-1414.md) | `float_ms_ssim` init rejects input below 176×176 with `-EINVAL` + clear error message. Addresses Netflix upstream issue #1414: the 5-level 11-tap MS-SSIM pyramid walks off the kernel footprint at a mid-level scale for inputs below 176×176 (QCIF and smaller), producing a confusing mid-run `error: scale below 1x1!` cascade. Fix checks `w < GAUSSIAN_LEN << (SCALES - 1)` at init time in `float_ms_ssim.c:init`, derived from the existing filter constants so it stays in sync. Extracted `ms_ssim_init_simd_dispatch` helper to keep the function under the ADR-0141 size budget. 3-subtest reducer in `test_float_ms_ssim_min_dim.c` covers 5 boundary rejections (including exact-under cases like 175×176 and 176×175) and 2 accept cases (176×176 exact + 576×324). Verified fail-without-fix + pass-with-fix. Visible behaviour: init now fails immediately instead of mid-stream; zero impact on inputs ≥176×176. Closes backlog item T1-4. | Accepted | correctness, ms-ssim, netflix-upstream | | [ADR-0154](0154-score-pooled-eagain-netflix-755.md) | `vmaf_score_pooled` (via `vmaf_feature_collector_get_score` + the inline `vmaf_feature_vector_get_score` fast-path) now returns `-EAGAIN` when a requested feature index is valid but not yet written (transient — e.g. motion2 waiting for the next frame or flush), distinguishing it from `-EINVAL` which remains for programmer errors (bad pointer, out-of-range, unknown feature). Addresses Netflix upstream issue #755: downstream integrations that want per-frame streaming VMAF output can now distinguish 'retry after next read or flush' from 'abort'. Inline-helper return was previously `-1`; now `-EINVAL` (structural) or `-EAGAIN` (pending). Drive-by: rename reserved `__VMAF_FEATURE_COLLECTOR_H__` header guard to `VMAF_FEATURE_COLLECTOR_INCLUDED` (ADR-0141 touched-file rule). 4-subtest reducer in `test_score_pooled_eagain.c` verified to fail pre-fix, pass post-fix. Closes backlog item T1-1. | Accepted | api, correctness, motion, netflix-upstream | @@ -1073,4 +1073,5 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-1249](1249-praetor-governance-adoption.md) | Adopt the praetor / HISS-16 governance system before `1.0.0-rc.1` as scaffolding and a ratchet only: declarations, the six compiled agent-context projections, generated policy artefacts and one required CI gate land now, while the five pre-migration epic tasks (#1444–#1448) wait until after the release and no product code is refactored for them. `AGENTS.md` becomes a harness of about 250 lines in the engine's internal register, which `compile-context --verify` and `audit` lint from praetor `e4b35cb` on; its hard rules and rebase-sensitive invariants move to `docs/development/` pages that the harness imports, and the rules only the hand-written `CLAUDE.md` carried move with them. The reviewer personas gain canonical sources in `.agents/agents/`. Lefthook owns the `pre-commit` and `pre-push` hooks and delegates both to the pre-commit framework, while the ADR-1241 dispatchers keep `commit-msg` and `pre-rebase`, superseding only ADR-0924's conclusion that a second hook manager is unacceptable. `sync --remote` stays unused: the engine hardcodes `main` while this fork's default branch is `master`. | Proposed | ci, process, agents, tooling, governance, docs, workspace | | [ADR-1256](1256-cambi-spatial-mask-simd-dispatch.md) | CAMBI's spatial-mask row kernels (`compute_dp_row`, `compute_mask_row`, from upstream `86da14d03`) are dispatched per ISA only where they measurably beat scalar: dp row on AVX2 / AVX-512 / NEON, mask row on AVX2 / AVX-512. Upstream's AVX2 dp row is 0.64–0.74x of scalar under Clang and icx, so the fork keeps a single add on the loop-carried chain (1.8–2.3x AVX2, 2.6–3.2x AVX-512 at 1080p on Zen 5); the AVX2 mask row compares with a 2^31 bias so it is exact for every input. `compute_mask_row_neon` stays parity-tested but undispatched because GCC and Clang already compile the scalar loop to the same NEON code. Scores are byte-identical. | Accepted | simd, avx2, avx512, neon, cambi, perf, upstream-port, fork-local | | [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0728 decided. The `Ubuntu i686 gcc` lane it removed had come back by accident in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151 is marked superseded by ADR-0728. | Accepted | build, ci, x86, netflix-upstream, fork-local | +| [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0691 decided and ADR-0728 restated in docs. The `Ubuntu i686 gcc` lane ADR-0691 removed (`9aa008e70`) had come back the same day in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151, whose lane ran until now, is superseded by this ADR. | Accepted | build, ci, x86, netflix-upstream, fork-local | | [ADR-1247](1247-scorecard-exact-head-gates.md) | Bind Scorecard gates to their measured source and scope | Accepted | ci, security, supply-chain | diff --git a/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md b/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md index d2a4e62e8..de5bc91ab 100644 --- a/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md +++ b/docs/adr/_index_fragments/0151-i686-ci-netflix-1481.md @@ -1 +1 @@ -| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Superseded by [ADR-0728](0728-native-build-sunset.md) | ci, build, x86, netflix-upstream | +| [ADR-0151](0151-i686-ci-netflix-1481.md) | Add a 32-bit x86 (i686) build-only row to `.github/workflows/libvmaf-build-matrix.yml` to reproduce Netflix upstream issue #1481. New cross-file `build-aux/i686-linux-gnu.ini` (gcc + `-m32`, `cpu_family = 'x86'`, `cpu = 'i686'`); new install-deps step for `gcc-multilib g++-multilib`; matrix row runs `meson_extra: --cross-file=build-aux/i686-linux-gnu.ini -Denable_asm=false` (pins upstream's documented workaround); test + tox steps skipped for the i686 leg because meson marks cross-built tests `SKIP 77`. Scope note: fixing the actual `_mm256_extract_epi64` compile failure (24 call sites in `adm_avx2.c`) is explicitly out of scope — this ADR adds the CI gate only. Closes backlog item T4-8. | Superseded by [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | ci, build, x86, netflix-upstream | diff --git a/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md b/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md index 3f7bb7fd4..bfdf5c027 100644 --- a/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md +++ b/docs/adr/_index_fragments/1258-keep-64-bit-only-retire-i686-lane.md @@ -1 +1 @@ -| [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0728 decided. The `Ubuntu i686 gcc` lane it removed had come back by accident in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151 is marked superseded by ADR-0728. | Accepted | build, ci, x86, netflix-upstream, fork-local | +| [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0691 decided and ADR-0728 restated in docs. The `Ubuntu i686 gcc` lane ADR-0691 removed (`9aa008e70`) had come back the same day in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151, whose lane ran until now, is superseded by this ADR. | Accepted | build, ci, x86, netflix-upstream, fork-local | diff --git a/docs/development/preflight.md b/docs/development/preflight.md index 082ed8645..317bc3900 100644 --- a/docs/development/preflight.md +++ b/docs/development/preflight.md @@ -19,7 +19,7 @@ afternoon. All three were green under local gcc: | What was written | Locally | In CI | | --- | --- | --- | -| `static_assert(UINT_MAX <= SIZE_MAX/2/sizeof(ptr))` | fine on 64-bit | the i686 lane of the time could not compile it (that lane is retired: the fork is 64-bit only, ADR-0728 / ADR-1258) | +| `static_assert(UINT_MAX <= SIZE_MAX/2/sizeof(ptr))` | fine on 64-bit | the i686 lane of the time could not compile it (that lane is retired: the fork is 64-bit only, ADR-1258) | | `__declspec(align((x)))` | gcc never compiles the MSVC branch | `Windows MSVC+CUDA` → C2059 on every use; `align()` needs a literal | | `__attribute__(noinline)` | gcc accepts the single paren | `Ubuntu clang`, `clang+DNN` and four Sanitizer lanes fail to compile | | `nullptr` in a `.c` file | gcc and clang accept it under `-std=c23` | `Windows MSVC+CUDA` → C2065 at every site; ADR-1138 keeps C TUs on `NULL` | @@ -51,8 +51,8 @@ for about three hours. A stage whose toolchain is missing is **skipped with a notice**, not failed, so the script is still useful on a partially provisioned machine. -There is no 32-bit stage. The fork is 64-bit only (ADR-0728), and the -`m32` stage went with the i686 lane it mirrored (ADR-1258). +There is no 32-bit stage. The fork is 64-bit only, and the `m32` stage went +with the i686 lane it mirrored (ADR-1258). ## Behaviours worth knowing diff --git a/docs/metrics/features.md b/docs/metrics/features.md index 85a7fe6d3..929a91344 100644 --- a/docs/metrics/features.md +++ b/docs/metrics/features.md @@ -454,7 +454,7 @@ Netflix ADM. **Backends** — `adm`: AVX2, AVX-512, NEON, CUDA, SYCL, HIP, Metal. `float_adm`: AVX2, AVX-512, NEON, CUDA, SYCL, HIP, Metal. -**32-bit x86** — the fork is 64-bit only (ADR-0728, ADR-1258). The ADM x86 +**32-bit x86** — the fork is 64-bit only (ADR-1258). The ADM x86 sources still extract 64-bit lanes through 32-bit-safe helpers (`extract_epi64()`, `extract_epi64_128()`), ported from upstream Netflix commits [`8a289703`](https://github.com/Netflix/vmaf/commit/8a289703) and diff --git a/docs/rebase-notes.md b/docs/rebase-notes.md index a76bb35f3..24d265d02 100644 --- a/docs/rebase-notes.md +++ b/docs/rebase-notes.md @@ -32,7 +32,7 @@ See [Research-2062](research/2062-cambi-spatial-mask-simd.md). ## ci/retire-i686-lane — the fork stays 64-bit only; x86 SIMD sources use no x86-64-only intrinsics (2026-09-18) - `.github/workflows/libvmaf-build-matrix.yml`: there is no i686 row - (ADR-0728, ADR-1258). Upstream Netflix/vmaf has its own 32-bit cross build + (ADR-0691, ADR-1258). Upstream Netflix/vmaf has its own 32-bit cross build (`f6d6dde1`); do not port it. A merge that restores `i686: true` rows is wrong: that is how the lane came back in `384d97d03`. - `core/src/feature/x86/adm_avx2.c`, `adm_avx512.c`: 64-bit lane extraction diff --git a/docs/state.md b/docs/state.md index 02024a920..2230bf5e6 100644 --- a/docs/state.md +++ b/docs/state.md @@ -458,6 +458,7 @@ landed fix yet._ | **T-AI-ONNX-IR14-UNLOADABLE-2026-09-18** | **The Tiny AI test job failed on master once `onnx` 1.23.0 was released.** `ai/pyproject.toml` allowed `onnx>=1.22.0,<2.0`, so CI resolved 1.23.0, whose `helper.make_model()` writes IR version 14 by default. The pinned `onnxruntime` 1.30.0 loads at most IR 13, so every test that builds a model with onnx and runs it in ORT failed with `Unsupported model IR version: 14, max supported IR version: 13` (21 tests across `test_cross_backend`, `test_learned_filter_audit`, `test_profile`, `test_bisect_model_quality` and the PTQ round trip). The same mismatch would make any model built in-repo with onnx 1.23 unloadable by the runtime the fork ships. Reproduced in isolation: the same Relu model is IR 14 and rejected with onnx 1.23.0, IR 13 and accepted with 1.22.0. **FIXED**: `onnx>=1.22.0,<1.23` in `ai/pyproject.toml` and `tools/ensemble-training-kit/pyproject.toml`, with a comment tying the cap to onnxruntime's IR limit. | — | `fix/onnx-ir-cap` | 2026-09-18 | closed | | **T-NO-ASM-SIMD-TEST-WARNINGS-2026-09-18** | **A `-Denable_asm=false` build emitted ~30 `-Wunused-function` / `-Wunused-const-variable` / `-Wunused-variable` warnings in SIMD parity-test files.** Ten files under `core/test/` (`test_vif_simd.c`, `test_ssimulacra2_simd.c`, `test_speed_simd.c`, `test_psnr_hvs_simd.c`, `test_ms_ssim_decimate.c`, `test_motion_v2_simd.c`, `test_iqa_convolve.c`, `test_integer_ssim_simd.c`, `test_cambi_simd.c`, `test_cambi.c`) defined scalar-reference helpers, fixture builders, and lookup tables/variables unconditionally while every caller sat under an ISA guard (`ARCH_X86`, `HAVE_AVX512`, `ARCH_AARCH64`, or the `ARCH_X86 \|\| ARCH_AARCH64` union `run_tests()` already uses). Measured on an i686 build; reproduces on x86-64 with asm off too. **FIXED**: each helper moved under the exact union of ISA conditions its callers use, walking the full dependency chain so the warning does not relocate to a `pick_*` dispatcher or `ref_*` scalar reference one level down (`test_ssimulacra2_simd.c` needed one guard spanning its whole pick/ref/test helper section for this reason). Nothing deleted, no `(void)`-cast or `[[maybe_unused]]` suppression. Verified zero warnings and an identical test count on `-Denable_asm=false` x86-64, default x86-64, and an aarch64 cross build under `qemu-aarch64-static`. The aarch64 build's two `vif_neon.c` `-Wunused-but-set-variable` warnings (dead `i_dst_stride` counters in the VIF statistic kernels) are fixed on the same branch, and `vif_neon.c` is split into helpers that clear its 36 clang-tidy findings with NEON output byte-identical under qemu (68 of 68 runs). | none | `fix/no-asm-test-warnings` | 2026-09-18 | closed | | **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for four months after ADR-0728 removed it.** ADR-0728 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The other ADR-0728 removals the same merge restored are audited separately. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0728](adr/0728-native-build-sunset.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | +| **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for almost four months after ADR-0691 removed it.** ADR-0691 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row, and ADR-0728 (#52, `bfd4c436b`) restated the removal in docs without touching a workflow. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The rest of the matrix is recorded as it runs in ADR-1259. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0691](adr/0691-vmafx-drop-legacy-build-paths.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-X86-64-ONLY-INTRINSICS-2026-09-18** | **The x86 SIMD sources did not compile for 32-bit x86 with asm enabled** (the cause of [Netflix#1481](https://github.com/Netflix/vmaf/issues/1481)). `adm_avx2.c` and `adm_avx512.c` called `_mm_extract_epi64` on 12 lines each, beside an `extract_epi64()` fallback that only covered the 256-bit form, and `psnr_avx2.c` called `_mm_cvtsi128_si64`. GCC declares both only on x86-64. **FIXED**: `extract_epi64_128()` (`_mm_extract_epi64` on x86-64, a 32-bit fallback otherwise) and `_mm_storel_epi64`. An i686 build with asm then compiles warning-free. x86-64 code is unchanged (fast suite 137/137; clang-tidy and cppcheck clean). 32-bit stays unsupported (ADR-0728, ADR-1258), so this is portability hygiene. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-PREFLIGHT-M32-FALSE-FAILURES-2026-09-18** | **`scripts/dev/preflight.sh`'s 32-bit sweep failed on code the i686 lane never compiles.** It swept every changed C file with `gcc -m32`, including `arm64/`, `x86/` SIMD and GPU trees, which the `Ubuntu i686 gcc` lane skips because it configures `-Denable_asm=false` and no GPU backend. Its missing-header filter also knew only clang's `file not found` and the German gcc text, while the script runs under `LC_ALL=C`, so gcc's `No such file or directory` (e.g. `arm_neon.h`) counted as a failure. Seen on the `fix/adm-cm-simd-bitexact` branch. **FIXED**: the stage skips those trees and accepts the English message. The underlying i686+asm gap stays as Netflix#1481 / ADR-0151 describe it. | none | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | | **T-EXPORTED-SYMBOLS-ASAN-GNU-LD-2026-09-18** | **`check_exported_symbols` failed in a GNU-ld ASan build.** GNU ld exports the linker-defined `__start_asan_globals` / `__stop_asan_globals` bounds of ASan's metadata section from `libvmaf.so`; lld, which the CI sanitizer lane uses, hides them. They are sanitizer runtime artefacts, not API. **FIXED**: the checker treats `__start_` / `__stop_` bounds of the ASan, HWASan and SanitizerCoverage sections as runtime-owned; any other section bound still fails. Also cleared the file's three ruff findings. | [ADR-0379](adr/0379-libvmaf-symbol-visibility.md) | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | diff --git a/scripts/dev/preflight.sh b/scripts/dev/preflight.sh index d2f367ef2..4225a3e97 100755 --- a/scripts/dev/preflight.sh +++ b/scripts/dev/preflight.sh @@ -12,7 +12,7 @@ # # * a `static_assert` on `UINT_MAX <= SIZE_MAX/2/sizeof(ptr)` — true on LP64, # FALSE on 32-bit, so the i686 lane of the time would not compile the file -# (that lane is retired: the fork is 64-bit only, ADR-0728 / ADR-1258); +# (that lane is retired: the fork is 64-bit only, ADR-1258); # * `#define ALIGNED(x) __declspec(align((x)))` — MSVC needs a literal there, # so `Windows MSVC+CUDA` failed C2059 on every use; # * `__attribute__(noinline)` (a paren accidentally stripped) — gcc accepted From e6c03f10d0e69a776bb50a80c38ff44b62f832b7 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:22:06 +0200 Subject: [PATCH 08/15] docs(adr): record the CI build matrix as it runs (ADR-1259) Four Accepted ADRs described a matrix CI does not run. ADR-0689 (0ab991afc) and ADR-0691 (9aa008e70) removed lanes that the libvmaf/ -> core/ rename merge 384d97d03 restored about an hour later. ADR-0710 (4e211736f) added build.yml and sanitizers.yml but removed nothing and left the aggregator alone. ADR-0728 (bfd4c436b) changed no workflow at all. ADR-1259 lists every lane in libvmaf-build-matrix.yml and build.yml with its required status and owning ADR, records the maintainer's three decisions (64-bit only via ADR-1258; the five lanes without a later ADR stay, not required; Cppcheck and the three Sanitizers jobs stay required) and supersedes all four ADRs. Their status lines and index rows are flipped, and the index rows for ADR-0691 and ADR-0710 now describe what those ADRs decided. It also records that two jobs report the required check name "Windows MSVC+CUDA", so the aggregator counts only whichever started last; that is opened as T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18. --- docs/adr/0689-vmafx-ci-matrix-dedupe.md | 2 +- .../adr/0691-vmafx-drop-legacy-build-paths.md | 2 +- docs/adr/0710-vmafx-ci-slim-down-v2.md | 2 +- docs/adr/0728-native-build-sunset.md | 2 +- .../1258-keep-64-bit-only-retire-i686-lane.md | 3 +- docs/adr/1259-ci-build-matrix-as-it-runs.md | 208 ++++++++++++++++++ docs/adr/README.md | 9 +- .../0689-vmafx-ci-matrix-dedupe.md | 2 +- .../0691-vmafx-drop-legacy-build-paths.md | 2 +- .../0710-vmafx-ci-slim-down-v2.md | 2 +- .../0728-native-build-sunset.md | 2 +- .../1259-ci-build-matrix-as-it-runs.md | 1 + docs/adr/_index_fragments/_order.txt | 1 + docs/adr/by-tag/build.md | 3 +- docs/adr/by-tag/ci.md | 3 +- docs/adr/by-tag/fork-local.md | 3 +- docs/adr/by-tag/index.md | 6 +- docs/state.md | 2 + mkdocs.yml | 1 + 19 files changed, 237 insertions(+), 19 deletions(-) create mode 100644 docs/adr/1259-ci-build-matrix-as-it-runs.md create mode 100644 docs/adr/_index_fragments/1259-ci-build-matrix-as-it-runs.md diff --git a/docs/adr/0689-vmafx-ci-matrix-dedupe.md b/docs/adr/0689-vmafx-ci-matrix-dedupe.md index 96e5f784a..2d8c6af46 100644 --- a/docs/adr/0689-vmafx-ci-matrix-dedupe.md +++ b/docs/adr/0689-vmafx-ci-matrix-dedupe.md @@ -1,7 +1,7 @@ # ADR-0689: VMAFX CI Matrix Deduplication -- **Status**: Accepted +- **Status**: Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) - **Date**: 2026-05-28 - **Deciders**: lusoris - **Tags**: `ci`, `build`, `vmafx` diff --git a/docs/adr/0691-vmafx-drop-legacy-build-paths.md b/docs/adr/0691-vmafx-drop-legacy-build-paths.md index e0cd9c35e..ea3b153eb 100644 --- a/docs/adr/0691-vmafx-drop-legacy-build-paths.md +++ b/docs/adr/0691-vmafx-drop-legacy-build-paths.md @@ -1,7 +1,7 @@ # ADR-0691: VMAFX Phase 1C — Drop Legacy Build Paths -- **Status**: Accepted +- **Status**: Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) - **Date**: 2026-05-28 - **Deciders**: lusoris - **Tags**: `ci`, `build`, `vmafx` diff --git a/docs/adr/0710-vmafx-ci-slim-down-v2.md b/docs/adr/0710-vmafx-ci-slim-down-v2.md index 99141b2eb..dff0204ce 100644 --- a/docs/adr/0710-vmafx-ci-slim-down-v2.md +++ b/docs/adr/0710-vmafx-ci-slim-down-v2.md @@ -1,7 +1,7 @@ # ADR-0710: VMAFX CI Slim-Down v2 — One Build per OS + State-of-the-Art Sanitizers -- **Status**: Accepted +- **Status**: Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) - **Date**: 2026-05-28 - **Deciders**: lusoris - **Tags**: `ci`, `build`, `sanitizers`, `vmafx` diff --git a/docs/adr/0728-native-build-sunset.md b/docs/adr/0728-native-build-sunset.md index 2b5808c2f..07510af40 100644 --- a/docs/adr/0728-native-build-sunset.md +++ b/docs/adr/0728-native-build-sunset.md @@ -1,7 +1,7 @@ # ADR-0728: Sunset Legacy Native Build Modes — Phase 4b.9 Follow-On -- **Status**: Accepted +- **Status**: Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) - **Date**: 2026-05-28 - **Deciders**: lusoris - **Tags**: `ci`, `build`, `vmafx`, `breaking` diff --git a/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md b/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md index 98b916a50..3191e7f5b 100644 --- a/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md +++ b/docs/adr/1258-keep-64-bit-only-retire-i686-lane.md @@ -84,7 +84,8 @@ the same day; the lane ADR-0151 set up ran until this ADR removed it. including x87 score drift unless they build with `-msse2 -mfpmath=sse`. - **Neutral / follow-ups**: the other lanes ADR-0691 and ADR-0689 removed, which the same merge restored, and the lane removals ADR-0728 describes but - never made, are recorded in ADR-1259. + never made, are recorded in + [ADR-1259](1259-ci-build-matrix-as-it-runs.md). ## References diff --git a/docs/adr/1259-ci-build-matrix-as-it-runs.md b/docs/adr/1259-ci-build-matrix-as-it-runs.md new file mode 100644 index 000000000..2b579ce09 --- /dev/null +++ b/docs/adr/1259-ci-build-matrix-as-it-runs.md @@ -0,0 +1,208 @@ + +# ADR-1259: Record the CI build matrix as it actually runs + +- **Status**: Accepted, Supersedes + [ADR-0689](0689-vmafx-ci-matrix-dedupe.md), + [ADR-0691](0691-vmafx-drop-legacy-build-paths.md), + [ADR-0710](0710-vmafx-ci-slim-down-v2.md), + [ADR-0728](0728-native-build-sunset.md) +- **Date**: 2026-09-18 +- **Deciders**: Lusoris +- **Tags**: ci, build, fork-local + +## Context + +Four Accepted ADRs from 2026-05-28 describe a smaller build matrix than the +one CI runs. Traced to their commits: + +- ADR-0689 (`0ab991afc`, #1567) removed five rows from + `.github/workflows/libvmaf-build-matrix.yml`: `Build — Ubuntu gcc (CPU)`, + `Build — Ubuntu clang (CPU)`, `Build — macOS clang (CPU)`, the MoltenVK row + (moved to `nightly.yml`) and the dynamic `Build — Ubuntu CUDA` row. +- ADR-0691 (`9aa008e70`, #1564) removed `Build — Windows MinGW64 (CPU)` from + the matrix and from the aggregator's required list, and removed + `Build — Ubuntu i686 gcc (CPU, no-asm)`. +- About an hour later, `384d97d03`, the squash-merge of the `libvmaf/` → + `core/` rename that "merge-resolves 42 master commits", put all of that back: + the ADR-0689 rows, both ADR-0691 lanes and the MinGW64 aggregator entry. It + also deleted the `nightly.yml` MoltenVK job. No ADR asked for any of it. +- ADR-0710 (`4e211736f`, #23) says `build.yml` "replaces + `libvmaf-build-matrix.yml`", drops 18 jobs including `Cppcheck` and the + three per-PR sanitizer rows, and updates the aggregator. The commit added + `build.yml` and `sanitizers.yml` and changed no other workflow, so it + dropped none of the 18 jobs, and both build workflows have run on every PR + since. +- ADR-0728 (`bfd4c436b`, #52) calls itself the mechanical execution of + ADR-0691 and ADR-0710: 14 lane removals including the Windows SYCL leg, + `Cppcheck` and the per-PR sanitizers out of the aggregator, and the + `build.yml` and `sanitizers.yml` names in. Its commit message and PR body + describe those workflow edits, but the commit changed three files: the ADR, + `changelog.d/removed/native-build-sunset.md` and + `docs/development/deprecations.md`. ADR-0728 was never in force. The lanes + it names that are gone today went for other reasons: the Vulkan and MoltenVK + rows with the Vulkan backend (ADR-0726), and the i686 lane with ADR-1258. + +Later ADRs were written against the matrix that actually ran. ADR-1234 gives +preflight stages that mirror `Ubuntu gcc(+DNN)`, `Ubuntu clang(+DNN)`, the two +`Windows MSVC` lanes, the sanitizers and `Cppcheck`, and cites failures on +`Windows MinGW64`. ADR-1253 and ADR-1254 fix defects found on the required +`Windows MinGW64` lane. ADR-1225 moves the required `Ubuntu HIP` lane to +ROCm 10. ADR-1245 and ADR-1246 extend the required `Cppcheck` job. Meanwhile +the ADR index showed the four ADRs as Accepted, and +`docs/development/deprecations.md` told contributors that MinGW64 was gone +while it was a required check. + +## Decision + +This ADR is the record of the CI build matrix: the lanes in the tables below +run as described. The maintainer settled three open points: + +1. **32-bit x86**: the fork stays 64-bit only. + [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) carries this and + removes the i686 lane and preflight's `m32` stage. +2. **Lanes with no later ADR**: `Ubuntu SYCL`, `Ubuntu SYCL+CUDA`, + `Ubuntu gcc static`, `Ubuntu CUDA static` and `macOS clang+DNN` stay as + they are, and stay not required. +3. **Required checks**: `Cppcheck` and the three + `Sanitizers (address|thread|undefined)` jobs stay required. The + `sanitizers.yml` jobs stay not required. + +ADR-0689, ADR-0691, ADR-0710 and ADR-0728 are superseded. What still holds of +them is restated here: `build.yml` runs its three rows and `sanitizers.yml` its +three jobs (from ADR-0710), and ADR-0691's 64-bit-only rule is now ADR-1258's. + +Decision 2 was asked about the five lanes ADR-0728 names. Five more lanes are +in the same position because ADR-0689 or ADR-0710 removed them only on paper: +`Ubuntu gcc`, `Ubuntu clang`, `macOS clang`, `Ubuntu ARM clang` and +`Ubuntu CUDA`. This ADR records them as they run and makes no new decision +about them. Removing any lane, or changing what is required, needs its own ADR +and the matching change to `required-aggregator.yml`. + +In the tables, **required** means the name is in the `required` list of +`.github/workflows/required-aggregator.yml`, which is the only context branch +protection enforces. **Not required** means the lane reports, but a failure +does not block a merge. **continue-on-error** marks the `experimental: true` +rows, whose failure does not even fail the workflow run. + +### `libvmaf-build-matrix.yml` (workflow `Builds`) + +Every lane runs on non-draft PRs and master pushes. The 14 rows of the +`libvmaf-build` job first ask the CI impact planner (ADR-1140) and report +success without building when a change does not touch the C core; the three +Windows lanes build every time. + +| Lane | What it runs | Status | Owning record | Later ADRs, notes | +| --- | --- | --- | --- | --- | +| `Ubuntu gcc` | CPU build, meson tests, tox | not required | upstream Netflix matrix, folded in by ADR-0115 | removed by ADR-0689, restored by `384d97d03`; mirrored by ADR-1234's `gcc` stage | +| `Ubuntu clang` | CPU build, meson tests, tox | not required | upstream, ADR-0115 | removed by ADR-0689, restored by `384d97d03`; ADR-1234's `clang` stage | +| `macOS clang` | CPU build, meson tests, tox | not required, continue-on-error | upstream, ADR-0115 | removed by ADR-0689, restored by `384d97d03` | +| `Ubuntu ARM clang` | CPU build, meson tests, tox on `ubuntu-24.04-arm` | not required | upstream (`b53406e36`) | dropped by ADR-0710 on paper only; ADR-1052 | +| `Ubuntu gcc+DNN` | CPU and ONNX Runtime build, meson dnn suite, meson tests, tox | required, also in `mustReport` | ADR-0120 | ADR-1234 | +| `Ubuntu clang+DNN` | as `Ubuntu gcc+DNN`, with clang | required | ADR-0120 | ADR-1234 | +| `macOS clang+DNN` | as `Ubuntu gcc+DNN`, on macOS | not required, continue-on-error | ADR-0120 | decision 2 | +| `Ubuntu HIP` | HIP host build against ROCm 10 without device kernels, HIP smoke test | required | ADR-0212 | ADR-1225, ADR-1234; ADR-1204 (Proposed) | +| `macOS Metal` | Metal build, meson tests including the Metal smoke test, tox | not required | ADR-0361 | ADR-1204 (Proposed); the verification lane in `docs/backends/metal/index.md` | +| `Ubuntu gcc static` | static CPU build, static pkg-config check, meson tests, tox | not required | fork-added before the ADR practice (`eaad70462`) | decision 2 | +| `Ubuntu CUDA static` | static CUDA build, CPU-only tests | not required | `eaad70462` | decision 2 | +| `Ubuntu SYCL` | icpx SYCL build, CPU-only tests | not required | `eaad70462` | decision 2 | +| `Ubuntu CUDA` | CUDA build, CPU-only tests | not required | `eaad70462` | removed by ADR-0689, restored by `384d97d03` | +| `Ubuntu SYCL+CUDA` | icpx SYCL and CUDA build, CPU-only tests | not required | `eaad70462` | decision 2 | +| `Windows MinGW64` | MSYS2 gcc static build, Win64 stack-alignment check, meson tests | required | ADR-0115, ADR-0116 | ADR-1234, ADR-1253, ADR-1254; removed by ADR-0691, restored by `384d97d03` | +| `Windows MSVC+CUDA` | MSVC and nvcc build, no tests | required; the name is shared, see below | ADR-0121 | ADR-1234 | +| `Windows MSVC+SYCL` | MSVC and oneAPI build, no tests | required | ADR-0121 | ADR-1234 | + +### `build.yml` (workflow `Build`) + +These rows run on non-draft PRs and master pushes, and skip PRs that change +only documentation (`paths-ignore`). + +| Lane | What it runs | Status | Owning record | Later ADRs, notes | +| --- | --- | --- | --- | --- | +| `Linux Intel LLVM` | icx/icpx build with CUDA, SYCL, HIP and DNN, meson dnn suite, meson tests, HIP smoke test; no tox | not required | ADR-0710 | GCC until #1161 (`195f88a22`); ADR-1185 | +| `macOS Clang+Metal` | CPU and Metal build, meson tests, tox | not required | ADR-0710 | ADR-1234 cites a failure it caught | +| `Windows MSVC+CUDA` | MSVC and nvcc build, CPU tests | not required in its own right; shares the required name, see below | ADR-0710 | — | + +### Sanitizer and static-analysis gates (decision 3) + +| Check | Workflow | Runs on | Status | +| --- | --- | --- | --- | +| `Sanitizers (address)`, `Sanitizers (undefined)`, `Sanitizers (thread)` | `tests-and-quality-gates.yml` | non-draft PRs, master pushes | required | +| `Cppcheck` | `lint-and-format.yml` | non-draft PRs, master pushes | required | +| `Sanitizers ASan+UBSan` | `sanitizers.yml` | non-draft PRs | not required | +| `Sanitizers TSan` | `sanitizers.yml` | master pushes | not required | +| `Fuzz ` | `sanitizers.yml` | nightly | not required | + +ADR-0015 put TSan on a nightly schedule. The required `Sanitizers (thread)` +job runs on every PR, and decision 3 keeps it there. The rest of ADR-0015 +stands, so it is not superseded. + +### The shared `Windows MSVC+CUDA` check name + +The aggregator looks required checks up by name and keeps one run per name, +the one that started last (`newestByName()` in `required-aggregator.yml`). +Since #1286 (`f93a0037f`) shortened the display names, the required ADR-0121 +lane and the `build.yml` row both report as `Windows MSVC+CUDA`, so either can +hide a failure of the other. On `7cc0cc91b` (origin/master) the `build.yml` +run was cancelled and the matrix run, which started one second later, +succeeded; only the success counts. `scripts/ci/check-aggregator-names.sh` +compares sets of names, so it cannot see a duplicate. This ADR records the +defect but does not fix it, because the fix changes a required name. It is +tracked as `T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in `docs/state.md`. + +## Alternatives considered + +| Option | Pros | Cons | Why not chosen | +| --- | --- | --- | --- | +| Record the running matrix and supersede the four ADRs (chosen) | Index, docs and CI agree; the later ADRs that depend on these lanes stay valid; no workflow change | Keeps the runner cost ADR-0710 set out to cut, and the overlap between `build.yml` and the matrix | — | +| Carry out ADR-0728 now | Eleven fewer build lanes per PR (its 14, less the Vulkan, MoltenVK and i686 rows already gone) | Removes required lanes that ADR-1225, ADR-1234, ADR-1253 and ADR-1254 depend on, and drops `Cppcheck`, which ADR-1245 and ADR-1246 extend | The maintainer chose to keep the lanes and the required checks (decisions 2 and 3) | +| Supersede ADR-0728 only | Smallest change to the record | ADR-0689, ADR-0691 and ADR-0710 would still read as in force although their removals were undone or never made | Leaves three of the four false records | +| Mark ADR-0728 Deprecated instead of Superseded | Says it never took effect | Gives no pointer to the record that is in force | A reader needs the successor | +| Leave the records as they are | No work | Contributors and agents keep acting on removals that never happened, as ADR-1234 did with the i686 lane | That is the problem this ADR fixes | + +## Consequences + +- **Positive**: the ADR index, the deprecations page, the unreleased changelog + fragments and the workflow header comments describe the matrix that runs. A + lane change now has one record to amend. +- **Negative**: every non-draft PR that touches the C core still runs 17 matrix + lanes and the three `build.yml` rows, and `build.yml` largely repeats the + matrix (`Linux Intel LLVM` against the SYCL, CUDA and HIP lanes, + `macOS Clang+Metal` against `macOS Metal`, and the two `Windows MSVC+CUDA` + jobs). The shared check name stays open. +- **Neutral / follow-ups**: the unreleased changelog fragments that announced + the ADR-0689, ADR-0691, ADR-0710 and ADR-0728 removals are removed or + corrected, so the first VMAFx release notes do not announce them. Trimming + the overlap, and renaming one of the two `Windows MSVC+CUDA` jobs, are + separate decisions. + +## References + +- Superseded: [ADR-0689](0689-vmafx-ci-matrix-dedupe.md), + [ADR-0691](0691-vmafx-drop-legacy-build-paths.md), + [ADR-0710](0710-vmafx-ci-slim-down-v2.md), + [ADR-0728](0728-native-build-sunset.md). +- Related: [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md), + [ADR-0015](0015-ci-matrix-asan-ubsan-tsan.md), + [ADR-0115](0115-ci-trigger-master-only-and-matrix-consolidation.md), + [ADR-0116](0116-ci-workflow-naming-convention.md), + [ADR-0120](0120-ai-enabled-ci-matrix-legs.md), + [ADR-0121](0121-windows-gpu-build-only-legs.md), + [ADR-0212](0212-hip-backend-scaffold.md), + [ADR-0313](0313-ci-required-checks-aggregator.md), + [ADR-0361](0361-metal-compute-backend.md), + [ADR-0726](0726-drop-vulkan-backend.md), + [ADR-1140](1140-ci-impact-planner.md), + [ADR-1225](1225-rocm-10-therock-migration.md), + [ADR-1234](1234-local-preflight-gate.md), + [ADR-1245](1245-cppcheck-exhaustive-configured-analysis.md), + [ADR-1246](1246-cppcheck-public-entrypoints.md), + [ADR-1253](1253-scalar-fma-not-fused-on-msvcrt.md), + [ADR-1254](1254-win64-cannot-realign-the-stack.md). +- Commits: `0ab991afc` (#1567, ADR-0689), `9aa008e70` (#1564, ADR-0691), + `384d97d03` (layout rename merge), `4e211736f` (#23, ADR-0710), + `bfd4c436b` (#52, ADR-0728), `b53406e36` (upstream ARM lane), `eaad70462` + (static, SYCL and CUDA lanes), `195f88a22` (#1161), `f93a0037f` (#1286). +- Popup, 2026-09-18, 32-bit x86: "Stay 64-bit only (ADR-0728)". +- Popup, 2026-09-18, lanes without a later ADR: "Keep them, record it + (Recommended)". +- Popup, 2026-09-18, required checks: "Keep the four required (Recommended)". diff --git a/docs/adr/README.md b/docs/adr/README.md index e9fb98714..dc71a2bd4 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -659,9 +659,9 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-0686](0686-vmafx-rebrand-aggressive-modernization.md) | VMAFX rebrand and aggressive modernization — umbrella ADR covering rename, Phase 1–4 plan, and multi-language strategy | Proposed | meta, vmafx, rebrand, modernization | | [ADR-0687](0687-chug-hdr-held-out-test-validator.md) | CHUG HDR MOS head — held-out test partition validator | Accepted | ai, training, hdr, validation, fork-local | | [ADR-0688](0688-hip-wave32-vif-motion-fix.md) | HIP wave32 carry-preserving int64 reduction for VIF and motion kernels | Accepted | hip, vif, motion, numerics, rocm, fork-local | -| [ADR-0689](0689-vmafx-ci-matrix-dedupe.md) | VMAFX CI matrix deduplication — remove redundant job axes | Accepted | ci, build, vmafx, fork-local | +| [ADR-0689](0689-vmafx-ci-matrix-dedupe.md) | VMAFX CI matrix deduplication — remove five redundant build rows (bare CPU legs, dynamic CUDA, MoltenVK to nightly). Undone the same day by the layout rename merge `384d97d03`; the rows still run. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | ci, build, vmafx, fork-local | | [ADR-0690](0690-vmafx-binary-and-ai-aliases.md) | VMAFX binary and AI tool aliases (`vmafx`, `vmafx-tune`, `vmafx-mcp`) | Accepted | cli, build, vmafx, aliases, fork-local | -| [ADR-0691](0691-vmafx-drop-legacy-build-paths.md) | VMAFX Phase 1C — drop legacy build paths (`build-cpu`, `build-cuda`, `build-all` → unified `build/`) | Accepted | build, meson, vmafx, phase1, fork-local | +| [ADR-0691](0691-vmafx-drop-legacy-build-paths.md) | VMAFX Phase 1C — drop the Windows MinGW64 and i686 CI lanes; the fork is 64-bit only. Both lanes came back the same day in the layout rename merge `384d97d03`; MinGW64 is a required lane again, and the 64-bit-only rule now lives in ADR-1258. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | build, meson, vmafx, phase1, fork-local | | [ADR-0692](0692-vmafx-c23-bump.md) | Bump C standard to C23 (VMAFX rebrand Phase 1D); fix `test_propagate_metadata` prototype mismatch; add `-Wimplicit-fallthrough`. | Accepted | build, c, standards, meson, fork-local, vmafx-rebrand | | [ADR-0694](0694-vmafx-lint-sanitizer-gates.md) | Tighten clang-tidy enforcement and confirm ASan/UBSan/TSan/MSan as required CI gates | Accepted | ci, lint, sanitizers, clang-tidy, fork-local | | [ADR-0696](0696-vmafx-netflix-compat.md) | `--netflix-compat` flag for restoring legacy VMAF defaults | Accepted | cli, compat, vmafx, fork-local | @@ -677,7 +677,7 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-0707](0707-vmafx-rust-pilot-feature.md) | TAD (Temporal Absolute Difference) — first Rust feature extractor; proves cbindgen → Meson → libvmaf.so integration | Accepted | rust, build, metrics, feature-extractor, phase4, fork-local | | [ADR-0708](0708-vmafx-cpp23-internals-pilot.md) | C++20 internals pilot: convert `metadata_handler.c` to `.cpp` with RAII `unique_ptr` linked-list teardown; establish per-file C++20 migration recipe for Wave 1–3 | Accepted | build, c++, cpp23, refactor, internals, fork-local, vmafx-rebrand | | [ADR-0709](0709-vmafx-phase4b-distributed-platform.md) | VMAFX Phase 4b — distributed video-quality, encoding, and ML platform: controller/node/operator, ffmpeg, rclone, eBPF | Proposed | go, k8s, operator, controller, ffmpeg, rclone, ebpf, onnx, phase4b, fork-local | -| [ADR-0710](0710-vmafx-ci-slim-down-v2.md) | VMAFX CI Slim-Down v2 — one build per OS, ASan/UBSan/MSan/TSan, GitHub-hosted-only runners | Accepted | ci, build, sanitizers, fork-local | +| [ADR-0710](0710-vmafx-ci-slim-down-v2.md) | VMAFX CI Slim-Down v2 — one build per OS (`build.yml`) and a combined ASan+UBSan / TSan / fuzz workflow (`sanitizers.yml`). Both were added, but `libvmaf-build-matrix.yml`, `Cppcheck` and the per-PR sanitizer matrix were never removed and the aggregator never changed. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | ci, build, sanitizers, fork-local | | [ADR-0711](0711-vmafx-controller-impl.md) | vmafx-controller Phase 4b.1 — Go service: gRPC + HTTP, in-memory job queue, persistent node registry, FIFO scheduler | Accepted | go, controller, phase4b, grpc, http, fork-local | | [ADR-0712](0712-ide-config-multilang-refresh.md) | IDE config audit and refresh for multi-language post-rebrand VMAFX: clangd, gopls, rust-analyzer, Python LSP | Accepted | ide, clangd, gopls, rust-analyzer, build, fork-local | | [ADR-0713](0713-vmafx-node-impl.md) | vmafx-node Go worker binary — ffmpeg decode/encode/score pipeline, gRPC heartbeat to controller | Proposed | go, node, ffmpeg, phase4b, fork-local | @@ -790,7 +790,7 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-0725](0725-cpp23-pilot-log-v2.md) | C++23 Pilot — `log.c` conversion (real C++23, supersedes ADR-0722) | Accepted | build, c, cpp23, refactor, internals, fork-local, vmafx-rebrand | | [ADR-0726](0726-drop-vulkan-backend.md) | Drop Vulkan backend | Accepted | vulkan, gpu, backend, build, breaking, fork-local | | [ADR-0727](0727-cpp23-wave2-bump-and-dict.md) | C++23 Wave 2: project-wide `cpp_std=c++23` + `dict.c` → `dict.cpp` with `std::expected`, `std::string_view`, `[[nodiscard]]`; toolchain floor gcc >= 13 / clang >= 16 | Accepted | 2026-05-28 | build, c++, cpp23, refactor, internals, fork-local, vmafx-rebrand | -| [ADR-0728](0728-native-build-sunset.md) | Sunset Legacy Native Build Modes — Phase 4b.9 Follow-On | Accepted | ci, build, vmafx, breaking | +| [ADR-0728](0728-native-build-sunset.md) | Sunset Legacy Native Build Modes — Phase 4b.9 Follow-On. Never carried out: its commit changed only the ADR, a changelog fragment and `deprecations.md`, and no lane it lists was removed by it. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | ci, build, vmafx, breaking | | [ADR-0729](0729-cpp23-wave3-bundle.md) | C++23 Wave 3 — feature_name, picture_copy, model | Accepted | build, cpp23, refactor | | [ADR-0730](0730-vmafx-tune-go-stage2.md) | vmafx-tune Go port — Stage 2 (ladder subcommand) | Accepted | go, vmafx-tune, language-modernization, cli, phase4, fork-local | | [ADR-0731](0731-cpp23-wave3-part-b.md) | C++23 Wave 3 Part B — psnr_tools, luminance_tools, mkdirp | Accepted | build, cpp23, modernization | @@ -1074,4 +1074,5 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-1256](1256-cambi-spatial-mask-simd-dispatch.md) | CAMBI's spatial-mask row kernels (`compute_dp_row`, `compute_mask_row`, from upstream `86da14d03`) are dispatched per ISA only where they measurably beat scalar: dp row on AVX2 / AVX-512 / NEON, mask row on AVX2 / AVX-512. Upstream's AVX2 dp row is 0.64–0.74x of scalar under Clang and icx, so the fork keeps a single add on the loop-carried chain (1.8–2.3x AVX2, 2.6–3.2x AVX-512 at 1080p on Zen 5); the AVX2 mask row compares with a 2^31 bias so it is exact for every input. `compute_mask_row_neon` stays parity-tested but undispatched because GCC and Clang already compile the scalar loop to the same NEON code. Scores are byte-identical. | Accepted | simd, avx2, avx512, neon, cambi, perf, upstream-port, fork-local | | [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0728 decided. The `Ubuntu i686 gcc` lane it removed had come back by accident in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151 is marked superseded by ADR-0728. | Accepted | build, ci, x86, netflix-upstream, fork-local | | [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0691 decided and ADR-0728 restated in docs. The `Ubuntu i686 gcc` lane ADR-0691 removed (`9aa008e70`) had come back the same day in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151, whose lane ran until now, is superseded by this ADR. | Accepted | build, ci, x86, netflix-upstream, fork-local | +| [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs. ADR-0689 and ADR-0691 removed lanes that the layout rename merge `384d97d03` restored about an hour later; ADR-0710 added `build.yml` and `sanitizers.yml` but removed nothing; ADR-0728's commit touched no workflow. All four are superseded. A table lists every lane in `libvmaf-build-matrix.yml` and `build.yml` with its required/advisory status and owning ADR. Maintainer decisions: the fork stays 64-bit only (ADR-1258); `Ubuntu SYCL`, `Ubuntu SYCL+CUDA`, `Ubuntu gcc static`, `Ubuntu CUDA static` and `macOS clang+DNN` stay, not required; `Cppcheck` and the three `Sanitizers (...)` jobs stay required. Records that two jobs report the required name `Windows MSVC+CUDA`. | Accepted | ci, build, fork-local | | [ADR-1247](1247-scorecard-exact-head-gates.md) | Bind Scorecard gates to their measured source and scope | Accepted | ci, security, supply-chain | diff --git a/docs/adr/_index_fragments/0689-vmafx-ci-matrix-dedupe.md b/docs/adr/_index_fragments/0689-vmafx-ci-matrix-dedupe.md index 7a2aced2a..b8b4193e7 100644 --- a/docs/adr/_index_fragments/0689-vmafx-ci-matrix-dedupe.md +++ b/docs/adr/_index_fragments/0689-vmafx-ci-matrix-dedupe.md @@ -1 +1 @@ -| [ADR-0689](0689-vmafx-ci-matrix-dedupe.md) | VMAFX CI matrix deduplication — remove redundant job axes | Accepted | ci, build, vmafx, fork-local | +| [ADR-0689](0689-vmafx-ci-matrix-dedupe.md) | VMAFX CI matrix deduplication — remove five redundant build rows (bare CPU legs, dynamic CUDA, MoltenVK to nightly). Undone the same day by the layout rename merge `384d97d03`; the rows still run. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | ci, build, vmafx, fork-local | diff --git a/docs/adr/_index_fragments/0691-vmafx-drop-legacy-build-paths.md b/docs/adr/_index_fragments/0691-vmafx-drop-legacy-build-paths.md index 7b4c7bb02..37ee66667 100644 --- a/docs/adr/_index_fragments/0691-vmafx-drop-legacy-build-paths.md +++ b/docs/adr/_index_fragments/0691-vmafx-drop-legacy-build-paths.md @@ -1 +1 @@ -| [ADR-0691](0691-vmafx-drop-legacy-build-paths.md) | VMAFX Phase 1C — drop legacy build paths (`build-cpu`, `build-cuda`, `build-all` → unified `build/`) | Accepted | build, meson, vmafx, phase1, fork-local | +| [ADR-0691](0691-vmafx-drop-legacy-build-paths.md) | VMAFX Phase 1C — drop the Windows MinGW64 and i686 CI lanes; the fork is 64-bit only. Both lanes came back the same day in the layout rename merge `384d97d03`; MinGW64 is a required lane again, and the 64-bit-only rule now lives in ADR-1258. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | build, meson, vmafx, phase1, fork-local | diff --git a/docs/adr/_index_fragments/0710-vmafx-ci-slim-down-v2.md b/docs/adr/_index_fragments/0710-vmafx-ci-slim-down-v2.md index 1c2323fc1..3257cf385 100644 --- a/docs/adr/_index_fragments/0710-vmafx-ci-slim-down-v2.md +++ b/docs/adr/_index_fragments/0710-vmafx-ci-slim-down-v2.md @@ -1 +1 @@ -| [ADR-0710](0710-vmafx-ci-slim-down-v2.md) | VMAFX CI Slim-Down v2 — one build per OS, ASan/UBSan/MSan/TSan, GitHub-hosted-only runners | Accepted | ci, build, sanitizers, fork-local | +| [ADR-0710](0710-vmafx-ci-slim-down-v2.md) | VMAFX CI Slim-Down v2 — one build per OS (`build.yml`) and a combined ASan+UBSan / TSan / fuzz workflow (`sanitizers.yml`). Both were added, but `libvmaf-build-matrix.yml`, `Cppcheck` and the per-PR sanitizer matrix were never removed and the aggregator never changed. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | ci, build, sanitizers, fork-local | diff --git a/docs/adr/_index_fragments/0728-native-build-sunset.md b/docs/adr/_index_fragments/0728-native-build-sunset.md index 54e9037be..327de4e5a 100644 --- a/docs/adr/_index_fragments/0728-native-build-sunset.md +++ b/docs/adr/_index_fragments/0728-native-build-sunset.md @@ -1 +1 @@ -| [ADR-0728](0728-native-build-sunset.md) | Sunset Legacy Native Build Modes — Phase 4b.9 Follow-On | Accepted | ci, build, vmafx, breaking | +| [ADR-0728](0728-native-build-sunset.md) | Sunset Legacy Native Build Modes — Phase 4b.9 Follow-On. Never carried out: its commit changed only the ADR, a changelog fragment and `deprecations.md`, and no lane it lists was removed by it. | Superseded by [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | ci, build, vmafx, breaking | diff --git a/docs/adr/_index_fragments/1259-ci-build-matrix-as-it-runs.md b/docs/adr/_index_fragments/1259-ci-build-matrix-as-it-runs.md new file mode 100644 index 000000000..0b15610f4 --- /dev/null +++ b/docs/adr/_index_fragments/1259-ci-build-matrix-as-it-runs.md @@ -0,0 +1 @@ +| [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs. ADR-0689 and ADR-0691 removed lanes that the layout rename merge `384d97d03` restored about an hour later; ADR-0710 added `build.yml` and `sanitizers.yml` but removed nothing; ADR-0728's commit touched no workflow. All four are superseded. A table lists every lane in `libvmaf-build-matrix.yml` and `build.yml` with its required/advisory status and owning ADR. Maintainer decisions: the fork stays 64-bit only (ADR-1258); `Ubuntu SYCL`, `Ubuntu SYCL+CUDA`, `Ubuntu gcc static`, `Ubuntu CUDA static` and `macOS clang+DNN` stay, not required; `Cppcheck` and the three `Sanitizers (...)` jobs stay required. Records that two jobs report the required name `Windows MSVC+CUDA`. | Accepted | ci, build, fork-local | diff --git a/docs/adr/_index_fragments/_order.txt b/docs/adr/_index_fragments/_order.txt index 17984973c..82bde6010 100644 --- a/docs/adr/_index_fragments/_order.txt +++ b/docs/adr/_index_fragments/_order.txt @@ -982,3 +982,4 @@ 1249-praetor-governance-adoption 1256-cambi-spatial-mask-simd-dispatch 1258-keep-64-bit-only-retire-i686-lane +1259-ci-build-matrix-as-it-runs diff --git a/docs/adr/by-tag/build.md b/docs/adr/by-tag/build.md index 935c55e1d..0479f248d 100644 --- a/docs/adr/by-tag/build.md +++ b/docs/adr/by-tag/build.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -163 ADR(s) carry this tag. +164 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -170,3 +170,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1254](../1254-win64-cannot-realign-the-stack.md) | Wide vector register pressure is a Win64 correctness constraint, not a performance one | | [ADR-1255](../1255-spdx-residual-identifier-correction.md) | Correct the SPDX identifiers PR #1457 does not reach, to the licence this repository already declares | | [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | +| [ADR-1259](../1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs | diff --git a/docs/adr/by-tag/ci.md b/docs/adr/by-tag/ci.md index 4dc0790e8..70f7b6fb6 100644 --- a/docs/adr/by-tag/ci.md +++ b/docs/adr/by-tag/ci.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -193 ADR(s) carry this tag. +194 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -200,3 +200,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1252](../1252-solo-maintainer-declared-bypass.md) | Declare the single maintainer's bypass actor | | [ADR-1254](../1254-win64-cannot-realign-the-stack.md) | Wide vector register pressure is a Win64 correctness constraint, not a performance one | | [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | +| [ADR-1259](../1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs | diff --git a/docs/adr/by-tag/fork-local.md b/docs/adr/by-tag/fork-local.md index f92cb700e..944a1b887 100644 --- a/docs/adr/by-tag/fork-local.md +++ b/docs/adr/by-tag/fork-local.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -405 ADR(s) carry this tag. +406 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -412,3 +412,4 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1251](../1251-renovate-draft-automerge-deadlock.md) | Renovate opens automerge-eligible and security bumps ready for review | | [ADR-1252](../1252-solo-maintainer-declared-bypass.md) | Declare the single maintainer's bypass actor | | [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | +| [ADR-1259](../1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs | diff --git a/docs/adr/by-tag/index.md b/docs/adr/by-tag/index.md index eb39708d2..7f605a2d8 100644 --- a/docs/adr/by-tag/index.md +++ b/docs/adr/by-tag/index.md @@ -72,7 +72,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [bug](bug.md) | 15 | | [bug-fix](bug-fix.md) | 8 | | [bugfix](bugfix.md) | 24 | -| [build](build.md) | 163 | +| [build](build.md) | 164 | | [bvi-dvc](bvi-dvc.md) | 2 | | [c](c.md) | 9 | | [c++](c++.md) | 12 | @@ -93,7 +93,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [chart](chart.md) | 2 | | [chroma](chroma.md) | 3 | | [chug](chug.md) | 11 | -| [ci](ci.md) | 193 | +| [ci](ci.md) | 194 | | [ci-gate](ci-gate.md) | 1 | | [ciede](ciede.md) | 2 | | [clang-tidy](clang-tidy.md) | 4 | @@ -215,7 +215,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [float_adm](float_adm.md) | 1 | | [float_moment](float_moment.md) | 1 | | [fork-internal](fork-internal.md) | 2 | -| [fork-local](fork-local.md) | 405 | +| [fork-local](fork-local.md) | 406 | | [fork-policy](fork-policy.md) | 1 | | [fp64](fp64.md) | 1 | | [fr](fr.md) | 1 | diff --git a/docs/state.md b/docs/state.md index 2230bf5e6..c57e2e8ca 100644 --- a/docs/state.md +++ b/docs/state.md @@ -392,6 +392,7 @@ landed fix yet._ | Bug | Summary | Reproducer | Owner | Target | | --- | --- | --- | --- | --- | +| **T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18** | **Two jobs report the required check name `Windows MSVC+CUDA`.** The required lane in `libvmaf-build-matrix.yml` (ADR-0121, build only) and the Windows row of `build.yml` (build plus CPU tests) both got that display name when #1286 (`f93a0037f`) shortened job names. The aggregator's `newestByName()` keeps one check run per name, the one that started last, so a failure in either job can be hidden by a success in the other. On `7cc0cc91b` (origin/master) the `build.yml` run was cancelled and the matrix run, started one second later, succeeded; only the success counted. `scripts/ci/check-aggregator-names.sh` compares sets of names, so it cannot see the duplicate. Recorded in [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md). | `grep -n 'name: Windows MSVC+CUDA' .github/workflows/build.yml .github/workflows/libvmaf-build-matrix.yml` lists both; `gh api repos/VMAFx/vmafx/commits/7cc0cc91b/check-runs --jq '.check_runs[] \| select(.name=="Windows MSVC+CUDA") \| [.conclusion, .started_at, .html_url]'` shows the two runs. | CI owner. Renaming either job changes a required name or adds one, so the fix needs an ADR and the matching `required-aggregator.yml` change. | Closes when every name in the aggregator's `required` list is reported by exactly one job, and `check-aggregator-names.sh` rejects a required name that two jobs share. | | **T-UPSTREAM-1109-ORIGINAL-PSNR-SYMPTOM-2026-09-08** | **Original cause unresolved; cap fix has narrower scope.** The reporter observed 72 dB from libvmaf where FFmpeg's `psnr` reported 28 dB. An upper cap alone cannot raise 28 to 72. The previously asserted frame-alignment/filter-order diagnosis was not established; the reporter rejected timestamp mismatch because both streams contain static frames. This investigation has not reproduced the original input/filtergraph combination. The *mechanism* the report describes — an unconditional per-frame `MIN` that truncates genuinely near-lossless frames — is real here and is tracked as T-UPSTREAM-1109-PSNR-CAP-TRUNCATES-2026-09-03. | [Research-1166](research/1166-upstream-issue-harvest-2026-09-03.md), [scope correction](research/1193-psnr-cap-truncation-2026-09-06.md#scope-correction-carried-forward) Reproduce original FFmpeg 5.1.1 input/filtergraph and compare with current upstream and fork; current affected status is unknown. | Upstream triage | Open — original symptom only | | **T-SYCL-MOTION-ADD-UV-TOLERANCE-RESOLUTION-2026-09-06** | `test_sycl_motion_add_uv_parity` compares the CPU **float** `float_motion` extractor against the **fixed-point** `motion_sycl`, so its `PARITY_TOL` of 2e-4 is a fixed-point-vs-float budget rather than a bit-exactness bound — the test's own comment says it "accounts for 3-plane fixed-point". That budget was calibrated for the single 256x144 fixture the test has always used. Measured on an Arc A380 at 960x540: cpu(float_motion)=50.27938271, sycl=50.27915223, delta 2.30e-04 against tol 2.00e-04 — 15% over a bound that was never derived for that resolution. Not a regression and not necessarily a defect: the quantisation error of a fixed-point path against a float reference is expected to scale with content and pixel count. It is tracked because the ADR-1206 large-fixture variant for this test is therefore **not** registered, so the resolution-dependent behaviour of `motion_add_uv` is currently untested. | `meson test -C build --suite sycl test_sycl_motion_add_uv_parity` after rebuilding that TU with `-DFIXTURE_W=960u -DFIXTURE_H=540u`, on a host with an Intel GPU. | Owner-driven. Needs a derivation, not a number: what fixed-point-vs-float error bound is justified as a function of resolution for a 3-plane motion sum? Raising 2e-4 to make one fixture pass would be exactly the test-weakening the fork forbids, so it needs its own ADR. | Closes when `test_sycl_motion_add_uv_parity` has a documented, derived tolerance valid across resolutions and is registered in `sycl_parity_large_fixture_tests`. | | **T-DISTS-PLACEHOLDER-CHECKPOINT-2026-09-08** | `docs/metrics/dists.md` documents DISTS (`float_dists_sq`) as computing Deep Image Structure and Texture Similarity via Ding et al., but `model/tiny/dists_sq.onnx` is a 3-op synthetic MSE smoke placeholder (`vmaf_tiny_dists_sq_placeholder_v0`, opset 13, `smoke: true` in `model/tiny/registry.json`), and `core/src/feature/feature_dists.c` lacks the learned deep multi-scale feature stack. While `core/src/feature/feature_dists.c` now logs a loud warning at point of use (`vmaf_log(..., VMAF_LOG_LEVEL_WARNING, "DISTS: using placeholder model checkpoint...")`) and `docs/metrics/dists.md` carries an unmissable banner, DISTS cannot be used as an accurate perceptual metric until real Ding et al. weights and feature extraction are trained and integrated. Flagged by epic #1270 as a 1.0.0 blocker. | Run `vmaf --feature float_dists_sq` or inspect `model/tiny/registry.json` (`dists_sq.onnx` has `"smoke": true`). Point-of-use warning logs at `VMAF_LOG_LEVEL_WARNING`. | unassigned | Deferred post-1.0.0; requires deep feature backbone training and allowlist validation for DISTS ops. | @@ -459,6 +460,7 @@ landed fix yet._ | **T-NO-ASM-SIMD-TEST-WARNINGS-2026-09-18** | **A `-Denable_asm=false` build emitted ~30 `-Wunused-function` / `-Wunused-const-variable` / `-Wunused-variable` warnings in SIMD parity-test files.** Ten files under `core/test/` (`test_vif_simd.c`, `test_ssimulacra2_simd.c`, `test_speed_simd.c`, `test_psnr_hvs_simd.c`, `test_ms_ssim_decimate.c`, `test_motion_v2_simd.c`, `test_iqa_convolve.c`, `test_integer_ssim_simd.c`, `test_cambi_simd.c`, `test_cambi.c`) defined scalar-reference helpers, fixture builders, and lookup tables/variables unconditionally while every caller sat under an ISA guard (`ARCH_X86`, `HAVE_AVX512`, `ARCH_AARCH64`, or the `ARCH_X86 \|\| ARCH_AARCH64` union `run_tests()` already uses). Measured on an i686 build; reproduces on x86-64 with asm off too. **FIXED**: each helper moved under the exact union of ISA conditions its callers use, walking the full dependency chain so the warning does not relocate to a `pick_*` dispatcher or `ref_*` scalar reference one level down (`test_ssimulacra2_simd.c` needed one guard spanning its whole pick/ref/test helper section for this reason). Nothing deleted, no `(void)`-cast or `[[maybe_unused]]` suppression. Verified zero warnings and an identical test count on `-Denable_asm=false` x86-64, default x86-64, and an aarch64 cross build under `qemu-aarch64-static`. The aarch64 build's two `vif_neon.c` `-Wunused-but-set-variable` warnings (dead `i_dst_stride` counters in the VIF statistic kernels) are fixed on the same branch, and `vif_neon.c` is split into helpers that clear its 36 clang-tidy findings with NEON output byte-identical under qemu (68 of 68 runs). | none | `fix/no-asm-test-warnings` | 2026-09-18 | closed | | **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for four months after ADR-0728 removed it.** ADR-0728 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The other ADR-0728 removals the same merge restored are audited separately. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0728](adr/0728-native-build-sunset.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for almost four months after ADR-0691 removed it.** ADR-0691 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row, and ADR-0728 (#52, `bfd4c436b`) restated the removal in docs without touching a workflow. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The rest of the matrix is recorded as it runs in ADR-1259. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0691](adr/0691-vmafx-drop-legacy-build-paths.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | +| **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for almost four months after ADR-0691 removed it.** ADR-0691 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row, and ADR-0728 (#52, `bfd4c436b`) restated the removal in docs without touching a workflow. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The rest of the matrix is recorded as it runs in ADR-1259. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0691](adr/0691-vmafx-drop-legacy-build-paths.md), [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-X86-64-ONLY-INTRINSICS-2026-09-18** | **The x86 SIMD sources did not compile for 32-bit x86 with asm enabled** (the cause of [Netflix#1481](https://github.com/Netflix/vmaf/issues/1481)). `adm_avx2.c` and `adm_avx512.c` called `_mm_extract_epi64` on 12 lines each, beside an `extract_epi64()` fallback that only covered the 256-bit form, and `psnr_avx2.c` called `_mm_cvtsi128_si64`. GCC declares both only on x86-64. **FIXED**: `extract_epi64_128()` (`_mm_extract_epi64` on x86-64, a 32-bit fallback otherwise) and `_mm_storel_epi64`. An i686 build with asm then compiles warning-free. x86-64 code is unchanged (fast suite 137/137; clang-tidy and cppcheck clean). 32-bit stays unsupported (ADR-0728, ADR-1258), so this is portability hygiene. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-PREFLIGHT-M32-FALSE-FAILURES-2026-09-18** | **`scripts/dev/preflight.sh`'s 32-bit sweep failed on code the i686 lane never compiles.** It swept every changed C file with `gcc -m32`, including `arm64/`, `x86/` SIMD and GPU trees, which the `Ubuntu i686 gcc` lane skips because it configures `-Denable_asm=false` and no GPU backend. Its missing-header filter also knew only clang's `file not found` and the German gcc text, while the script runs under `LC_ALL=C`, so gcc's `No such file or directory` (e.g. `arm_neon.h`) counted as a failure. Seen on the `fix/adm-cm-simd-bitexact` branch. **FIXED**: the stage skips those trees and accepts the English message. The underlying i686+asm gap stays as Netflix#1481 / ADR-0151 describe it. | none | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | | **T-EXPORTED-SYMBOLS-ASAN-GNU-LD-2026-09-18** | **`check_exported_symbols` failed in a GNU-ld ASan build.** GNU ld exports the linker-defined `__start_asan_globals` / `__stop_asan_globals` bounds of ASan's metadata section from `libvmaf.so`; lld, which the CI sanitizer lane uses, hides them. They are sanitizer runtime artefacts, not API. **FIXED**: the checker treats `__start_` / `__stop_` bounds of the ASan, HWASan and SanitizerCoverage sections as runtime-owned; any other section bound still fails. Also cleared the file's three ruff findings. | [ADR-0379](adr/0379-libvmaf-symbol-visibility.md) | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | diff --git a/mkdocs.yml b/mkdocs.yml index 4ee4c48fe..ee2ea9cab 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1382,6 +1382,7 @@ nav: - "ADR-1255: spdx-residual-identifier-correction": adr/1255-spdx-residual-identifier-correction.md - "ADR-1256: cambi-spatial-mask-simd-dispatch": adr/1256-cambi-spatial-mask-simd-dispatch.md - "ADR-1258: keep-64-bit-only-retire-i686-lane": adr/1258-keep-64-bit-only-retire-i686-lane.md + - "ADR-1259: ci-build-matrix-as-it-runs": adr/1259-ci-build-matrix-as-it-runs.md - "By tag": - Overview: adr/by-tag/index.md - "aarch64": adr/by-tag/aarch64.md From 1b544b1d2aa268ff30ee3eb2da700edd2a25f482 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:44:29 +0200 Subject: [PATCH 09/15] docs: stop describing CI lane removals that never happened ADR-1259 records the build matrix as it runs; these pages and fragments still described ADR-0689, ADR-0691, ADR-0710 and ADR-0728 as carried out. - deprecations.md: the ADR-0728 entry is marked withdrawn and says what became of each configuration it listed (MinGW64, Cppcheck and the three sanitizers are required checks); a 2026-09-18 entry records the i686 lane; tox does not run in the Linux all-backends lane. - changelog.d: the unreleased native-build-sunset, 0691 and 0689 fragments announced removals that never happened and are deleted; ci-slim-down-v2 now says build.yml and sanitizers.yml were added alongside the matrix. CHANGELOG.md is re-rendered. - release.md: the required-check inventory has 40 names, not 34 (including the Standards & Invariant Verification Gate from ADR-1249), and Semgrep is required. - ci-job-names.md: the "Previous Name" column now holds the names #1286 actually replaced (from the diff of f93a0037f), the eleven non-required matrix lanes and six later required checks are added, and the count is corrected to 40 names across eleven workflows. - ci.md: lists build.yml and sanitizers.yml and points at ADR-1259. - windows-d3d11-import.md: Windows CI is not "MinGW + no-SYCL"; the required Windows MSVC+SYCL lane compiles d3d11_import.cpp. - metal/index.md: the Metal lanes are not required checks. - .github/AGENTS.md: the MoltenVK lane section is replaced by the ADR-1259 invariant; macOS clang and macOS clang+DNN are the continue-on-error rows; current Tidy SYCL job name. - core/src/hip/AGENTS.md: the lane is "Ubuntu HIP" and installs ROCm 10. - rebase-notes.md: do not restore the deleted fragments or drop lanes in a merge resolution. --- .github/AGENTS.md | 66 +++---- CHANGELOG.md | 85 ++------- .../changed/0689-vmafx-ci-matrix-dedupe.md | 9 - changelog.d/changed/ci-slim-down-v2.md | 32 ++-- .../0691-vmafx-drop-legacy-build-paths.md | 8 - changelog.d/removed/native-build-sunset.md | 30 ---- core/src/hip/AGENTS.md | 11 +- docs/backends/metal/index.md | 13 +- docs/development/ci-job-names.md | 161 +++++++++++------- docs/development/ci.md | 8 +- docs/development/deprecations.md | 82 +++++---- docs/development/release.md | 29 +++- docs/development/windows-d3d11-import.md | 13 +- docs/rebase-notes.md | 12 ++ 14 files changed, 256 insertions(+), 303 deletions(-) delete mode 100644 changelog.d/changed/0689-vmafx-ci-matrix-dedupe.md delete mode 100644 changelog.d/removed/0691-vmafx-drop-legacy-build-paths.md delete mode 100644 changelog.d/removed/native-build-sunset.md diff --git a/.github/AGENTS.md b/.github/AGENTS.md index 970bbdb94..442fc1b39 100644 --- a/.github/AGENTS.md +++ b/.github/AGENTS.md @@ -439,14 +439,13 @@ step in `libvmaf-build` job: Rebase-sensitive invariants: -- `if:` triple condition is load-bearing. **All three clauses must be - preserved together.** Dropping `github.event_name == 'workflow_dispatch'` - causes step to open blocking SSH session on every failing PR push, - stranding macOS runner for up to 30 minutes per failure. -- Step must remain **after** `Run tests` step and **before** - `Run Vulkan smoke tests (macOS MoltenVK)` step so it fires only when - test failure has already set job status to `failure()`. -- Action is pinned to commit SHA per fork's Renovate +- `if:` triple condition load-bearing. **All three clauses preserved + together.** Dropping `github.event_name == 'workflow_dispatch'` causes step + to open blocking SSH session on every failing PR push. Strands macOS runner + up to 30 minutes per failure. +- Step stays **after** `Run tests` step -> fires only when test failure already + set job status to `failure()`. +- Action pinned to commit SHA per fork's Renovate `helpers:pinGitHubActionDigests` policy. Renovate will propose digest bumps; accept only after verifying new SHA corresponds to signed release tag. - Step is intentionally present in shared matrix job (not separate @@ -456,33 +455,19 @@ Rebase-sensitive invariants: See [ADR-0626](../docs/adr/0626-macos-ci-tmate-debug-on-failure.md) and [`docs/development/ci-tmate-debug.md`](../docs/development/ci-tmate-debug.md). -## macOS Vulkan-via-MoltenVK lane (ADR-0338) - -`libvmaf-build-matrix.yml` carries advisory lane -`Build — macOS Vulkan via MoltenVK (advisory)` that runs on -`macos-latest` (Apple Silicon). Rebase-sensitive invariants: - -- Lane is gated `continue-on-error: ${{ matrix.experimental == - true && matrix.moltenvk == true }}`. Compound predicate is - load-bearing — matrix has other `experimental: true` rows - (macOS DNN lane) that must keep their default fail-fast - behaviour. Naive simplification to `${{ matrix.experimental }}` - would silently make those other rows advisory. -- `VK_ICD_FILENAMES` MUST point at - `/opt/homebrew/etc/vulkan/icd.d/MoltenVK_icd.json` — homebrew - formula `molten-vk` lays JSON under `etc/vulkan/`, NOT - `share/vulkan/`. Never "fix" path; verify against - `Formula/m/molten-vk.rb` if in doubt. -- Lane must NOT be added to `required-aggregator.yml` until one - green run lands on `master`. See ADR-0338 §Decision. -- Existing `Run tests` / cache / tox steps gate on - `!matrix.moltenvk` — moltenvk lane runs its own dedicated - Vulkan-only smoke step. Never unify or lane will try to run - tox tests against Apple-Vulkan build, which is not lane's - contract. - -See [ADR-0338](../docs/adr/0338-macos-vulkan-via-moltenvk-lane.md) -and [`docs/backends/vulkan/moltenvk.md`](../docs/backends/vulkan/moltenvk.md). +## Build matrix of record (ADR-1259) + +[ADR-1259](../docs/adr/1259-ci-build-matrix-as-it-runs.md) lists every lane in +`libvmaf-build-matrix.yml` and `build.yml` and which ones are required. +ADR-0689, ADR-0691, ADR-0710 and ADR-0728 are superseded: do not remove a lane +on their authority, and do not let a merge resolution drop or restore a lane +without an ADR. That is how `384d97d03` undid two of them. + +The MoltenVK lane (ADR-0338) went with the Vulkan backend (ADR-0726). The +`libvmaf-build` job's `continue-on-error` is now +`${{ matrix.experimental == true }}`, so the two `experimental: true` rows, +`macOS clang` and `macOS clang+DNN`, are advisory: their failure does not +fail the workflow run. Neither is a required check. ## Renovate (ADR-0363) supersedes Dependabot @@ -495,7 +480,9 @@ Note: pin updates to `codeql-action/upload-sarif` now arrive via Renovate - [ADR-1247](../docs/adr/1247-scorecard-exact-head-gates.md) — current OSSF Scorecard policy; ADR-0263 is superseded - [ADR-0338](../docs/adr/0338-macos-vulkan-via-moltenvk-lane.md) — macOS - Vulkan-via-MoltenVK advisory lane + Vulkan-via-MoltenVK advisory lane (removed with the Vulkan backend, ADR-0726) +- [ADR-1259](../docs/adr/1259-ci-build-matrix-as-it-runs.md) — the CI build + matrix as it runs - [Research-0002](../docs/research/0002-automated-rule-enforcement.md) — investigation - [Research-0053](../docs/research/0053-ossf-scorecard-investigation.md) — OSSF Scorecard per-check breakdown @@ -593,9 +580,10 @@ Unknown attribute kind (102) ``` Pass `-Db_lto=false` on every icpx/SYCL `meson setup` in CI. Both SYCL legs of -`libvmaf-build-matrix.yml` already do, and `Clang-Tidy SYCL (Changed Files, -Advisory)` now does too. Pinning older oneAPI does not help — mismatch -is against *system* linker plugin, not specific compiler release. +`libvmaf-build-matrix.yml` already do, as do `build.yml`'s `Linux Intel LLVM` +row and the `Tidy SYCL (advisory)` job. Pinning an older oneAPI does not +help — the mismatch is against the *system* linker plugin, not a specific +compiler release. ## Scorecard scope and report authenticity (ADR-1247) diff --git a/CHANGELOG.md b/CHANGELOG.md index 56acb7476..9ad0102f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10398,17 +10398,6 @@ capture used by the newer AI corpus/report tooling. `sys.path` bootstrap blocks. -**CI matrix deduplication (VMAFX Phase 1B, ADR-0689):** Remove five redundant -build rows from the PR matrix — three bare CPU legs subsumed by their DNN -counterparts (`Build — Ubuntu gcc (CPU)`, `Build — Ubuntu clang (CPU)`, -`Build — macOS clang (CPU)`), the advisory macOS MoltenVK Vulkan lane moved to -nightly, and the dynamic-only Ubuntu CUDA leg subsumed by the SYCL+CUDA -combined leg. Drop the duplicate `vulkan-vif-cross-backend` job in -`tests-and-quality-gates.yml` (the `vulkan-parity-matrix-gate` is a strict -superset). Required checks are unchanged. Approximate saving: ~15–25 min of -runner time per PR. - - - **refactor(core):** Pilot C++20 conversion of `core/src/metadata_handler.c` (renamed to `.cpp` via `git mv`). `vmaf_metadata_destroy` now uses a `std::unique_ptr` with a custom `CallbackListDeleter` that @@ -12543,28 +12532,24 @@ Ubuntu 26.04 (expected H2 2026). CUDA pins (13.2.0), Windows runners images remain at their existing pins. ADR-0802. -### CI matrix slimmed to 1 build per OS + state-of-the-art sanitizers (ADR-0710) - -`libvmaf-build-matrix.yml` (15 build rows post-ADR-0689) is replaced by -`build.yml` with three matrix rows: - -- **Linux** — GCC + ALL backends (CUDA + SYCL + Vulkan + HIP + CPU + DNN); runs - the full meson test suite including Netflix golden assertions. -- **macOS** — Apple Clang + CPU + Metal scaffold; runs meson suite + tox. -- **Windows** — MSVC + CPU + CUDA (build-only); runs CPU unit tests. +### CI: `build.yml` and `sanitizers.yml` added alongside the existing matrix (ADR-0710) -Sanitizer jobs moved from `tests-and-quality-gates.yml` into a new -`sanitizers.yml`: +`build.yml` adds one all-in-one build per OS, next to +`libvmaf-build-matrix.yml` rather than in place of it: -- `Sanitizers — ASan + UBSan (PR gate)` — combined `-fsanitize=address,undefined` - on every non-draft PR (replaces the 3-way per-PR matrix). -- `Sanitizers — TSan (master push)` — thread-sanitizer fires only on master push. -- `Fuzz — * (nightly)` — libFuzzer + ASan against all harnesses nightly. +- **`Linux Intel LLVM`**: icx/icpx with CUDA, SYCL, HIP and DNN; runs the meson + suite and the HIP smoke test. +- **`macOS Clang+Metal`**: Apple Clang with CPU and Metal; runs the meson + suite and tox. +- **`Windows MSVC+CUDA`**: MSVC with CPU and CUDA; builds and runs the CPU + unit tests. -`Cppcheck (Whole Project)` removed from lint-and-format.yml; clang-tidy provides -a superset of its signal. `Required Checks Aggregator` updated accordingly. +`sanitizers.yml` adds a combined `Sanitizers ASan+UBSan` job on pull requests, +`Sanitizers TSan` on pushes to master and nightly libFuzzer runs. -Estimated per-PR runner-time reduction: ~70% versus the pre-ADR-0689 baseline. +None of these jobs is a required check. `libvmaf-build-matrix.yml`, `Cppcheck` +and the required `Sanitizers (address|thread|undefined)` matrix are unchanged; +ADR-1259 records the matrix as it runs. - CI merge throughput: the `Required Checks Aggregator` poll deadline is raised @@ -16448,16 +16433,6 @@ See [ADR-0356](docs/adr/0356-vulkan-two-level-gpu-reduction.md) and ### Removed -- **CI: MinGW64 Windows build removed** — the `Build — Windows MinGW64 (CPU)` - job (MSYS2 / MinGW-w64 GCC, static link, `vmaf.exe` artifact) is no longer - present in the CI matrix. Windows coverage is provided by the MSVC + CUDA - and MSVC + oneAPI SYCL build-only legs. (ADR-0691, VMAFX Phase 1C) -- **CI: i686 / no-asm 32-bit Linux build removed** — the `Build — Ubuntu i686 - gcc (CPU, no-asm)` matrix entry (`--cross-file=build-aux/i686-linux-gnu.ini - -Denable_asm=false`) is no longer present. The fork targets 64-bit x86-64 - and ARM64 exclusively. (ADR-0691, VMAFX Phase 1C) - - - **testdata: orphan debug scripts and slim-schema snapshot removed** — three fork-added files under `testdata/` with zero in-tree references were deleted: `check_borders.py` (one-off DWT-subband / ADM-border arithmetic debug script @@ -16554,38 +16529,6 @@ No build/test/runtime impact — the `enable_vulkan` meson option was already re yet fired. No removals. -**BREAKING: Sunset legacy native build modes (ADR-0728)** - -The following CI build configurations have been removed from -`.github/workflows/libvmaf-build-matrix.yml` and the required-checks list: - -**Removed from CI matrix:** -- `Build — Windows MinGW64 (CPU)` — MinGW64 is not a VMAFX production target -- `Build — Ubuntu i686 gcc (CPU, no-asm)` — fork is 64-bit only -- `Build — Ubuntu gcc (CPU) + DNN` — superseded by Linux full-build in build.yml -- `Build — Ubuntu clang (CPU) + DNN` — superseded by Linux full-build in build.yml -- `Build — macOS clang (CPU) + DNN` — superseded by macOS leg in build.yml -- `Build — Ubuntu Vulkan (T5-1b runtime)` — folded into Linux full-build -- `Build — macOS Vulkan via MoltenVK (advisory)` — too fragile; no required gate -- `Build — Ubuntu HIP (T7-10b runtime)` — folded into Linux full-build -- `Build — macOS Metal (T8-1 scaffold)` — folded into macOS leg in build.yml -- `Build — Ubuntu gcc Static (CPU)` — pkgconfig verified within Linux full-build -- `Build — Ubuntu CUDA Static` — NVCC-static covered by Linux full-build -- `Build — Ubuntu SYCL` — folded into Linux full-build -- `Build — Ubuntu SYCL + CUDA` — folded into Linux full-build -- `Build — Windows MSVC + oneAPI SYCL (build only)` — SYCL in Linux full-build - -**New canonical build matrix** (`build.yml`, ADR-0710): -- `Build — Linux (GCC, all backends)` — full stack: CUDA + SYCL + Vulkan + HIP + DNN + CPU -- `Build — macOS (Clang, CPU + Metal)` — Apple Clang + CPU + Metal scaffold -- `Build — Windows (MSVC + CUDA)` — MSVC + CPU + CUDA (build-only) - -**Required-checks aggregator updated** to use new check names from `build.yml` -and `sanitizers.yml`. `Cppcheck (Whole Project)` removed (clang-tidy superset). - -Implements ADR-0691 + ADR-0710. No functional change to the C library or CLI. - - ### Removed - Reverted float-ADM SIMD dispatch wiring (PR #685, commit `b1a6c0d62`): the diff --git a/changelog.d/changed/0689-vmafx-ci-matrix-dedupe.md b/changelog.d/changed/0689-vmafx-ci-matrix-dedupe.md deleted file mode 100644 index 487de1039..000000000 --- a/changelog.d/changed/0689-vmafx-ci-matrix-dedupe.md +++ /dev/null @@ -1,9 +0,0 @@ -**CI matrix deduplication (VMAFX Phase 1B, ADR-0689):** Remove five redundant -build rows from the PR matrix — three bare CPU legs subsumed by their DNN -counterparts (`Build — Ubuntu gcc (CPU)`, `Build — Ubuntu clang (CPU)`, -`Build — macOS clang (CPU)`), the advisory macOS MoltenVK Vulkan lane moved to -nightly, and the dynamic-only Ubuntu CUDA leg subsumed by the SYCL+CUDA -combined leg. Drop the duplicate `vulkan-vif-cross-backend` job in -`tests-and-quality-gates.yml` (the `vulkan-parity-matrix-gate` is a strict -superset). Required checks are unchanged. Approximate saving: ~15–25 min of -runner time per PR. diff --git a/changelog.d/changed/ci-slim-down-v2.md b/changelog.d/changed/ci-slim-down-v2.md index 8599a1c4f..7051fe7ee 100644 --- a/changelog.d/changed/ci-slim-down-v2.md +++ b/changelog.d/changed/ci-slim-down-v2.md @@ -1,22 +1,18 @@ -### CI matrix slimmed to 1 build per OS + state-of-the-art sanitizers (ADR-0710) +### CI: `build.yml` and `sanitizers.yml` added alongside the existing matrix (ADR-0710) -`libvmaf-build-matrix.yml` (15 build rows post-ADR-0689) is replaced by -`build.yml` with three matrix rows: +`build.yml` adds one all-in-one build per OS, next to +`libvmaf-build-matrix.yml` rather than in place of it: -- **Linux** — GCC + ALL backends (CUDA + SYCL + Vulkan + HIP + CPU + DNN); runs - the full meson test suite including Netflix golden assertions. -- **macOS** — Apple Clang + CPU + Metal scaffold; runs meson suite + tox. -- **Windows** — MSVC + CPU + CUDA (build-only); runs CPU unit tests. +- **`Linux Intel LLVM`**: icx/icpx with CUDA, SYCL, HIP and DNN; runs the meson + suite and the HIP smoke test. +- **`macOS Clang+Metal`**: Apple Clang with CPU and Metal; runs the meson + suite and tox. +- **`Windows MSVC+CUDA`**: MSVC with CPU and CUDA; builds and runs the CPU + unit tests. -Sanitizer jobs moved from `tests-and-quality-gates.yml` into a new -`sanitizers.yml`: +`sanitizers.yml` adds a combined `Sanitizers ASan+UBSan` job on pull requests, +`Sanitizers TSan` on pushes to master and nightly libFuzzer runs. -- `Sanitizers — ASan + UBSan (PR gate)` — combined `-fsanitize=address,undefined` - on every non-draft PR (replaces the 3-way per-PR matrix). -- `Sanitizers — TSan (master push)` — thread-sanitizer fires only on master push. -- `Fuzz — * (nightly)` — libFuzzer + ASan against all harnesses nightly. - -`Cppcheck (Whole Project)` removed from lint-and-format.yml; clang-tidy provides -a superset of its signal. `Required Checks Aggregator` updated accordingly. - -Estimated per-PR runner-time reduction: ~70% versus the pre-ADR-0689 baseline. +None of these jobs is a required check. `libvmaf-build-matrix.yml`, `Cppcheck` +and the required `Sanitizers (address|thread|undefined)` matrix are unchanged; +ADR-1259 records the matrix as it runs. diff --git a/changelog.d/removed/0691-vmafx-drop-legacy-build-paths.md b/changelog.d/removed/0691-vmafx-drop-legacy-build-paths.md deleted file mode 100644 index c1b070f23..000000000 --- a/changelog.d/removed/0691-vmafx-drop-legacy-build-paths.md +++ /dev/null @@ -1,8 +0,0 @@ -- **CI: MinGW64 Windows build removed** — the `Build — Windows MinGW64 (CPU)` - job (MSYS2 / MinGW-w64 GCC, static link, `vmaf.exe` artifact) is no longer - present in the CI matrix. Windows coverage is provided by the MSVC + CUDA - and MSVC + oneAPI SYCL build-only legs. (ADR-0691, VMAFX Phase 1C) -- **CI: i686 / no-asm 32-bit Linux build removed** — the `Build — Ubuntu i686 - gcc (CPU, no-asm)` matrix entry (`--cross-file=build-aux/i686-linux-gnu.ini - -Denable_asm=false`) is no longer present. The fork targets 64-bit x86-64 - and ARM64 exclusively. (ADR-0691, VMAFX Phase 1C) diff --git a/changelog.d/removed/native-build-sunset.md b/changelog.d/removed/native-build-sunset.md deleted file mode 100644 index e6542330f..000000000 --- a/changelog.d/removed/native-build-sunset.md +++ /dev/null @@ -1,30 +0,0 @@ -# BREAKING: Sunset legacy native build modes (ADR-0728) - -The following CI build configurations have been removed from -`.github/workflows/libvmaf-build-matrix.yml` and the required-checks list: - -**Removed from CI matrix:** -- `Build — Windows MinGW64 (CPU)` — MinGW64 is not a VMAFX production target -- `Build — Ubuntu i686 gcc (CPU, no-asm)` — fork is 64-bit only -- `Build — Ubuntu gcc (CPU) + DNN` — superseded by Linux full-build in build.yml -- `Build — Ubuntu clang (CPU) + DNN` — superseded by Linux full-build in build.yml -- `Build — macOS clang (CPU) + DNN` — superseded by macOS leg in build.yml -- `Build — Ubuntu Vulkan (T5-1b runtime)` — folded into Linux full-build -- `Build — macOS Vulkan via MoltenVK (advisory)` — too fragile; no required gate -- `Build — Ubuntu HIP (T7-10b runtime)` — folded into Linux full-build -- `Build — macOS Metal (T8-1 scaffold)` — folded into macOS leg in build.yml -- `Build — Ubuntu gcc Static (CPU)` — pkgconfig verified within Linux full-build -- `Build — Ubuntu CUDA Static` — NVCC-static covered by Linux full-build -- `Build — Ubuntu SYCL` — folded into Linux full-build -- `Build — Ubuntu SYCL + CUDA` — folded into Linux full-build -- `Build — Windows MSVC + oneAPI SYCL (build only)` — SYCL in Linux full-build - -**New canonical build matrix** (`build.yml`, ADR-0710): -- `Build — Linux (GCC, all backends)` — full stack: CUDA + SYCL + Vulkan + HIP + DNN + CPU -- `Build — macOS (Clang, CPU + Metal)` — Apple Clang + CPU + Metal scaffold -- `Build — Windows (MSVC + CUDA)` — MSVC + CPU + CUDA (build-only) - -**Required-checks aggregator updated** to use new check names from `build.yml` -and `sanitizers.yml`. `Cppcheck (Whole Project)` removed (clang-tidy superset). - -Implements ADR-0691 + ADR-0710. No functional change to the C library or CLI. diff --git a/core/src/hip/AGENTS.md b/core/src/hip/AGENTS.md index e4ad47860..0c82d1a44 100644 --- a/core/src/hip/AGENTS.md +++ b/core/src/hip/AGENTS.md @@ -34,8 +34,8 @@ Two additional consumers promoted from scaffold to real kernels in ADR-0372 (batch-1, this PR). 1. **T7-10 audit-first scaffold** (ADR-0212) — common, picture, dispatch, - feature stubs, public header `libvmaf_hip.h`, CI lane - `Build — Ubuntu HIP (T7-10 scaffold)`, smoke-only `enable_hip` build. + feature stubs, public header `libvmaf_hip.h`, CI lane (now `Ubuntu HIP`, + a required check), smoke-only `enable_hip` build. Every public C-API entry point returns `-ENOSYS`. 2. **T7-10 first consumer** (ADR-0241) — `kernel_template.{h,c}` (mirror of `cuda/kernel_template.h`) + `feature/hip/integer_psnr_hip.{c,h}` @@ -406,9 +406,10 @@ meson setup build_full -Denable_hip=true -Denable_hipcc=true \ ninja -C build_full ``` -CI lane `Build — Ubuntu HIP (T7-10 scaffold)` uses -`-Denable_hipcc=false` so it runs without ROCm SDK. Kernel-enabled -builds (`-Denable_hipcc=true`) require `hipcc` in `PATH` and ROCm 6+. +The required CI lane `Ubuntu HIP` leaves `enable_hipcc` at its default +(`false`): it installs ROCm 10.0.0 from a digest-pinned image (ADR-1225) and +builds the host side without device kernels. Kernel-enabled builds +(`-Denable_hipcc=true`) require `hipcc` in `PATH` and ROCm 7.0+. ## The HIP backend is host-pic — stage before you launch (ADR-1211) diff --git a/docs/backends/metal/index.md b/docs/backends/metal/index.md index 02b23c7c2..508c0207b 100644 --- a/docs/backends/metal/index.md +++ b/docs/backends/metal/index.md @@ -224,11 +224,14 @@ available, CPU SIMD wins otherwise. ## Verification -The macOS CI lane `Build — macOS Metal` is the ground-truth gate; it -runs on every PR with `-Denable_metal=enabled` and exercises the smoke -test plus the currently wired kernel batch. Linux-host dev sessions -cannot reproduce the lane locally because `Metal.framework` only exists -on macOS hosts. +The macOS CI lane `macOS Metal` in `libvmaf-build-matrix.yml` is where the +backend is verified; it runs on every non-draft PR that touches the C core, +with `-Denable_metal=enabled`, and exercises the smoke test plus the currently +wired kernel batch. `build.yml`'s `macOS Clang+Metal` row also builds and tests +the Metal backend. Neither is a required check, so a red Metal run does not +block a merge ([ADR-1259](../../adr/1259-ci-build-matrix-as-it-runs.md)); check +it before merging Metal changes. Linux-host dev sessions cannot reproduce the +lane locally because `Metal.framework` only exists on macOS hosts. Reviewers verifying locally on a Mac: diff --git a/docs/development/ci-job-names.md b/docs/development/ci-job-names.md index 9d7c22f44..2fb4205d6 100644 --- a/docs/development/ci-job-names.md +++ b/docs/development/ci-job-names.md @@ -55,7 +55,15 @@ means re-pointing every badge. Branch protection targets a single context: `Required Checks Aggregator` in [`.github/workflows/required-aggregator.yml`](../../.github/workflows/required-aggregator.yml). -The aggregator monitors 34 required checks defined across five workflows. +The aggregator's `required` list holds 40 check names defined across eleven +workflows; each run evaluates 39 of them, because only the Scorecard gate for +the triggering event applies. + +A required name must be reported by exactly one job, because the aggregator +keeps one check run per name. `Windows MSVC+CUDA` currently breaks that: the +required `libvmaf-build-matrix.yml` lane and the `build.yml` row share it +(`T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in [state.md](../state.md); +[ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md)). To prevent drift between workflow job definitions and the aggregator's required check array, all required checks are tagged in their defining workflow with @@ -69,77 +77,98 @@ The gate is wired into: ## Complete mapping table +"Previous Name" is the display name before PR #1286 (`f93a0037f`) shortened +it; "unchanged" means that PR did not rename the job. The table covers every +job the PR renamed, every required check and every build lane. + | Workflow | Previous Name | Shortened Name | Length | Required | | --- | --- | --- | --- | --- | -| `libvmaf-build-matrix.yml` | `Linux CI (GCC + OpenVINO DNN)` | `Ubuntu gcc+DNN` | 14 | Yes | -| `libvmaf-build-matrix.yml` | `Linux CI (Clang + OpenVINO DNN)` | `Ubuntu clang+DNN` | 16 | Yes | -| `libvmaf-build-matrix.yml` | `Linux CI (AMD ROCm / HIP)` | `Ubuntu HIP` | 10 | Yes | -| `libvmaf-build-matrix.yml` | `Windows CI (MinGW-w64 GCC)` | `Windows MinGW64` | 15 | Yes | -| `libvmaf-build-matrix.yml` | `Windows CI (MSVC + CUDA 13)` | `Windows MSVC+CUDA` | 18 | Yes | -| `libvmaf-build-matrix.yml` | `Windows CI (MSVC + Intel oneAPI SYCL)` | `Windows MSVC+SYCL` | 18 | Yes | -| `lint-and-format.yml` | `Pre-Commit All-Files Hygiene Gate` | `Pre-Commit` | 10 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu gcc (CPU) + DNN` | `Ubuntu gcc+DNN` | 14 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu clang (CPU) + DNN` | `Ubuntu clang+DNN` | 16 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu HIP (T7-10b runtime)` | `Ubuntu HIP` | 10 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Windows MinGW64 (CPU)` | `Windows MinGW64` | 15 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Windows MSVC + CUDA (build only)` | `Windows MSVC+CUDA` | 18 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Windows MSVC + oneAPI SYCL (build only)` | `Windows MSVC+SYCL` | 18 | Yes | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu gcc (CPU)` | `Ubuntu gcc` | 10 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu clang (CPU)` | `Ubuntu clang` | 12 | No | +| `libvmaf-build-matrix.yml` | `Build — macOS clang (CPU)` | `macOS clang` | 11 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu ARM clang (CPU)` | `Ubuntu ARM clang` | 16 | No | +| `libvmaf-build-matrix.yml` | `Build — macOS clang (CPU) + DNN` | `macOS clang+DNN` | 15 | No | +| `libvmaf-build-matrix.yml` | `Build — macOS Metal (T8-1 scaffold)` | `macOS Metal` | 11 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu gcc Static (CPU)` | `Ubuntu gcc static` | 17 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu CUDA Static` | `Ubuntu CUDA static` | 18 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu SYCL` | `Ubuntu SYCL` | 11 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu CUDA` | `Ubuntu CUDA` | 11 | No | +| `libvmaf-build-matrix.yml` | `Build — Ubuntu SYCL + CUDA` | `Ubuntu SYCL+CUDA` | 16 | No | +| `lint-and-format.yml` | `Pre-Commit (Formatters + Basic Checks)` | `Pre-Commit` | 10 | Yes | | `lint-and-format.yml` | `Clang-Tidy (Changed C/C++ Files)` | `Tidy Changed` | 12 | Yes | | `lint-and-format.yml` | `Clang-Tidy Ratchet (Whole Tree)` | `Tidy Ratchet` | 12 | Yes | -| `lint-and-format.yml` | `C/C++ Static Analysis (cppcheck)` | `Cppcheck` | 8 | Yes | -| `lint-and-format.yml` | `Python Linters (black + ruff + mypy)` | `Python Lint` | 11 | Yes | -| `lint-and-format.yml` | `Docs Build (Doxygen + Sphinx)` | `Docs` | 4 | Yes | -| `lint-and-format.yml` | `Twin Drift (C/C++ Source Parity Gate)` | `Twin Drift` | 10 | Yes | -| `lint-and-format.yml` | `Shell Scripts (ShellCheck + shfmt)` | `ShellCheck + shfmt` | 17 | Yes | -| `lint-and-format.yml` | `Clang-Tidy (Full SYCL Codebase — Advisory)` | `Tidy SYCL (advisory)` | 20 | No | -| `lint-and-format.yml` | `No Conflict Markers Staged / Committed` | `No Conflict Markers` | 19 | No | -| `lint-and-format.yml` | `Markdown Lint (ADR-0866, changed files)` | `Markdown Lint` | 13 | No | -| `rule-enforcement.yml` | `Deep-Dive Deliverables Checklist Gate` | `Deliverables Checklist` | 22 | Yes | -| `rule-enforcement.yml` | `Doc-Substance Gate (ADR-0100 / ADR-0042)` | `Doc-Substance Gate` | 18 | Yes | -| `rule-enforcement.yml` | `docs/state.md Freshness Gate` | `docs/state.md Gate` | 18 | Yes | -| `rule-enforcement.yml` | `FFmpeg-Patches libvmaf Surface Parity Gate` | `FFmpeg-Patches Surface Sync` | 27 | Yes | +| `lint-and-format.yml` | `Cppcheck (Whole Project)` | `Cppcheck` | 8 | Yes | +| `lint-and-format.yml` | `Python Lint (Ruff + Black + mypy)` | `Python Lint` | 11 | Yes | +| `lint-and-format.yml` | unchanged | `Docs` | 4 | Yes | +| `lint-and-format.yml` | `Twin Drift + Stale Source Refs (ADR-1135)` | `Twin Drift` | 10 | Yes | +| `lint-and-format.yml` | `ShellCheck + shfmt (All *.sh)` | `ShellCheck + shfmt` | 17 | Yes | +| `lint-and-format.yml` | `Clang-Tidy SYCL (Changed Files, Advisory)` | `Tidy SYCL (advisory)` | 20 | No | +| `lint-and-format.yml` | `Check — No committed conflict markers` | `No Conflict Markers` | 19 | No | +| `lint-and-format.yml` | `Markdown lint (markdownlint-cli2)` | `Markdown Lint` | 13 | No | +| `rule-enforcement.yml` | `Deep-Dive Deliverables Checklist (ADR-0108)` | `Deliverables Checklist` | 22 | Yes | +| `rule-enforcement.yml` | `Doc-Substance Gate (ADR-0100 / 0167)` | `Doc-Substance Gate` | 18 | Yes | +| `rule-enforcement.yml` | `docs/state.md Touch Gate (ADR-0165)` | `docs/state.md Gate` | 18 | Yes | +| `rule-enforcement.yml` | `FFmpeg-Patches Surface Sync (CLAUDE.md §12 r14, ADR-0356)` | `FFmpeg-Patches Surface Sync` | 27 | Yes | | `rule-enforcement.yml` | `ADR Number Collision Guard (ADR-0386 / ADR-0628)` | `ADR Collision Guard` | 19 | Yes | -| `rule-enforcement.yml` | `Release Script Contract Tests` | `Release Script Contract` | 23 | Yes | -| `rule-enforcement.yml` | `ADR-Backfill Scope Check (Advisory)` | `ADR-Backfill Advisory` | 21 | No | -| `security-scans.yml` | `Semgrep Code Quality & Security Scan` | `Semgrep` | 7 | Yes | -| `security-scans.yml` | `CodeQL (C/C++ Analysis)` | `CodeQL (C/C++)` | 15 | Yes | -| `security-scans.yml` | `CodeQL (Python Analysis)` | `CodeQL (Python)` | 15 | Yes | -| `security-scans.yml` | `CodeQL (GitHub Actions Analysis)` | `CodeQL (Actions)` | 16 | Yes | -| `security-scans.yml` | `CodeQL (Unified Analysis Status)` | `CodeQL` | 6 | Yes | -| `security-scans.yml` | `Gitleaks Secrets Scan` | `Gitleaks` | 8 | Yes | -| `security-scans.yml` | `Dependency Review (PR Dependency Changes)` | `Dependency Review` | 17 | Yes | -| `tests-and-quality-gates.yml` | `Netflix Golden Data (x86_64 CPU)` | `Netflix CPU Golden` | 18 | Yes | -| `tests-and-quality-gates.yml` | `Sanitizers (AddressSanitizer + LeakSanitizer)` | `Sanitizers (address)` | 20 | Yes | -| `tests-and-quality-gates.yml` | `Sanitizers (ThreadSanitizer)` | `Sanitizers (thread)` | 19 | Yes | -| `tests-and-quality-gates.yml` | `Sanitizers (UndefinedBehaviorSanitizer)` | `Sanitizers (undefined)` | 22 | Yes | -| `tests-and-quality-gates.yml` | `Tiny AI Model Artifacts & Architecture Tests` | `Tiny AI` | 7 | Yes | -| `tests-and-quality-gates.yml` | `SYCL float_ssim Parity Gate` | `SYCL float_ssim Parity` | 23 | Yes | -| `tests-and-quality-gates.yml` | `Assertion Density (CERT C ENV33-C / JPL Rule 5)` | `Assertion Density` | 17 | Yes | -| `tests-and-quality-gates.yml` | `Coverage Gate (C Engine + Python Core)` | `Coverage Gate` | 13 | No | -| `tests-and-quality-gates.yml` | `Cross-Backend Parity (Floating-Point ULP Gate)` | `Cross-Backend ULP Diff` | 22 | No | -| `tests-and-quality-gates.yml` | `Coverage (C Engine GPU Twins — Advisory)` | `Coverage GPU (advisory)` | 23 | No | -| `tests-and-quality-gates.yml` | `MCP Server Smoke Test (stdio / JSON-RPC)` | `MCP Smoke` | 9 | No | -| `build.yml` | `Linux Intel LLVM / SYCL (Arc GPU / Level Zero)` | `Linux Intel LLVM` | 16 | No | -| `build.yml` | `macOS Clang (Metal GPU)` | `macOS Clang+Metal` | 17 | No | -| `build.yml` | `Windows MSVC (CUDA 13 GPU)` | `Windows MSVC+CUDA` | 18 | No | -| `ffmpeg-integration.yml` | `FFmpeg Integration (Ubuntu gcc)` | `FFmpeg Ubuntu gcc` | 17 | No | -| `ffmpeg-integration.yml` | `FFmpeg Integration (macOS clang)` | `FFmpeg macOS clang` | 18 | No | -| `ffmpeg-integration.yml` | `FFmpeg Integration (Ubuntu icpx SYCL)` | `FFmpeg SYCL` | 11 | No | -| `sanitizers.yml` | `Sanitizers (AddressSanitizer + UndefinedBehaviorSanitizer)` | `Sanitizers ASan+UBSan` | 20 | No | -| `sanitizers.yml` | `Sanitizers (ThreadSanitizer Data Race Detection)` | `Sanitizers TSan` | 15 | No | -| `sanitizers.yml` | `Fuzz Target Regression Suite (${{ matrix.target }})` | `Fuzz ${{ matrix.target }}` | <=25 | No | -| `rust-ci.yml` | `vmafx-sys Rust FFI Bindings & Integration Tests` | `vmafx-sys CI` | 12 | No | -| `rust-ci.yml` | `Cargo Deny (Licenses + Advisories + Bans)` | `cargo-deny` | 10 | No | -| `supply-chain.yml` | `Validate release version metadata` | `Validate release versions` | 25 | No | -| `supply-chain.yml` | `Build multi-platform release artifacts` | `Build Linux artifacts` | 21 | No | -| `supply-chain.yml` | `Verify release binary runtime compatibility` | `Verify Linux runtime` | 20 | No | -| `supply-chain.yml` | `Generate CycloneDX SBOMs and hashes` | `Generate SBOMs` | 14 | No | -| `supply-chain.yml` | `Sigstore keyless signing of release artifacts` | `Sigstore sign artifacts` | 23 | No | -| `supply-chain.yml` | `Generate SLSA provenance for libvmaf artifacts` | `SLSA libvmaf` | 12 | No | -| `supply-chain.yml` | `Sigstore keyless signing of vmaf-mcp wheels` | `Sigstore sign vmaf-mcp` | 22 | No | -| `supply-chain.yml` | `Generate SLSA provenance for vmaf-mcp wheel` | `SLSA vmaf-mcp` | 12 | No | -| `supply-chain.yml` | `Publish vmaf-mcp to PyPI via Trusted Publishing` | `Publish vmaf-mcp PyPI` | 20 | No | -| `supply-chain.yml` | `Attach SLSA provenance and SBOM to GitHub Release` | `Attach release assets` | 21 | No | +| `rule-enforcement.yml` | `Release Script Contract (ADR-1128)` | `Release Script Contract` | 23 | Yes | +| `rule-enforcement.yml` | `ADR-Backfill Advisory (ADR-0106)` | `ADR-Backfill Advisory` | 21 | No | +| `security-scans.yml` | `Semgrep (CWE Top 25 + CERT-C + Custom)` | `Semgrep` | 7 | Yes | +| `security-scans.yml` | unchanged | `CodeQL (C/C++)` | 15 | Yes | +| `security-scans.yml` | unchanged | `CodeQL (Python)` | 15 | Yes | +| `security-scans.yml` | unchanged | `CodeQL (Actions)` | 16 | Yes | +| `security-scans.yml` | unchanged | `CodeQL` | 6 | Yes | +| `security-scans.yml` | `Gitleaks (Secret Scan)` | `Gitleaks` | 8 | Yes | +| `security-scans.yml` | `Dependency Review (PR Diff)` | `Dependency Review` | 17 | Yes | +| `tests-and-quality-gates.yml` | `Netflix CPU Golden Tests (D24)` | `Netflix CPU Golden` | 18 | Yes | +| `tests-and-quality-gates.yml` | `Sanitizers — ASan + UBSan + MSan (address)` | `Sanitizers (address)` | 20 | Yes | +| `tests-and-quality-gates.yml` | `Sanitizers — ASan + UBSan + MSan (thread)` | `Sanitizers (thread)` | 19 | Yes | +| `tests-and-quality-gates.yml` | `Sanitizers — ASan + UBSan + MSan (undefined)` | `Sanitizers (undefined)` | 22 | Yes | +| `tests-and-quality-gates.yml` | `Tiny AI (DNN Suite + ai/ Pytests)` | `Tiny AI` | 7 | Yes | +| `tests-and-quality-gates.yml` | `SYCL float_ssim Parity (Arc DG2-G10)` | `SYCL float_ssim Parity` | 23 | Yes | +| `tests-and-quality-gates.yml` | `Assertion Density (Power of 10 §5)` | `Assertion Density` | 17 | Yes | +| `tests-and-quality-gates.yml` | `Coverage Gate (Ramping to 70% / 85% Critical)` | `Coverage Gate` | 13 | No | +| `tests-and-quality-gates.yml` | `Cross-Backend ULP Diff (CPU Sanity)` | `Cross-Backend ULP Diff` | 22 | No | +| `tests-and-quality-gates.yml` | `Coverage Gate — GPU Backends (Advisory)` | `Coverage GPU (advisory)` | 23 | No | +| `tests-and-quality-gates.yml` | `MCP Smoke (Embedded C + Python Server)` | `MCP Smoke` | 9 | No | +| `go-ci.yml` | unchanged | `go vet + go test` | 16 | Yes | +| `scorecard-policy.yml` | added after #1286 | `Scorecard PR Gate` | 17 | Yes | +| `scorecard.yml` | added after #1286 | `Scorecard Master Gate` | 21 | Yes | +| `sycl-parity.yml` | added after #1286 | `SYCL Parity (Arc A380)` | 22 | Yes | +| `ffmpeg-patch-stack.yml` | added after #1286 | `FFmpeg Patch Stack` | 18 | Yes | +| `standards-gate.yml` | added after #1286 | `Standards & Invariant Verification Gate` | 39 | Yes | +| `build.yml` | `Build — Linux (Intel LLVM, all backends)` | `Linux Intel LLVM` | 16 | No | +| `build.yml` | `Build — macOS (Clang, CPU + Metal)` | `macOS Clang+Metal` | 17 | No | +| `build.yml` | `Build — Windows (MSVC + CUDA)` | `Windows MSVC+CUDA` | 18 | No | +| `ffmpeg-integration.yml` | `FFmpeg — Ubuntu gcc (Build Only)` | `FFmpeg Ubuntu gcc` | 17 | No | +| `ffmpeg-integration.yml` | `FFmpeg — macOS clang (Build Only)` | `FFmpeg macOS clang` | 18 | No | +| `ffmpeg-integration.yml` | `FFmpeg — SYCL (Build Only)` | `FFmpeg SYCL` | 11 | No | +| `sanitizers.yml` | `Sanitizers — ASan + UBSan (PR gate)` | `Sanitizers ASan+UBSan` | 20 | No | +| `sanitizers.yml` | `Sanitizers — TSan (master push)` | `Sanitizers TSan` | 15 | No | +| `sanitizers.yml` | `Fuzz — ${{ matrix.target }} (nightly)` | `Fuzz ${{ matrix.target }}` | <=25 | No | +| `rust-ci.yml` | `vmafx-sys (fmt + clippy + test)` | `vmafx-sys CI` | 12 | No | +| `rust-ci.yml` | `cargo-deny (licenses, bans, advisories, sources)` | `cargo-deny` | 10 | No | +| `supply-chain.yml` | `Validate ordinary tag and coordinated versions` | `Validate release versions` | 25 | No | +| `supply-chain.yml` | `Build Linux release artifacts (libvmaf.so chain, vmaf CLI, models)` | `Build Linux artifacts` | 21 | No | +| `supply-chain.yml` | `Verify downloaded Linux release runtime` | `Verify Linux runtime` | 20 | No | +| `supply-chain.yml` | `Generate libvmaf + vmaf-mcp SBOMs (SPDX + CycloneDX)` | `Generate SBOMs` | 14 | No | +| `supply-chain.yml` | `Sigstore keyless sign (release artifacts + SBOMs)` | `Sigstore sign artifacts` | 23 | No | +| `supply-chain.yml` | `SLSA L3 provenance — libvmaf artifacts` | `SLSA libvmaf` | 12 | No | +| `supply-chain.yml` | `Sigstore keyless sign — vmaf-mcp` | `Sigstore sign vmaf-mcp` | 22 | No | +| `supply-chain.yml` | `SLSA L3 provenance — vmaf-mcp distributions` | `SLSA vmaf-mcp` | 12 | No | +| `supply-chain.yml` | `Publish vmaf-mcp to PyPI (Trusted Publishing)` | `Publish vmaf-mcp PyPI` | 20 | No | +| `supply-chain.yml` | `Attach SBOM + signatures to GitHub Release` | `Attach release assets` | 21 | No | | `docker-publish-operator-node.yml` | `Validate published ordinary tag` | `Validate tag` | 12 | No | -| `docker-publish-operator-node.yml` | `Build + push vmafx-operator image (amd64 + arm64)` | `Publish vmafx-operator` | 22 | No | -| `docker-publish-operator-node.yml` | `Build + push vmafx-server image (amd64 + arm64)` | `Publish vmafx-server` | 20 | No | -| `docker-publish-operator-node.yml` | `Build + push vmafx-node-cpu image (amd64 + arm64)` | `Publish vmafx-node CPU` | 21 | No | -| `docker-publish-operator-node.yml` | `Smoke-test operator/node image entrypoints` | `Smoke-test images` | 17 | No | -| `docker-publish-operator-node.yml` | `All operator/node images published` | `Images published` | 16 | No | +| `docker-publish-operator-node.yml` | `Build + push vmafx-operator (amd64 + arm64)` | `Publish vmafx-operator` | 22 | No | +| `docker-publish-operator-node.yml` | `Build + push vmafx-server (amd64 + arm64)` | `Publish vmafx-server` | 20 | No | +| `docker-publish-operator-node.yml` | `Build + push vmafx-node CPU (amd64 + arm64)` | `Publish vmafx-node CPU` | 21 | No | +| `docker-publish-operator-node.yml` | `Smoke-test operator + server + node images` | `Smoke-test images` | 17 | No | +| `docker-publish-operator-node.yml` | `All Go service images published` | `Images published` | 16 | No | | `docker-publish-production.yml` | `Validate published ordinary tag` | `Validate tag` | 12 | No | | `docker-publish-production.yml` | `Build + push CPU image (amd64 + arm64)` | `Publish CPU image` | 17 | No | | `docker-publish-production.yml` | `Build + push CUDA 13 image (amd64)` | `Publish CUDA 13 image` | 21 | No | diff --git a/docs/development/ci.md b/docs/development/ci.md index ee7d1c014..9936569c3 100644 --- a/docs/development/ci.md +++ b/docs/development/ci.md @@ -18,13 +18,17 @@ The main `pull_request`-triggered workflows include: | [`required-aggregator.yml`](../../.github/workflows/required-aggregator.yml) | Single required-check aggregator (ADR-0313). | | [`go-ci.yml`](../../.github/workflows/go-ci.yml) | Required Go vet, security scan, runner smoke, and tests (ADR-1238). | | [`ffmpeg-integration.yml`](../../.github/workflows/ffmpeg-integration.yml) | FFmpeg + libvmaf build (Linux GCC / macOS Clang / SYCL). | -| [`libvmaf-build-matrix.yml`](../../.github/workflows/libvmaf-build-matrix.yml) | Cross-platform / cross-backend libvmaf build matrix. | +| [`libvmaf-build-matrix.yml`](../../.github/workflows/libvmaf-build-matrix.yml) | Cross-platform / cross-backend libvmaf build matrix: 17 lanes, six of them required. | +| [`build.yml`](../../.github/workflows/build.yml) | One all-backend build per OS (`Linux Intel LLVM`, `macOS Clang+Metal`, `Windows MSVC+CUDA`), alongside the matrix; not required. | | [`rule-enforcement.yml`](../../.github/workflows/rule-enforcement.yml) | ADR-0100 / 0106 / 0108 / 0165 process gates. | | [`tests-and-quality-gates.yml`](../../.github/workflows/tests-and-quality-gates.yml) | Netflix golden, sanitizers, tiny-AI, MCP, coverage, assertion-density. | +| [`sanitizers.yml`](../../.github/workflows/sanitizers.yml) | Combined ASan+UBSan on PRs, TSan on master pushes, nightly fuzzing; not required (the required sanitizers are in `tests-and-quality-gates.yml`). | | [`sycl-parity.yml`](../../.github/workflows/sycl-parity.yml) | SYCL parity tests on self-hosted Intel Arc A380 runner (ADR-1177; see [runbook](ci-self-hosted-sycl.md)). | For the complete inventory, mapping of shortened names, and conventions, -see [CI job display names](ci-job-names.md). +see [CI job display names](ci-job-names.md). For every build lane, whether it +is required and which ADR owns it, see +[ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md). ## Draft pull requests defer heavy CI diff --git a/docs/development/deprecations.md b/docs/development/deprecations.md index 5380422c2..1cc1f5843 100644 --- a/docs/development/deprecations.md +++ b/docs/development/deprecations.md @@ -6,6 +6,25 @@ deprecated or removed. Entries are ordered newest-first. --- +## 2026-09-18 — 32-bit x86 CI lane (ADR-1258) + +**Status**: Removed + +The `Ubuntu i686 gcc` lane (formerly `Build — Ubuntu i686 gcc (CPU, no-asm)`), +a compile-only 32-bit cross build with asm disabled, and the `m32` stage of +`scripts/dev/preflight.sh` that mirrored it are gone. ADR-0691 had removed the +lane in May, but a merge restored it the same day, and it never ran a test. + +**Migration**: none for users of the published containers or 64-bit builds. +32-bit x86 is unsupported; the fork is 64-bit only. If you build for 32-bit +x86 yourself, add `-msse2 -mfpmath=sse`: without it the x87 FPU's 80-bit +intermediates change scalar scores. + +**References**: [ADR-1258](../adr/1258-keep-64-bit-only-retire-i686-lane.md), +[ADR-0691](../adr/0691-vmafx-drop-legacy-build-paths.md) + +--- + ## 2026-05-28 — `ansnr` / `float_ansnr` feature extractor (ADR-0865) **Status**: Removed @@ -51,43 +70,31 @@ PR #87 --- -## 2026-05-28 — Legacy native build modes (ADR-0728) +## 2026-05-28 — Legacy native build modes (ADR-0728): not carried out -**Status**: Removed +**Status**: Withdrawn. [ADR-0728](../adr/0728-native-build-sunset.md) is +superseded by [ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md). + +This entry used to list 16 CI configurations as removed. None of them was +removed by ADR-0728: its commit changed only this page, the ADR and a changelog +fragment. Of the configurations it listed: + +- `Build — Ubuntu i686 gcc (CPU, no-asm)` was removed on 2026-09-18 (see the + entry above). +- `Build — Ubuntu Vulkan (T5-1b runtime)` and + `Build — macOS Vulkan via MoltenVK (advisory)` went with the Vulkan backend + ([ADR-0726](../adr/0726-drop-vulkan-backend.md)). +- Everything else still runs, under the short names it has had since #1286. + Required: `Windows MinGW64`, `Ubuntu gcc+DNN`, `Ubuntu clang+DNN`, + `Ubuntu HIP`, `Windows MSVC+SYCL`, `Cppcheck`, `Sanitizers (address)`, + `Sanitizers (thread)` and `Sanitizers (undefined)`. Not required: + `macOS clang+DNN`, `macOS Metal`, `Ubuntu gcc static`, `Ubuntu CUDA static`, + `Ubuntu SYCL` and `Ubuntu SYCL+CUDA`. -**What was removed**: The following CI build configurations no longer run as -required or advisory CI lanes: - -| Removed configuration | Last active | -|---|---| -| `Build — Windows MinGW64 (CPU)` (MSYS2 / MinGW-w64) | pre-ADR-0728 | -| `Build — Ubuntu i686 gcc (CPU, no-asm)` (32-bit x86 cross-build) | pre-ADR-0728 | -| `Build — Ubuntu gcc (CPU) + DNN` | pre-ADR-0728 | -| `Build — Ubuntu clang (CPU) + DNN` | pre-ADR-0728 | -| `Build — macOS clang (CPU) + DNN` | pre-ADR-0728 | -| `Build — Ubuntu Vulkan (T5-1b runtime)` | pre-ADR-0728 | -| `Build — macOS Vulkan via MoltenVK (advisory)` | pre-ADR-0728 | -| `Build — Ubuntu HIP (T7-10b runtime)` | pre-ADR-0728 | -| `Build — macOS Metal (T8-1 scaffold)` | pre-ADR-0728 | -| `Build — Ubuntu gcc Static (CPU)` | pre-ADR-0728 | -| `Build — Ubuntu CUDA Static` | pre-ADR-0728 | -| `Build — Ubuntu SYCL` | pre-ADR-0728 | -| `Build — Ubuntu SYCL + CUDA` | pre-ADR-0728 | -| `Build — Windows MSVC + oneAPI SYCL (build only)` | pre-ADR-0728 | -| `Cppcheck (Whole Project)` (required-check) | pre-ADR-0728 | -| `Sanitizers — ASan + UBSan + MSan (address/thread/undefined)` (3 separate required-checks) | pre-ADR-0728 | - -**Why**: Container-first VMAFX posture (ADR-0686, ADR-0701). Docker images and -the Helm chart are the user-facing artifacts; per-backend host binaries are no -longer published. The canonical build matrix is now `build.yml` (ADR-0710): -one job per OS (Linux GCC all-backends, macOS Clang, Windows MSVC+CUDA). - -**Migration**: No migration needed for end users — the C library API and CLI -flags are unchanged. Contributors who previously tested against MinGW64 should -use the Windows MSVC + CUDA path or the `vmafx-dev-mcp` Docker container. -32-bit x86 is unsupported; the fork is 64-bit only. - -**References**: ADR-0691, ADR-0710, ADR-0728 +`build.yml` (Linux Intel LLVM, macOS Clang+Metal, Windows MSVC+CUDA) runs +alongside `libvmaf-build-matrix.yml`; it did not replace it. A change that +breaks `Windows MinGW64` blocks the merge like any other required check. +ADR-1259 lists every lane and which checks are required. --- @@ -100,7 +107,8 @@ never published as a standalone wheel to PyPI. No CI job performed wheel publication for this package at the time of audit. ADR-0691 §4 records this as a no-op for traceability. -The Netflix Python harness (`python/test/`) continues to run via `tox` in the -Linux full-build leg. +The Netflix Python harness (`python/test/`) continues to run via `tox` in +every Linux lane without a GPU backend and every macOS lane, and the required +`Netflix CPU Golden` job runs the golden assertions. --- diff --git a/docs/development/release.md b/docs/development/release.md index b26e32254..e6da0c9f4 100644 --- a/docs/development/release.md +++ b/docs/development/release.md @@ -591,19 +591,30 @@ is enforced at the host, not just honored by convention. - **Required status check (1):** `Required Checks Aggregator`. Branch protection names exactly this one context; every other gate is enforced *through* it. The aggregator's own `required` array - (`.github/workflows/required-aggregator.yml`) is the real inventory — **34 - entries** as of ADR-1151: + (`.github/workflows/required-aggregator.yml`) is the real inventory — **40 + entries**, of which each run checks 39 (only the Scorecard gate for its own + event applies): - **Builds (7):** Ubuntu gcc+DNN, Ubuntu clang+DNN, Windows MinGW64, Windows MSVC+CUDA, Windows MSVC+SYCL, - Ubuntu HIP, SYCL float_ssim Parity. + Ubuntu HIP, SYCL float_ssim Parity. All but the last are + `libvmaf-build-matrix.yml` lanes; no `build.yml` row is required + ([ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md)). The `build.yml` + Windows row also reports as `Windows MSVC+CUDA`, and the aggregator keeps + only the later of the two runs (`T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` + in [state.md](../state.md)). - **Static analysis (10):** CodeQL ×4 (CodeQL, CodeQL (C/C++), CodeQL (Python), CodeQL (Actions)), Pre-Commit, Python Lint, Semgrep, Tidy Changed, Tidy Ratchet, Cppcheck. - - **Supply chain / docs (4):** Dependency Review, Gitleaks, Docs, - ShellCheck + shfmt. - - **Tests (7):** Netflix CPU Golden, Sanitizers ×3 (Sanitizers (address), + - **Supply chain / docs (6):** Dependency Review, Gitleaks, Docs, + ShellCheck + shfmt, Scorecard PR Gate (pull requests), Scorecard Master + Gate (master pushes). + - **Tests (9):** Netflix CPU Golden, Sanitizers ×3 (Sanitizers (address), Sanitizers (thread), Sanitizers (undefined)), Assertion Density, Twin Drift, - Tiny AI. + Tiny AI, go vet + go test, SYCL Parity (Arc A380) (enforced only while the + `SYCL_ARC_RUNNER_ENABLED` variable is `true`). + - **FFmpeg (1):** FFmpeg Patch Stack. + - **Governance (1):** Standards & Invariant Verification Gate + ([ADR-1249](../adr/1249-praetor-governance-adoption.md)). - **Process gates (6):** Deliverables Checklist, Doc-Substance Gate, docs/state.md Gate, FFmpeg-Patches Surface Sync, ADR Collision Guard, Release Script Contract. These report on every non-draft PR; four of the @@ -620,7 +631,9 @@ is enforced at the host, not just honored by convention. - **Not required (non-blocking signals):** Coverage gate (~40 min — built with `-fprofile-update=atomic` since 2026-04-18 to survive parallel-meson SIMD-counter races, see [ADR-0110](../adr/0110-coverage-gate-fprofile-update-atomic.md)), - GPU-advisory jobs, Semgrep OSS. + GPU-advisory jobs, the other eleven `libvmaf-build-matrix.yml` lanes, the + three `build.yml` rows and the `sanitizers.yml` jobs + ([ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md)). Management: `gh api --method PUT repos/VMAFx/vmafx/branches/master/protection` with a JSON payload. The current rule set is documented in diff --git a/docs/development/windows-d3d11-import.md b/docs/development/windows-d3d11-import.md index e69bfd1e8..8a4834721 100644 --- a/docs/development/windows-d3d11-import.md +++ b/docs/development/windows-d3d11-import.md @@ -1,9 +1,11 @@ # Validating `vmaf_sycl_import_d3d11_surface` in a local Windows VM This doc is a reproducer for the Windows-only surface-import path added in -ADR-0103. CI does not exercise it (the fork's Windows CI is MinGW + no-SYCL, -and the D3D11 API requires Intel oneAPI DPC++ on Windows). Validation -happens manually in a local Windows VM. +ADR-0103. CI compiles and links it but never runs it: the required +`Windows MSVC+SYCL` lane builds the SYCL backend with MSVC and Intel oneAPI, +including `d3d11_import.cpp` against `d3d11` and `dxgi`, and the Windows +runners have no GPU, so that lane runs no tests. Validation of the import +itself happens manually in a local Windows VM. ## When you need to run this @@ -12,8 +14,9 @@ happens manually in a local Windows VM. in a way that affects `vmaf_sycl_upload_plane` (the import path's sink). - You bumped the Intel oneAPI DPC++ toolkit version. -If none of the above, the Linux SYCL + Windows MinGW no-SYCL CI jobs are -enough. +If none of the above, CI is enough: `Windows MSVC+SYCL` proves the import +path still compiles and links, and the Linux SYCL lanes compile the shared +SYCL code. ## Prerequisites on the Windows VM diff --git a/docs/rebase-notes.md b/docs/rebase-notes.md index 24d265d02..36d557c13 100644 --- a/docs/rebase-notes.md +++ b/docs/rebase-notes.md @@ -42,6 +42,18 @@ See [Research-2062](research/2062-cambi-spatial-mask-simd.md). - `core/src/feature/x86/psnr_avx2.c`, `psnr_sse_line_16_avx2()`: the final 64-bit sum is read with `_mm_storel_epi64`, not `_mm_cvtsi128_si64`. +## ci/retire-i686-lane — the CI build matrix of record (ADR-1259) (2026-09-18) + +- `.github/workflows/libvmaf-build-matrix.yml` and `build.yml` both run; + `build.yml` does not replace the matrix. ADR-0689, ADR-0691, ADR-0710 and + ADR-0728 are superseded by ADR-1259. A merge resolution that drops or + restores a lane is wrong unless an ADR asks for it: `384d97d03` undid + ADR-0689 and ADR-0691 that way. +- The unreleased changelog fragments `changelog.d/removed/native-build-sunset.md`, + `changelog.d/removed/0691-vmafx-drop-legacy-build-paths.md` and + `changelog.d/changed/0689-vmafx-ci-matrix-dedupe.md` are deleted on purpose: + they announced removals that never happened. Do not restore them. + ## Canonical envtest installer (2026-09-08) Keep Make, Go CI and controller-suite guidance on `scripts/ci/setup-envtest.sh`. From 9720e9ab11a8165bb0fea329ebafca5ee437a608 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:26:16 +0200 Subject: [PATCH 10/15] ci: correct the build.yml and sanitizers.yml header comments build.yml's header said it supersedes libvmaf-build-matrix.yml and that the ARM, static and DNN legs were retired; sanitizers.yml's said it replaced the per-PR sanitizer matrix. Neither happened (ADR-1259): both workflows run alongside the ones they were meant to replace, and none of their jobs is a required check. The headers now say so, and build.yml's notes that its Windows row shares the required name Windows MSVC+CUDA. actionlint findings in sanitizers.yml are fixed on the way: the workflow_dispatch trigger now declares the max_total_time input that fuzz-nightly already reads (default 60, as before, mirroring fuzz.yml), and the -max_total_time argument is quoted. No job, trigger or required check changes. --- .github/workflows/build.yml | 23 +++++++++++++---------- .github/workflows/sanitizers.yml | 23 ++++++++++++++--------- 2 files changed, 27 insertions(+), 19 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 20ef1035c..42cc9546a 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -1,16 +1,19 @@ -# VMAFX build matrix — 1 job per OS (ADR-0710) +# VMAFX build matrix — 1 job per OS (ADR-0710, recorded in ADR-1259) # -# Supersedes libvmaf-build-matrix.yml (ADR-0689 dedup pass → ADR-0710 slim-down). -# Three matrix rows only: -# - linux-full: Intel LLVM + ALL Linux backends (CUDA + SYCL + HIP + CPU + DNN) -# - macos-cpu-metal: Apple Clang + CPU + Metal scaffold -# - windows-msvc-cuda: MSVC + CPU + CUDA +# Runs alongside libvmaf-build-matrix.yml, which keeps the per-backend, ARM, +# static, DNN and required lanes; ADR-0710 meant this file to replace it, but +# that never happened and ADR-1259 records both as they run. +# Three matrix rows, none of them a required check: +# - Linux Intel LLVM: Intel LLVM + ALL Linux backends (CUDA + SYCL + HIP + CPU + DNN) +# - macOS Clang+Metal: Apple Clang + CPU + Metal scaffold +# - Windows MSVC+CUDA: MSVC + CPU + CUDA. The required lane in +# libvmaf-build-matrix.yml has the same display name, and the aggregator +# keeps one run per name: see T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18 +# in docs/state.md. # # Rationale: container-first VMAFX philosophy (ADR-0686, ADR-0701) means the -# production artefact is the `vmafx-dev-mcp` image, not a per-backend binary. -# Per-backend build legs (SYCL-only, CUDA-only, HIP-only, Vulkan-only, etc.) -# are folded into the single Linux full build. ARM, i686, static-only, DNN- -# only, and MoltenVK advisory legs are retired. See ADR-0710. +# production artefact is the `vmafx-dev-mcp` image, not a per-backend binary; +# these rows prove the whole stack builds on each OS. # # Netflix golden assertions are pinned to the dedicated CPU/GCC jobs in # tests-and-quality-gates.yml and libvmaf-build-matrix.yml. The all-backend diff --git a/.github/workflows/sanitizers.yml b/.github/workflows/sanitizers.yml index e247ccb55..04eaf789a 100644 --- a/.github/workflows/sanitizers.yml +++ b/.github/workflows/sanitizers.yml @@ -10,11 +10,11 @@ # CFI is a future hardening step once the C23 bump (ADR-0692) and the # Rust FFI boundary stabilise. # -# Replaces the 3-way `matrix.sanitizer: [address, undefined, thread]` -# job in tests-and-quality-gates.yml and the separate fuzz.yml schedule. -# fuzz.yml is retained as the single source-of-truth for libFuzzer but -# its schedule block is kept there; this file adds the PR-gate sanitizer -# jobs that were previously embedded in tests-and-quality-gates.yml. +# ADR-0710 meant these jobs to replace the 3-way +# `matrix.sanitizer: [address, undefined, thread]` job in +# tests-and-quality-gates.yml. That never happened: the matrix still runs on +# every PR and its three `Sanitizers (...)` checks are required, while none of +# the jobs in this file is (ADR-1259). fuzz.yml keeps its own nightly schedule. name: Sanitizers @@ -28,6 +28,11 @@ on: # Fuzz nightly at 04:30 UTC. TSan fires on push to master (above). - cron: "30 4 * * *" workflow_dispatch: + inputs: + max_total_time: + description: "Per-target wall-clock seconds for fuzz-nightly (libFuzzer -max_total_time)." + required: false + default: "60" permissions: contents: read @@ -41,9 +46,9 @@ env: jobs: # ── ASan + UBSan combined (every PR) ───────────────────────────────── - # Single job: `-fsanitize=address,undefined` in one clang build. - # Replaces the two separate address + undefined matrix rows from - # tests-and-quality-gates.yml. Per ADR-0347, the `function` UBSan + # Single job: `-fsanitize=address,undefined` in one clang build, next to + # the separate address + undefined rows in tests-and-quality-gates.yml, + # which stay the required checks. Per ADR-0347, the `function` UBSan # check is suppressed because ~50 K&R-prototype test harness files # trip it unconditionally (rewrite tracked separately). asan-ubsan: @@ -344,7 +349,7 @@ jobs: mkdir -p fuzz-runtime-corpus set +e ./build-fuzz/test/fuzz/${{ matrix.target }} \ - -max_total_time=${MAX_TOTAL_TIME} \ + -max_total_time="${MAX_TOTAL_TIME}" \ -rss_limit_mb=2048 \ -malloc_limit_mb=1024 \ -timeout=10 \ From 546af650489ed8f849ab3b9edbf2b71df1e3aba0 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:27:26 +0200 Subject: [PATCH 11/15] docs: use the current CI lane names and pins in the backend and CI pages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit lanes that no longer exist: "Build — Ubuntu ARM clang (CPU)", "Build — Ubuntu HIP", "Build — macOS clang (CPU)", "Build — Windows MSVC + CUDA (build only)", "Build — Ubuntu SYCL". They now use the names the checks report under. Facts in the same sentences are corrected against the workflows: the ARM lane runs on ubuntu-24.04-arm, the Windows CUDA leg installs 13.3.1 (packages *_13.3, CUDA_PATH_V13_3), and oneAPI comes from Intel's apt repository at ONEAPI_VERSION (Linux) and a pinned offline installer (Windows), not from the intel/oneapi-runtime-toolkit action. --- .github/AGENTS.md | 8 ++++---- docs/backends/arm/overview.md | 4 ++-- docs/backends/hip/overview.md | 2 +- docs/development/ci-runners.md | 18 +++++++++--------- docs/development/ci-tmate-debug.md | 2 +- docs/development/oneapi-install.md | 17 +++++++++-------- 6 files changed, 26 insertions(+), 25 deletions(-) diff --git a/.github/AGENTS.md b/.github/AGENTS.md index 442fc1b39..1525eddce 100644 --- a/.github/AGENTS.md +++ b/.github/AGENTS.md @@ -323,10 +323,10 @@ must keep enumerating from `meson test --list`. ## Windows CUDA setup path (ADR-0664) -`libvmaf-build-matrix.yml` installs CUDA 13.2.0 directly in -`Build — Windows MSVC + CUDA (build only)` leg. Never restore -`Jimver/cuda-toolkit` for that Windows leg without superseding ADR -and green required Windows CUDA run: v0.2.35 failed before setup on +`libvmaf-build-matrix.yml` installs CUDA 13.3.1 directly in the +`Windows MSVC+CUDA` leg. Do not restore +`Jimver/cuda-toolkit` for that Windows leg without a superseding ADR +and a green required Windows CUDA run: v0.2.35 failed before setup on PR #1463, blocked merge train without Meson or compiler output. Linux CUDA legs still use `Jimver/cuda-toolkit`; ADR-0664 only diff --git a/docs/backends/arm/overview.md b/docs/backends/arm/overview.md index 394c90201..8072e22b0 100644 --- a/docs/backends/arm/overview.md +++ b/docs/backends/arm/overview.md @@ -98,9 +98,9 @@ explicit byte-identity contract. The Netflix golden CPU gate ## Build / CI matrix -The `Build — Ubuntu ARM clang (CPU)` job in the libvmaf build matrix +The `Ubuntu ARM clang` job in the libvmaf build matrix ([`libvmaf-build-matrix.yml`][libvmaf-build-matrix]) runs on -`ubuntu-26.04-arm` against clang and exercises the full unit-test + tox +`ubuntu-24.04-arm` against clang and exercises the full unit-test + tox suite on real aarch64 hardware (not qemu). [libvmaf-build-matrix]: ../../../.github/workflows/libvmaf-build-matrix.yml diff --git a/docs/backends/hip/overview.md b/docs/backends/hip/overview.md index 643b66008..8eff72ed6 100644 --- a/docs/backends/hip/overview.md +++ b/docs/backends/hip/overview.md @@ -257,7 +257,7 @@ Each returns `-ENOSYS` at `init()`. Tracked in `uintptr_t`. This keeps `libvmaf_hip.h` free of ``, mirroring the pattern Vulkan adopted in ADR-0184. - No CI runner with a real AMD GPU exists on GitHub-hosted infrastructure. - The CI compile lane (`Build — Ubuntu HIP`) runs with `-Denable_hip=true` + The CI compile lane (`Ubuntu HIP`) runs with `-Denable_hip=true` but `-Denable_hipcc=false`, so kernels are not compiled or exercised on CI. ## References diff --git a/docs/development/ci-runners.md b/docs/development/ci-runners.md index a7ab048d2..7c2e58895 100644 --- a/docs/development/ci-runners.md +++ b/docs/development/ci-runners.md @@ -62,21 +62,21 @@ at least 5 PRs, ramp up to: ## Windows GPU Build Setup -`Build — Windows MSVC + CUDA (build only)` and `Build — Windows MSVC + oneAPI -SYCL (build only)` are required compile-only gates. GitHub-hosted Windows +`Windows MSVC+CUDA` and `Windows MSVC+SYCL` in `libvmaf-build-matrix.yml` +are required compile-only gates. GitHub-hosted Windows runners do not expose GPUs, so these jobs verify that the MSVC toolchain, headers, libraries, and backend compile/link paths stay healthy. -The CUDA leg installs CUDA 13.2.0 directly from NVIDIA's Windows network +The CUDA leg installs CUDA 13.3.1 directly from NVIDIA's Windows network installer. It requests only the packages needed by the build: -- `nvcc_13.2` -- `cudart_13.2` -- `crt_13.2` -- `nvvm_13.2` -- `visual_studio_integration_13.2` +- `nvcc_13.3` +- `cudart_13.3` +- `crt_13.3` +- `nvvm_13.3` +- `visual_studio_integration_13.3` -The workflow exports `CUDA_PATH`, `CUDA_PATH_V13_2`, and the CUDA `bin` +The workflow exports `CUDA_PATH`, `CUDA_PATH_V13_3`, and the CUDA `bin` directory before running `nvcc.exe --version`. If a future CUDA bump changes Windows package names or install paths, update [ADR-0664](../adr/0664-windows-cuda-toolkit-installer.md) and the workflow diff --git a/docs/development/ci-tmate-debug.md b/docs/development/ci-tmate-debug.md index 71d9a536a..ebdda01c1 100644 --- a/docs/development/ci-tmate-debug.md +++ b/docs/development/ci-tmate-debug.md @@ -24,7 +24,7 @@ Or from the GitHub UI: **Actions → Builds → Run workflow**. ## Finding the tmate URL in the logs 1. Open the Actions run that you triggered. -2. Select the failing macOS job (e.g. `Build — macOS clang (CPU)`). +2. Select the failing macOS job (e.g. `macOS clang`). 3. Expand the **SSH debug session on test failure** step. 4. The step prints two lines: diff --git a/docs/development/oneapi-install.md b/docs/development/oneapi-install.md index 573b55a27..f8ff7f018 100644 --- a/docs/development/oneapi-install.md +++ b/docs/development/oneapi-install.md @@ -5,8 +5,9 @@ DPC++ compiler `icpx`. This page documents the minimum install needed for the SYCL build + clang-tidy lint cycle, the version we pin against, and the upgrade procedure when a newer Intel release ships. -CI installs oneAPI via the official `intel/oneapi-runtime-toolkit` -GitHub Action; this page covers the **local developer machine** path. +CI installs oneAPI from Intel's apt repository on Linux and from Intel's +offline installer on Windows; this page covers the **local developer machine** +path. ## Pinned version @@ -214,12 +215,12 @@ ICPX_ROOT=/opt/intel/oneapi-2025.3/compiler/latest/linux \ ## CI vs local -CI uses the `intel/oneapi-runtime-toolkit` action which installs -whatever version Intel currently publishes as the "stable" tag -(updated independently of this fork). The CI lane therefore may pick -up a newer version than the local pin documented here. Local-vs-CI -divergence on the SYCL kernel binaries is acceptable as long as both -build cleanly and the `Build — Ubuntu SYCL` matrix row stays green; +The Linux SYCL lanes install `${ONEAPI_APT_PACKAGE}` from Intel's apt +repository; its version is set once, as `ONEAPI_VERSION` in +`build-config.env`, and can differ from the local pin documented here. The +`Windows MSVC+SYCL` lane pins its own offline-installer release in the +workflow. Local-vs-CI divergence on the SYCL kernel binaries is acceptable as +long as both build cleanly and the `Ubuntu SYCL` matrix row stays green; bit-identical SYCL output is **not** a guaranteed invariant across Intel oneAPI releases. From a72adb4e5cd18e4d7ae8e6e25579e7d6c99181d4 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Fri, 18 Sep 2026 22:31:55 +0200 Subject: [PATCH 12/15] docs(adr): cite the shared-check-name evidence by commit, not branch tip origin/master moved past 7cc0cc91b while ADR-1259 was written, so the evidence sentence in ADR-1259 and the T-CI-MSVC-CUDA-SHARED-CHECK-NAME state row name the master commit instead of calling it origin/master. --- docs/adr/1259-ci-build-matrix-as-it-runs.md | 2 +- docs/state.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/1259-ci-build-matrix-as-it-runs.md b/docs/adr/1259-ci-build-matrix-as-it-runs.md index 2b579ce09..257046cc3 100644 --- a/docs/adr/1259-ci-build-matrix-as-it-runs.md +++ b/docs/adr/1259-ci-build-matrix-as-it-runs.md @@ -142,7 +142,7 @@ The aggregator looks required checks up by name and keeps one run per name, the one that started last (`newestByName()` in `required-aggregator.yml`). Since #1286 (`f93a0037f`) shortened the display names, the required ADR-0121 lane and the `build.yml` row both report as `Windows MSVC+CUDA`, so either can -hide a failure of the other. On `7cc0cc91b` (origin/master) the `build.yml` +hide a failure of the other. On master commit `7cc0cc91b` the `build.yml` run was cancelled and the matrix run, which started one second later, succeeded; only the success counts. `scripts/ci/check-aggregator-names.sh` compares sets of names, so it cannot see a duplicate. This ADR records the diff --git a/docs/state.md b/docs/state.md index c57e2e8ca..67d17d7b8 100644 --- a/docs/state.md +++ b/docs/state.md @@ -392,7 +392,7 @@ landed fix yet._ | Bug | Summary | Reproducer | Owner | Target | | --- | --- | --- | --- | --- | -| **T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18** | **Two jobs report the required check name `Windows MSVC+CUDA`.** The required lane in `libvmaf-build-matrix.yml` (ADR-0121, build only) and the Windows row of `build.yml` (build plus CPU tests) both got that display name when #1286 (`f93a0037f`) shortened job names. The aggregator's `newestByName()` keeps one check run per name, the one that started last, so a failure in either job can be hidden by a success in the other. On `7cc0cc91b` (origin/master) the `build.yml` run was cancelled and the matrix run, started one second later, succeeded; only the success counted. `scripts/ci/check-aggregator-names.sh` compares sets of names, so it cannot see the duplicate. Recorded in [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md). | `grep -n 'name: Windows MSVC+CUDA' .github/workflows/build.yml .github/workflows/libvmaf-build-matrix.yml` lists both; `gh api repos/VMAFx/vmafx/commits/7cc0cc91b/check-runs --jq '.check_runs[] \| select(.name=="Windows MSVC+CUDA") \| [.conclusion, .started_at, .html_url]'` shows the two runs. | CI owner. Renaming either job changes a required name or adds one, so the fix needs an ADR and the matching `required-aggregator.yml` change. | Closes when every name in the aggregator's `required` list is reported by exactly one job, and `check-aggregator-names.sh` rejects a required name that two jobs share. | +| **T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18** | **Two jobs report the required check name `Windows MSVC+CUDA`.** The required lane in `libvmaf-build-matrix.yml` (ADR-0121, build only) and the Windows row of `build.yml` (build plus CPU tests) both got that display name when #1286 (`f93a0037f`) shortened job names. The aggregator's `newestByName()` keeps one check run per name, the one that started last, so a failure in either job can be hidden by a success in the other. On master commit `7cc0cc91b` the `build.yml` run was cancelled and the matrix run, started one second later, succeeded; only the success counted. `scripts/ci/check-aggregator-names.sh` compares sets of names, so it cannot see the duplicate. Recorded in [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md). | `grep -n 'name: Windows MSVC+CUDA' .github/workflows/build.yml .github/workflows/libvmaf-build-matrix.yml` lists both; `gh api repos/VMAFx/vmafx/commits/7cc0cc91b/check-runs --jq '.check_runs[] \| select(.name=="Windows MSVC+CUDA") \| [.conclusion, .started_at, .html_url]'` shows the two runs. | CI owner. Renaming either job changes a required name or adds one, so the fix needs an ADR and the matching `required-aggregator.yml` change. | Closes when every name in the aggregator's `required` list is reported by exactly one job, and `check-aggregator-names.sh` rejects a required name that two jobs share. | | **T-UPSTREAM-1109-ORIGINAL-PSNR-SYMPTOM-2026-09-08** | **Original cause unresolved; cap fix has narrower scope.** The reporter observed 72 dB from libvmaf where FFmpeg's `psnr` reported 28 dB. An upper cap alone cannot raise 28 to 72. The previously asserted frame-alignment/filter-order diagnosis was not established; the reporter rejected timestamp mismatch because both streams contain static frames. This investigation has not reproduced the original input/filtergraph combination. The *mechanism* the report describes — an unconditional per-frame `MIN` that truncates genuinely near-lossless frames — is real here and is tracked as T-UPSTREAM-1109-PSNR-CAP-TRUNCATES-2026-09-03. | [Research-1166](research/1166-upstream-issue-harvest-2026-09-03.md), [scope correction](research/1193-psnr-cap-truncation-2026-09-06.md#scope-correction-carried-forward) Reproduce original FFmpeg 5.1.1 input/filtergraph and compare with current upstream and fork; current affected status is unknown. | Upstream triage | Open — original symptom only | | **T-SYCL-MOTION-ADD-UV-TOLERANCE-RESOLUTION-2026-09-06** | `test_sycl_motion_add_uv_parity` compares the CPU **float** `float_motion` extractor against the **fixed-point** `motion_sycl`, so its `PARITY_TOL` of 2e-4 is a fixed-point-vs-float budget rather than a bit-exactness bound — the test's own comment says it "accounts for 3-plane fixed-point". That budget was calibrated for the single 256x144 fixture the test has always used. Measured on an Arc A380 at 960x540: cpu(float_motion)=50.27938271, sycl=50.27915223, delta 2.30e-04 against tol 2.00e-04 — 15% over a bound that was never derived for that resolution. Not a regression and not necessarily a defect: the quantisation error of a fixed-point path against a float reference is expected to scale with content and pixel count. It is tracked because the ADR-1206 large-fixture variant for this test is therefore **not** registered, so the resolution-dependent behaviour of `motion_add_uv` is currently untested. | `meson test -C build --suite sycl test_sycl_motion_add_uv_parity` after rebuilding that TU with `-DFIXTURE_W=960u -DFIXTURE_H=540u`, on a host with an Intel GPU. | Owner-driven. Needs a derivation, not a number: what fixed-point-vs-float error bound is justified as a function of resolution for a 3-plane motion sum? Raising 2e-4 to make one fixture pass would be exactly the test-weakening the fork forbids, so it needs its own ADR. | Closes when `test_sycl_motion_add_uv_parity` has a documented, derived tolerance valid across resolutions and is registered in `sycl_parity_large_fixture_tests`. | | **T-DISTS-PLACEHOLDER-CHECKPOINT-2026-09-08** | `docs/metrics/dists.md` documents DISTS (`float_dists_sq`) as computing Deep Image Structure and Texture Similarity via Ding et al., but `model/tiny/dists_sq.onnx` is a 3-op synthetic MSE smoke placeholder (`vmaf_tiny_dists_sq_placeholder_v0`, opset 13, `smoke: true` in `model/tiny/registry.json`), and `core/src/feature/feature_dists.c` lacks the learned deep multi-scale feature stack. While `core/src/feature/feature_dists.c` now logs a loud warning at point of use (`vmaf_log(..., VMAF_LOG_LEVEL_WARNING, "DISTS: using placeholder model checkpoint...")`) and `docs/metrics/dists.md` carries an unmissable banner, DISTS cannot be used as an accurate perceptual metric until real Ding et al. weights and feature extraction are trained and integrated. Flagged by epic #1270 as a 1.0.0 blocker. | Run `vmaf --feature float_dists_sq` or inspect `model/tiny/registry.json` (`dists_sq.onnx` has `"smoke": true`). Point-of-use warning logs at `VMAF_LOG_LEVEL_WARNING`. | unassigned | Deferred post-1.0.0; requires deep feature backbone training and allowlist validation for DISTS ops. | From 7f8a8d9f6ac0f7707e3b6ff0c8ee12ea57bfce5f Mon Sep 17 00:00:00 2001 From: Lusoris Date: Sat, 19 Sep 2026 11:01:54 +0200 Subject: [PATCH 13/15] ci: give the build.yml Windows job its own check name, and gate on shared names libvmaf-build-matrix.yml's required lane and build.yml's Windows row both reported as 'Windows MSVC+CUDA'. The required checks aggregator keeps one run per name, the newest, so either job could mask the other's failure. The build.yml job becomes 'Windows MSVC+CUDA (full)' (the maintainer's choice; the required name and the ruleset stay as they are). check-aggregator-names.sh compared sets of names and could not see the duplicate. It now also fails when more than one job reports a required name, ignoring step names and workflow titles. A fixture test covers the shared-name case and runs in rule-enforcement.yml. ADR-1259 and the CI pages record the rename; T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18 is closed. --- .github/workflows/build.yml | 11 ++- .github/workflows/rule-enforcement.yml | 9 ++ .pre-commit-config.yaml | 2 +- CHANGELOG.md | 7 ++ .../fixed/ci-shared-required-check-name.md | 5 + docs/adr/1259-ci-build-matrix-as-it-runs.md | 21 +++-- docs/development/ci-job-names.md | 11 ++- docs/development/ci.md | 2 +- docs/development/release.md | 5 +- docs/state.md | 2 +- scripts/ci/AGENTS.md | 2 + scripts/ci/check-aggregator-names.sh | 51 +++++++++- .../ci/tests/test-check-aggregator-names.sh | 94 +++++++++++++++++++ 13 files changed, 194 insertions(+), 28 deletions(-) create mode 100644 changelog.d/fixed/ci-shared-required-check-name.md create mode 100755 scripts/ci/tests/test-check-aggregator-names.sh diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 42cc9546a..d91b894c4 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -6,10 +6,11 @@ # Three matrix rows, none of them a required check: # - Linux Intel LLVM: Intel LLVM + ALL Linux backends (CUDA + SYCL + HIP + CPU + DNN) # - macOS Clang+Metal: Apple Clang + CPU + Metal scaffold -# - Windows MSVC+CUDA: MSVC + CPU + CUDA. The required lane in -# libvmaf-build-matrix.yml has the same display name, and the aggregator -# keeps one run per name: see T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18 -# in docs/state.md. +# - Windows MSVC+CUDA (full): MSVC + CPU + CUDA, static, with the CPU +# tests. Named apart from the required `Windows MSVC+CUDA` lane in +# libvmaf-build-matrix.yml: the aggregator keeps one run per name, so a +# shared name let either job mask the other's failure +# (scripts/ci/check-aggregator-names.sh now rejects a shared required name). # # Rationale: container-first VMAFX philosophy (ADR-0686, ADR-0701) means the # production artefact is the `vmafx-dev-mcp` image, not a per-backend binary; @@ -101,7 +102,7 @@ jobs: # windows-2025 runner). CPU-only meson tests are run after build. # ────────────────────────────────────────────────────────────── - os: windows-2025 - name: Windows MSVC+CUDA + name: Windows MSVC+CUDA (full) cuda: true windows: true meson_extra: >- diff --git a/.github/workflows/rule-enforcement.yml b/.github/workflows/rule-enforcement.yml index 2b4d53fe0..f8c2d70a7 100644 --- a/.github/workflows/rule-enforcement.yml +++ b/.github/workflows/rule-enforcement.yml @@ -690,6 +690,15 @@ jobs: bash scripts/ci/check-state-md-rows.sh bash scripts/ci/tests/test-check-state-md-rows.sh + - name: Required check names have one reporter each (ADR-1259) + # The aggregator keeps one check run per name, so two jobs sharing a + # required name can mask each other's failure (`Windows MSVC+CUDA` was + # reported by two workflows until 2026-09-19). The gate also runs as a + # pre-commit hook; its hermetic self-test runs here. + run: | + bash scripts/ci/check-aggregator-names.sh + bash scripts/ci/tests/test-check-aggregator-names.sh + - name: Test the self-hosted SYCL runner probe # ADR-1177. Hermetic unit suite for check-runner-available.sh. run: bash scripts/ci/tests/test-runner-available.sh diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 0e2170e7f..108e9e82b 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -286,7 +286,7 @@ repos: name: Check required CI checks aggregator list matches workflows entry: scripts/ci/check-aggregator-names.sh language: script - files: '^\.github/workflows/.*\.ya?ml$|^scripts/ci/check-aggregator-names\.sh$' + files: '^\.github/workflows/.*\.ya?ml$|^scripts/ci/(tests/test-)?check-aggregator-names\.sh$' pass_filenames: false - id: check-copyright name: ADR-0105 copyright header present (C/C++/CUDA) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ad0102f5..b48be3477 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20746,6 +20746,13 @@ Resolved 8 pre-existing required-aggregator failures that blocked every PR post- Research digest: `docs/research/0735-ci-required-failures-round-3-2026-05-28.md` +- **CI: a required check can no longer be masked by a second job with the + same name.** Two workflows reported `Windows MSVC+CUDA`, and the required + checks aggregator keeps only the newest run per name. The `build.yml` job is + now `Windows MSVC+CUDA (full)`, and `scripts/ci/check-aggregator-names.sh` + fails when more than one job reports a required name. + + **CI: fix build-matrix stale `libvmaf` source paths + ASan + motion_v2 coverage leaks** Three CI regressions fixed in a single sweep: diff --git a/changelog.d/fixed/ci-shared-required-check-name.md b/changelog.d/fixed/ci-shared-required-check-name.md new file mode 100644 index 000000000..172bba826 --- /dev/null +++ b/changelog.d/fixed/ci-shared-required-check-name.md @@ -0,0 +1,5 @@ +- **CI: a required check can no longer be masked by a second job with the + same name.** Two workflows reported `Windows MSVC+CUDA`, and the required + checks aggregator keeps only the newest run per name. The `build.yml` job is + now `Windows MSVC+CUDA (full)`, and `scripts/ci/check-aggregator-names.sh` + fails when more than one job reports a required name. diff --git a/docs/adr/1259-ci-build-matrix-as-it-runs.md b/docs/adr/1259-ci-build-matrix-as-it-runs.md index 257046cc3..3552373ac 100644 --- a/docs/adr/1259-ci-build-matrix-as-it-runs.md +++ b/docs/adr/1259-ci-build-matrix-as-it-runs.md @@ -108,7 +108,7 @@ Windows lanes build every time. | `Ubuntu CUDA` | CUDA build, CPU-only tests | not required | `eaad70462` | removed by ADR-0689, restored by `384d97d03` | | `Ubuntu SYCL+CUDA` | icpx SYCL and CUDA build, CPU-only tests | not required | `eaad70462` | decision 2 | | `Windows MinGW64` | MSYS2 gcc static build, Win64 stack-alignment check, meson tests | required | ADR-0115, ADR-0116 | ADR-1234, ADR-1253, ADR-1254; removed by ADR-0691, restored by `384d97d03` | -| `Windows MSVC+CUDA` | MSVC and nvcc build, no tests | required; the name is shared, see below | ADR-0121 | ADR-1234 | +| `Windows MSVC+CUDA` | MSVC and nvcc build, no tests | required | ADR-0121 | ADR-1234 | | `Windows MSVC+SYCL` | MSVC and oneAPI build, no tests | required | ADR-0121 | ADR-1234 | ### `build.yml` (workflow `Build`) @@ -120,7 +120,7 @@ only documentation (`paths-ignore`). | --- | --- | --- | --- | --- | | `Linux Intel LLVM` | icx/icpx build with CUDA, SYCL, HIP and DNN, meson dnn suite, meson tests, HIP smoke test; no tox | not required | ADR-0710 | GCC until #1161 (`195f88a22`); ADR-1185 | | `macOS Clang+Metal` | CPU and Metal build, meson tests, tox | not required | ADR-0710 | ADR-1234 cites a failure it caught | -| `Windows MSVC+CUDA` | MSVC and nvcc build, CPU tests | not required in its own right; shares the required name, see below | ADR-0710 | — | +| `Windows MSVC+CUDA (full)` | MSVC and nvcc build, CPU tests | not required; renamed by this ADR, see below | ADR-0710 | — | ### Sanitizer and static-analysis gates (decision 3) @@ -145,9 +145,11 @@ lane and the `build.yml` row both report as `Windows MSVC+CUDA`, so either can hide a failure of the other. On master commit `7cc0cc91b` the `build.yml` run was cancelled and the matrix run, which started one second later, succeeded; only the success counts. `scripts/ci/check-aggregator-names.sh` -compares sets of names, so it cannot see a duplicate. This ADR records the -defect but does not fix it, because the fix changes a required name. It is -tracked as `T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in `docs/state.md`. +compared sets of names, so it could not see a duplicate. The maintainer chose +to rename the `build.yml` job (see References), which leaves the required name +and the ruleset untouched: it is now `Windows MSVC+CUDA (full)`, and +`check-aggregator-names.sh` fails when more than one job reports a required +name (`T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in `docs/state.md`). ## Alternatives considered @@ -167,13 +169,12 @@ tracked as `T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in `docs/state.md`. - **Negative**: every non-draft PR that touches the C core still runs 17 matrix lanes and the three `build.yml` rows, and `build.yml` largely repeats the matrix (`Linux Intel LLVM` against the SYCL, CUDA and HIP lanes, - `macOS Clang+Metal` against `macOS Metal`, and the two `Windows MSVC+CUDA` - jobs). The shared check name stays open. + `macOS Clang+Metal` against `macOS Metal`, and the two Windows MSVC and + CUDA jobs). - **Neutral / follow-ups**: the unreleased changelog fragments that announced the ADR-0689, ADR-0691, ADR-0710 and ADR-0728 removals are removed or corrected, so the first VMAFx release notes do not announce them. Trimming - the overlap, and renaming one of the two `Windows MSVC+CUDA` jobs, are - separate decisions. + the overlap is a separate decision. ## References @@ -206,3 +207,5 @@ tracked as `T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in `docs/state.md`. - Popup, 2026-09-18, lanes without a later ADR: "Keep them, record it (Recommended)". - Popup, 2026-09-18, required checks: "Keep the four required (Recommended)". +- Popup, 2026-09-19, shared `Windows MSVC+CUDA` check name: "Rename build.yml's + job (Recommended)". diff --git a/docs/development/ci-job-names.md b/docs/development/ci-job-names.md index 2fb4205d6..2aad0d9d9 100644 --- a/docs/development/ci-job-names.md +++ b/docs/development/ci-job-names.md @@ -60,10 +60,11 @@ workflows; each run evaluates 39 of them, because only the Scorecard gate for the triggering event applies. A required name must be reported by exactly one job, because the aggregator -keeps one check run per name. `Windows MSVC+CUDA` currently breaks that: the -required `libvmaf-build-matrix.yml` lane and the `build.yml` row share it -(`T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` in [state.md](../state.md); -[ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md)). +keeps one check run per name, so two jobs sharing a name can mask each other's +failure. `scripts/ci/check-aggregator-names.sh` enforces it. The `build.yml` +Windows row used to share `Windows MSVC+CUDA` with the required +`libvmaf-build-matrix.yml` lane and is now `Windows MSVC+CUDA (full)` +([ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md)). To prevent drift between workflow job definitions and the aggregator's required check array, all required checks are tagged in their defining workflow with @@ -144,7 +145,7 @@ job the PR renamed, every required check and every build lane. | `standards-gate.yml` | added after #1286 | `Standards & Invariant Verification Gate` | 39 | Yes | | `build.yml` | `Build — Linux (Intel LLVM, all backends)` | `Linux Intel LLVM` | 16 | No | | `build.yml` | `Build — macOS (Clang, CPU + Metal)` | `macOS Clang+Metal` | 17 | No | -| `build.yml` | `Build — Windows (MSVC + CUDA)` | `Windows MSVC+CUDA` | 18 | No | +| `build.yml` | `Build — Windows (MSVC + CUDA)` | `Windows MSVC+CUDA (full)` | 24 | No | | `ffmpeg-integration.yml` | `FFmpeg — Ubuntu gcc (Build Only)` | `FFmpeg Ubuntu gcc` | 17 | No | | `ffmpeg-integration.yml` | `FFmpeg — macOS clang (Build Only)` | `FFmpeg macOS clang` | 18 | No | | `ffmpeg-integration.yml` | `FFmpeg — SYCL (Build Only)` | `FFmpeg SYCL` | 11 | No | diff --git a/docs/development/ci.md b/docs/development/ci.md index 9936569c3..458d9e90c 100644 --- a/docs/development/ci.md +++ b/docs/development/ci.md @@ -19,7 +19,7 @@ The main `pull_request`-triggered workflows include: | [`go-ci.yml`](../../.github/workflows/go-ci.yml) | Required Go vet, security scan, runner smoke, and tests (ADR-1238). | | [`ffmpeg-integration.yml`](../../.github/workflows/ffmpeg-integration.yml) | FFmpeg + libvmaf build (Linux GCC / macOS Clang / SYCL). | | [`libvmaf-build-matrix.yml`](../../.github/workflows/libvmaf-build-matrix.yml) | Cross-platform / cross-backend libvmaf build matrix: 17 lanes, six of them required. | -| [`build.yml`](../../.github/workflows/build.yml) | One all-backend build per OS (`Linux Intel LLVM`, `macOS Clang+Metal`, `Windows MSVC+CUDA`), alongside the matrix; not required. | +| [`build.yml`](../../.github/workflows/build.yml) | One all-backend build per OS (`Linux Intel LLVM`, `macOS Clang+Metal`, `Windows MSVC+CUDA (full)`), alongside the matrix; not required. | | [`rule-enforcement.yml`](../../.github/workflows/rule-enforcement.yml) | ADR-0100 / 0106 / 0108 / 0165 process gates. | | [`tests-and-quality-gates.yml`](../../.github/workflows/tests-and-quality-gates.yml) | Netflix golden, sanitizers, tiny-AI, MCP, coverage, assertion-density. | | [`sanitizers.yml`](../../.github/workflows/sanitizers.yml) | Combined ASan+UBSan on PRs, TSan on master pushes, nightly fuzzing; not required (the required sanitizers are in `tests-and-quality-gates.yml`). | diff --git a/docs/development/release.md b/docs/development/release.md index e6da0c9f4..06e23ae29 100644 --- a/docs/development/release.md +++ b/docs/development/release.md @@ -599,9 +599,8 @@ is enforced at the host, not just honored by convention. Ubuntu HIP, SYCL float_ssim Parity. All but the last are `libvmaf-build-matrix.yml` lanes; no `build.yml` row is required ([ADR-1259](../adr/1259-ci-build-matrix-as-it-runs.md)). The `build.yml` - Windows row also reports as `Windows MSVC+CUDA`, and the aggregator keeps - only the later of the two runs (`T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18` - in [state.md](../state.md)). + Windows row is named `Windows MSVC+CUDA (full)` so that it cannot + stand in for the required lane. - **Static analysis (10):** CodeQL ×4 (CodeQL, CodeQL (C/C++), CodeQL (Python), CodeQL (Actions)), Pre-Commit, Python Lint, Semgrep, Tidy Changed, Tidy Ratchet, Cppcheck. diff --git a/docs/state.md b/docs/state.md index 67d17d7b8..e9b887299 100644 --- a/docs/state.md +++ b/docs/state.md @@ -392,7 +392,6 @@ landed fix yet._ | Bug | Summary | Reproducer | Owner | Target | | --- | --- | --- | --- | --- | -| **T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18** | **Two jobs report the required check name `Windows MSVC+CUDA`.** The required lane in `libvmaf-build-matrix.yml` (ADR-0121, build only) and the Windows row of `build.yml` (build plus CPU tests) both got that display name when #1286 (`f93a0037f`) shortened job names. The aggregator's `newestByName()` keeps one check run per name, the one that started last, so a failure in either job can be hidden by a success in the other. On master commit `7cc0cc91b` the `build.yml` run was cancelled and the matrix run, started one second later, succeeded; only the success counted. `scripts/ci/check-aggregator-names.sh` compares sets of names, so it cannot see the duplicate. Recorded in [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md). | `grep -n 'name: Windows MSVC+CUDA' .github/workflows/build.yml .github/workflows/libvmaf-build-matrix.yml` lists both; `gh api repos/VMAFx/vmafx/commits/7cc0cc91b/check-runs --jq '.check_runs[] \| select(.name=="Windows MSVC+CUDA") \| [.conclusion, .started_at, .html_url]'` shows the two runs. | CI owner. Renaming either job changes a required name or adds one, so the fix needs an ADR and the matching `required-aggregator.yml` change. | Closes when every name in the aggregator's `required` list is reported by exactly one job, and `check-aggregator-names.sh` rejects a required name that two jobs share. | | **T-UPSTREAM-1109-ORIGINAL-PSNR-SYMPTOM-2026-09-08** | **Original cause unresolved; cap fix has narrower scope.** The reporter observed 72 dB from libvmaf where FFmpeg's `psnr` reported 28 dB. An upper cap alone cannot raise 28 to 72. The previously asserted frame-alignment/filter-order diagnosis was not established; the reporter rejected timestamp mismatch because both streams contain static frames. This investigation has not reproduced the original input/filtergraph combination. The *mechanism* the report describes — an unconditional per-frame `MIN` that truncates genuinely near-lossless frames — is real here and is tracked as T-UPSTREAM-1109-PSNR-CAP-TRUNCATES-2026-09-03. | [Research-1166](research/1166-upstream-issue-harvest-2026-09-03.md), [scope correction](research/1193-psnr-cap-truncation-2026-09-06.md#scope-correction-carried-forward) Reproduce original FFmpeg 5.1.1 input/filtergraph and compare with current upstream and fork; current affected status is unknown. | Upstream triage | Open — original symptom only | | **T-SYCL-MOTION-ADD-UV-TOLERANCE-RESOLUTION-2026-09-06** | `test_sycl_motion_add_uv_parity` compares the CPU **float** `float_motion` extractor against the **fixed-point** `motion_sycl`, so its `PARITY_TOL` of 2e-4 is a fixed-point-vs-float budget rather than a bit-exactness bound — the test's own comment says it "accounts for 3-plane fixed-point". That budget was calibrated for the single 256x144 fixture the test has always used. Measured on an Arc A380 at 960x540: cpu(float_motion)=50.27938271, sycl=50.27915223, delta 2.30e-04 against tol 2.00e-04 — 15% over a bound that was never derived for that resolution. Not a regression and not necessarily a defect: the quantisation error of a fixed-point path against a float reference is expected to scale with content and pixel count. It is tracked because the ADR-1206 large-fixture variant for this test is therefore **not** registered, so the resolution-dependent behaviour of `motion_add_uv` is currently untested. | `meson test -C build --suite sycl test_sycl_motion_add_uv_parity` after rebuilding that TU with `-DFIXTURE_W=960u -DFIXTURE_H=540u`, on a host with an Intel GPU. | Owner-driven. Needs a derivation, not a number: what fixed-point-vs-float error bound is justified as a function of resolution for a 3-plane motion sum? Raising 2e-4 to make one fixture pass would be exactly the test-weakening the fork forbids, so it needs its own ADR. | Closes when `test_sycl_motion_add_uv_parity` has a documented, derived tolerance valid across resolutions and is registered in `sycl_parity_large_fixture_tests`. | | **T-DISTS-PLACEHOLDER-CHECKPOINT-2026-09-08** | `docs/metrics/dists.md` documents DISTS (`float_dists_sq`) as computing Deep Image Structure and Texture Similarity via Ding et al., but `model/tiny/dists_sq.onnx` is a 3-op synthetic MSE smoke placeholder (`vmaf_tiny_dists_sq_placeholder_v0`, opset 13, `smoke: true` in `model/tiny/registry.json`), and `core/src/feature/feature_dists.c` lacks the learned deep multi-scale feature stack. While `core/src/feature/feature_dists.c` now logs a loud warning at point of use (`vmaf_log(..., VMAF_LOG_LEVEL_WARNING, "DISTS: using placeholder model checkpoint...")`) and `docs/metrics/dists.md` carries an unmissable banner, DISTS cannot be used as an accurate perceptual metric until real Ding et al. weights and feature extraction are trained and integrated. Flagged by epic #1270 as a 1.0.0 blocker. | Run `vmaf --feature float_dists_sq` or inspect `model/tiny/registry.json` (`dists_sq.onnx` has `"smoke": true`). Point-of-use warning logs at `VMAF_LOG_LEVEL_WARNING`. | unassigned | Deferred post-1.0.0; requires deep feature backbone training and allowlist validation for DISTS ops. | @@ -456,6 +455,7 @@ landed fix yet._ ## Recently closed | **T-CI-FORMATTER-PINS-DRIFT-2026-09-19** | **`make lint-tools` installed an older ruff than the pre-commit hook ran.** The `Makefile` states that its `RUFF_VERSION` / `BLACK_VERSION` are identical to `.pre-commit-config.yaml`, but nothing enforced it and Renovate only managed the hook side: the hook reached ruff 0.16.8 while the `Makefile` installed 0.16.5 and one install hint still named 0.15.17, so the local gate and the hook could disagree about what counts as a violation. **FIXED**: pins aligned, install hints use the variables, `scripts/ci/check-workflow-versions.py` gained `check_formatter_pins` (fails on a version mismatch or a literal `ruff==` / `black==` in a recipe; fixture `scripts/ci/tests/test_formatter_pins_single_source.py`), and `renovate.json` gained two regex managers plus a rule that groups the `Makefile` pins with the pre-commit hooks. The same PR holds `onnx` below 1.23 in Renovate (see T-AI-ONNX-IR14-UNLOADABLE-2026-09-18) and adds `.devcontainer/**` to `ignorePaths`, because a digest bump edited the generated `Dockerfile.praetor` and the governance audit then reported the devcontainer as out of sync. | — | `integration/deps-20260919` | 2026-09-19 | closed | +| **T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18** | **Two jobs reported the required check name `Windows MSVC+CUDA`.** Since #1286 (`f93a0037f`) shortened the display names, the required lane in `libvmaf-build-matrix.yml` (ADR-0121) and the `build.yml` Windows row shared it. The aggregator keeps one check run per name, the newest (`newestByName()`), so either job could mask the other's failure: on master `7cc0cc91b` the `build.yml` run was cancelled and only the matrix run's success counted. `scripts/ci/check-aggregator-names.sh` compared name sets and could not see a duplicate. **FIXED**: the `build.yml` job is `Windows MSVC+CUDA (full)` (the maintainer's choice; the required name and the ruleset are untouched), and the gate now fails when more than one job reports a required name. It ignores step names and workflow titles; `scripts/ci/tests/test-check-aggregator-names.sh` covers the shared-name case and runs in `rule-enforcement.yml`. | [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md) | `ci/retire-i686-lane` | 2026-09-19 | closed | | **T-AI-ONNX-IR14-UNLOADABLE-2026-09-18** | **The Tiny AI test job failed on master once `onnx` 1.23.0 was released.** `ai/pyproject.toml` allowed `onnx>=1.22.0,<2.0`, so CI resolved 1.23.0, whose `helper.make_model()` writes IR version 14 by default. The pinned `onnxruntime` 1.30.0 loads at most IR 13, so every test that builds a model with onnx and runs it in ORT failed with `Unsupported model IR version: 14, max supported IR version: 13` (21 tests across `test_cross_backend`, `test_learned_filter_audit`, `test_profile`, `test_bisect_model_quality` and the PTQ round trip). The same mismatch would make any model built in-repo with onnx 1.23 unloadable by the runtime the fork ships. Reproduced in isolation: the same Relu model is IR 14 and rejected with onnx 1.23.0, IR 13 and accepted with 1.22.0. **FIXED**: `onnx>=1.22.0,<1.23` in `ai/pyproject.toml` and `tools/ensemble-training-kit/pyproject.toml`, with a comment tying the cap to onnxruntime's IR limit. | — | `fix/onnx-ir-cap` | 2026-09-18 | closed | | **T-NO-ASM-SIMD-TEST-WARNINGS-2026-09-18** | **A `-Denable_asm=false` build emitted ~30 `-Wunused-function` / `-Wunused-const-variable` / `-Wunused-variable` warnings in SIMD parity-test files.** Ten files under `core/test/` (`test_vif_simd.c`, `test_ssimulacra2_simd.c`, `test_speed_simd.c`, `test_psnr_hvs_simd.c`, `test_ms_ssim_decimate.c`, `test_motion_v2_simd.c`, `test_iqa_convolve.c`, `test_integer_ssim_simd.c`, `test_cambi_simd.c`, `test_cambi.c`) defined scalar-reference helpers, fixture builders, and lookup tables/variables unconditionally while every caller sat under an ISA guard (`ARCH_X86`, `HAVE_AVX512`, `ARCH_AARCH64`, or the `ARCH_X86 \|\| ARCH_AARCH64` union `run_tests()` already uses). Measured on an i686 build; reproduces on x86-64 with asm off too. **FIXED**: each helper moved under the exact union of ISA conditions its callers use, walking the full dependency chain so the warning does not relocate to a `pick_*` dispatcher or `ref_*` scalar reference one level down (`test_ssimulacra2_simd.c` needed one guard spanning its whole pick/ref/test helper section for this reason). Nothing deleted, no `(void)`-cast or `[[maybe_unused]]` suppression. Verified zero warnings and an identical test count on `-Denable_asm=false` x86-64, default x86-64, and an aarch64 cross build under `qemu-aarch64-static`. The aarch64 build's two `vif_neon.c` `-Wunused-but-set-variable` warnings (dead `i_dst_stride` counters in the VIF statistic kernels) are fixed on the same branch, and `vif_neon.c` is split into helpers that clear its 36 clang-tidy findings with NEON output byte-identical under qemu (68 of 68 runs). | none | `fix/no-asm-test-warnings` | 2026-09-18 | closed | | **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for four months after ADR-0728 removed it.** ADR-0728 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The other ADR-0728 removals the same merge restored are audited separately. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0728](adr/0728-native-build-sunset.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | diff --git a/scripts/ci/AGENTS.md b/scripts/ci/AGENTS.md index 6e368c988..29225082f 100644 --- a/scripts/ci/AGENTS.md +++ b/scripts/ci/AGENTS.md @@ -133,6 +133,8 @@ until master fixed. | `test-state-md-touch-check.sh` | (local-only fixture driver, not invoked by CI) | Run before pushing changes to `state-md-touch-check.sh`; constructs throw-away `mktemp -d` git repos so the test is hermetic. | +| `check-aggregator-names.sh` | `check-aggregator-names` pre-commit hook; `rule-enforcement.yml` — `Required check names have one reporter each (ADR-1259)` (gate + `tests/test-check-aggregator-names.sh`) | Two invariants: the aggregator's `required` list equals the `# required-aggregator`-marked names, and each required name is reported by exactly one job (the aggregator keeps only the newest run per name, so a shared name lets one job mask the other's failure — `Windows MSVC+CUDA`, fixed 2026-09-19). `job_names()` skips everything under a `steps:` key and the workflow's top-level `name:`; change that parser and the fixture test together. A new lane must not reuse a required name. | + | `twin-drift-check.sh` + `twin-drift-allowlist.txt` | `lint-and-format.yml` — `twin-drift-check` job ([ADR-1135](../../docs/adr/1135-ci-twin-drift-gate.md)); the `twin-drift-check` pre-push hook in `.pre-commit-config.yaml` | The job `name:` (`Twin Drift + Stale Source Refs (ADR-1135)`) is listed verbatim in `required-aggregator.yml` — rename both in the same commit or every PR blocks on a phantom check. The allowlist path is the default of `TWIN_DRIFT_ALLOWLIST`; each row is ` ` and is validated (reason mandatory; a row whose file is gone, whose side is compiled again, or whose pair no longer exists fails the gate). The source-extension regex (`c cpp cc cxx cu hip m mm metal pyx`), the `output:` / `@…@` / absolute-path skip rules, the `var + 'x.c'` and `os.path.join` resolution, the suffix-search fallback and the `twin-drift-ignore: ` marker are the parser contract — change them in the script AND in `tests/test-twin-drift-check.sh` together. The awk program must stay POSIX (mawk is Ubuntu's default `awk`): no `gensub`, no `length(array)`, no `--re-interval`-only syntax. | | `tests/test-twin-drift-check.sh` | (local-only fixture driver, not invoked by CI) | Run before pushing changes to `twin-drift-check.sh`; 24 hermetic `mktemp -d` git-repo cases covering both predicates, the allowlist validation and every resolution rule. Also run it under `gawk --posix` when touching the awk. | diff --git a/scripts/ci/check-aggregator-names.sh b/scripts/ci/check-aggregator-names.sh index fc1b973df..8022ed17b 100755 --- a/scripts/ci/check-aggregator-names.sh +++ b/scripts/ci/check-aggregator-names.sh @@ -3,12 +3,17 @@ # SPDX-License-Identifier: EUPL-1.2 # # check-aggregator-names.sh — verify that required-aggregator.yml's required list -# and the set of # required-aggregator check names in .github/workflows/*.yml are identical. +# and the set of # required-aggregator check names in .github/workflows/*.yml are identical, +# and that every required name is reported by exactly one job. The aggregator keeps one check +# run per name (the newest), so two jobs sharing a required name can mask each other's failure +# (T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18). +# +# Usage: check-aggregator-names.sh [repo-root] (the argument exists for the fixture tests) set -euo pipefail SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd -- "$SCRIPT_DIR/../.." && pwd)" +REPO_ROOT="${1:-$(cd -- "$SCRIPT_DIR/../.." && pwd)}" python3 - "$REPO_ROOT" <<'PYEOF' import re @@ -54,5 +59,45 @@ if diff: msg.append(f" - '{n}'") sys.exit("\n".join(msg)) -print(f"OK: all {len(agg_names)} required checks in required-aggregator.yml match workflow definitions.") +def job_names(path: Path) -> list[str]: + """Every job or matrix-row display name in a workflow. + + Step names are not check names, so everything nested under a `steps:` key + is skipped, as is the workflow's own top-level `name:`. + """ + names = [] + steps_indent = None + for line in path.read_text(encoding="utf-8").splitlines(): + stripped = line.lstrip() + if not stripped or stripped.startswith("#"): + continue + indent = len(line) - len(stripped) + if steps_indent is not None: + if indent > steps_indent or (indent == steps_indent and stripped.startswith("- ")): + continue + steps_indent = None + if re.match(r"steps:\s*(#.*)?$", stripped): + steps_indent = indent + continue + m = re.match(r"(?:-\s*)?name:\s*[\"']?([^\"'#\n]+?)[\"']?\s*(?:#.*)?$", stripped) + if m and indent > 0: + names.append(m.group(1).strip()) + return names + + +reporters: dict[str, list[str]] = {} +for p in sorted((repo_root / ".github" / "workflows").glob("*.yml")): + if p.name == "required-aggregator.yml": + continue + for name in job_names(p): + if name in agg_names: + reporters.setdefault(name, []).append(p.name) +shared = {n: files for n, files in reporters.items() if len(files) > 1} +if shared: + msg = ["Required check names reported by more than one job (the aggregator keeps only the newest run per name):"] + for n in sorted(shared): + msg.append(f" - '{n}': {', '.join(shared[n])}") + sys.exit("\n".join(msg)) + +print(f"OK: all {len(agg_names)} required checks in required-aggregator.yml match workflow definitions, each reported by one job.") PYEOF diff --git a/scripts/ci/tests/test-check-aggregator-names.sh b/scripts/ci/tests/test-check-aggregator-names.sh new file mode 100755 index 000000000..0c6bf8678 --- /dev/null +++ b/scripts/ci/tests/test-check-aggregator-names.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# Copyright 2026 Lusoris +# SPDX-License-Identifier: EUPL-1.2 + +# Tests for scripts/ci/check-aggregator-names.sh: the required list must match +# the marked workflow names, and each required name has exactly one reporter. +set -euo pipefail +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +script="$here/../check-aggregator-names.sh" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +expect() { # $1 label, $2 expected rc, $3 fixture root + local rc=0 + bash "$script" "$3" >/dev/null 2>&1 || rc=$? + if [ "$rc" -ne "$2" ]; then + echo "FAIL $1: expected rc=$2 got rc=$rc" >&2 + exit 1 + fi + echo "ok $1 (rc=$rc)" +} + +fixture() { # $1 root; stdin = second workflow + mkdir -p "$1/.github/workflows" + cat >"$1/.github/workflows/required-aggregator.yml" <<'YML' +jobs: + aggregate: + steps: + - name: Evaluate + with: + script: | + const required = [ + 'Alpha Build', + 'Beta Lint', + ]; +YML + cat >"$1/.github/workflows/matrix.yml" <<'YML' +name: Builds +jobs: + build: + name: ${{ matrix.name }} + strategy: + matrix: + include: + # required-aggregator + - os: ubuntu-latest + name: Alpha Build + steps: + - name: Alpha Build + run: true + lint: + # required-aggregator + name: Beta Lint + steps: + - name: Beta Lint + run: true +YML + cat >"$1/.github/workflows/other.yml" +} + +fixture "$tmp/clean" <<'YML' +name: Beta Lint +jobs: + extra: + name: Gamma Extra + steps: + - name: Alpha Build + run: true +YML +expect "one reporter per required name passes (step and workflow names ignored)" 0 "$tmp/clean" + +fixture "$tmp/shared" <<'YML' +name: Other +jobs: + full: + name: ${{ matrix.name }} + strategy: + matrix: + include: + - os: windows-2025 + name: Alpha Build +YML +expect "a required name reported by a second workflow's job fails" 1 "$tmp/shared" + +fixture "$tmp/unmarked" <<'YML' +name: Other +jobs: + extra: + # required-aggregator + name: Delta Unlisted +YML +expect "a marked name missing from the required list fails" 1 "$tmp/unmarked" + +echo "all check-aggregator-names tests passed" From 4937dd337bb84f74e3084b0b4526b8894b5b8ec3 Mon Sep 17 00:00:00 2001 From: Lusoris Date: Sat, 19 Sep 2026 13:17:38 +0200 Subject: [PATCH 14/15] docs(state): keep one copy of the i686 lane row after the rebase The union resolution left three copies: the pre-correction wording that credited ADR-0728, and two copies of the corrected one that credits ADR-0691. Keeps the corrected row. Also carries the HISS baseline re-recorded from a clean clone with the pinned engine: the CAMBI SIMD dispatch merged in the meantime and moved the lines the line-keyed file tracks in cambi.c; no finding is new. --- .standards-baseline.json | 84 +++++++++++----------------------------- docs/state.md | 2 - 2 files changed, 22 insertions(+), 64 deletions(-) diff --git a/.standards-baseline.json b/.standards-baseline.json index 8f77e967d..d877b1490 100644 --- a/.standards-baseline.json +++ b/.standards-baseline.json @@ -1,9 +1,9 @@ { "version": 1, - "generated_at": "2026-09-18T20:40:41Z", + "generated_at": "2026-09-19T11:17:17Z", "repository": "", "commit_sha": "", - "total_infractions": 1627, + "total_infractions": 1622, "infractions": [ { "rule_id": "HISS-04", @@ -2581,14 +2581,6 @@ "message": "Function '{' (212 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", "fingerprint": "core/src/feature/arm64/adm_neon.c:64:HISS-04" }, - { - "rule_id": "HISS-04", - "file_path": "core/src/feature/arm64/cambi_neon.c", - "line_number": 125, - "symbol": "{", - "message": "Function '{' (86 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/arm64/cambi_neon.c:125:HISS-04" - }, { "rule_id": "HISS-04", "file_path": "core/src/feature/arm64/float_adm_dwt2_neon.c", @@ -2717,38 +2709,6 @@ "message": "Function '{' (114 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", "fingerprint": "core/src/feature/arm64/ssimulacra2_sve2.c:673:HISS-04" }, - { - "rule_id": "HISS-04", - "file_path": "core/src/feature/arm64/vif_neon.c", - "line_number": 292, - "symbol": "vif_subsample_rd_8_neon", - "message": "Function 'vif_subsample_rd_8_neon' (132 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/arm64/vif_neon.c:292:HISS-04" - }, - { - "rule_id": "HISS-04", - "file_path": "core/src/feature/arm64/vif_neon.c", - "line_number": 427, - "symbol": "{", - "message": "Function '{' (137 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/arm64/vif_neon.c:427:HISS-04" - }, - { - "rule_id": "HISS-04", - "file_path": "core/src/feature/arm64/vif_neon.c", - "line_number": 566, - "symbol": "vif_statistic_8_neon", - "message": "Function 'vif_statistic_8_neon' (326 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/arm64/vif_neon.c:566:HISS-04" - }, - { - "rule_id": "HISS-04", - "file_path": "core/src/feature/arm64/vif_neon.c", - "line_number": 895, - "symbol": "{", - "message": "Function '{' (326 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/arm64/vif_neon.c:895:HISS-04" - }, { "rule_id": "HISS-04", "file_path": "core/src/feature/brisque_math.h", @@ -2760,53 +2720,53 @@ { "rule_id": "HISS-04", "file_path": "core/src/feature/cambi.c", - "line_number": 189, + "line_number": 196, "symbol": "static const VmafOption options[] = {", "message": "Function 'static const VmafOption options[] = {' (184 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/cambi.c:189:HISS-04" + "fingerprint": "core/src/feature/cambi.c:196:HISS-04" }, { "rule_id": "HISS-02", "file_path": "core/src/feature/cambi.c", - "line_number": 433, + "line_number": 440, "message": "Legacy unbounded loop in native code", - "fingerprint": "core/src/feature/cambi.c:433:HISS-02" + "fingerprint": "core/src/feature/cambi.c:440:HISS-02" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 834, + "line_number": 857, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:834:HISS-01" + "fingerprint": "core/src/feature/cambi.c:857:HISS-01" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 840, + "line_number": 863, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:840:HISS-01" + "fingerprint": "core/src/feature/cambi.c:863:HISS-01" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 847, + "line_number": 870, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:847:HISS-01" + "fingerprint": "core/src/feature/cambi.c:870:HISS-01" }, { "rule_id": "HISS-01", "file_path": "core/src/feature/cambi.c", - "line_number": 851, + "line_number": 874, "message": "Legacy non-DAG control flow jump (goto)", - "fingerprint": "core/src/feature/cambi.c:851:HISS-01" + "fingerprint": "core/src/feature/cambi.c:874:HISS-01" }, { "rule_id": "HISS-04", "file_path": "core/src/feature/cambi.h", - "line_number": 40, + "line_number": 49, "symbol": "static const float reciprocal_lut[CAMBI_RECIPROCAL_LUT_SIZE] = {0.0f,", "message": "Function 'static const float reciprocal_lut[CAMBI_RECIPROCAL_LUT_SIZE] = {0.0f,' (4226 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/src/feature/cambi.h:40:HISS-04" + "fingerprint": "core/src/feature/cambi.h:49:HISS-04" }, { "rule_id": "HISS-04", @@ -9777,10 +9737,10 @@ { "rule_id": "HISS-04", "file_path": "core/test/test_psnr_hvs_simd.c", - "line_number": 157, + "line_number": 164, "symbol": "{", "message": "Function '{' (115 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/test/test_psnr_hvs_simd.c:157:HISS-04" + "fingerprint": "core/test/test_psnr_hvs_simd.c:164:HISS-04" }, { "rule_id": "HISS-04", @@ -9809,18 +9769,18 @@ { "rule_id": "HISS-04", "file_path": "core/test/test_ssimulacra2_simd.c", - "line_number": 758, + "line_number": 768, "symbol": "{", "message": "Function '{' (76 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/test/test_ssimulacra2_simd.c:758:HISS-04" + "fingerprint": "core/test/test_ssimulacra2_simd.c:768:HISS-04" }, { "rule_id": "HISS-04", "file_path": "core/test/test_ssimulacra2_simd.c", - "line_number": 864, + "line_number": 874, "symbol": "test_ptlr_one", "message": "Function 'test_ptlr_one' (104 LOC) exceeds HISS-04 / NASA Rule 4 limit of 60 LOC", - "fingerprint": "core/test/test_ssimulacra2_simd.c:864:HISS-04" + "fingerprint": "core/test/test_ssimulacra2_simd.c:874:HISS-04" }, { "rule_id": "HISS-04", diff --git a/docs/state.md b/docs/state.md index e9b887299..06d97f3e2 100644 --- a/docs/state.md +++ b/docs/state.md @@ -458,9 +458,7 @@ landed fix yet._ | **T-CI-MSVC-CUDA-SHARED-CHECK-NAME-2026-09-18** | **Two jobs reported the required check name `Windows MSVC+CUDA`.** Since #1286 (`f93a0037f`) shortened the display names, the required lane in `libvmaf-build-matrix.yml` (ADR-0121) and the `build.yml` Windows row shared it. The aggregator keeps one check run per name, the newest (`newestByName()`), so either job could mask the other's failure: on master `7cc0cc91b` the `build.yml` run was cancelled and only the matrix run's success counted. `scripts/ci/check-aggregator-names.sh` compared name sets and could not see a duplicate. **FIXED**: the `build.yml` job is `Windows MSVC+CUDA (full)` (the maintainer's choice; the required name and the ruleset are untouched), and the gate now fails when more than one job reports a required name. It ignores step names and workflow titles; `scripts/ci/tests/test-check-aggregator-names.sh` covers the shared-name case and runs in `rule-enforcement.yml`. | [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md) | `ci/retire-i686-lane` | 2026-09-19 | closed | | **T-AI-ONNX-IR14-UNLOADABLE-2026-09-18** | **The Tiny AI test job failed on master once `onnx` 1.23.0 was released.** `ai/pyproject.toml` allowed `onnx>=1.22.0,<2.0`, so CI resolved 1.23.0, whose `helper.make_model()` writes IR version 14 by default. The pinned `onnxruntime` 1.30.0 loads at most IR 13, so every test that builds a model with onnx and runs it in ORT failed with `Unsupported model IR version: 14, max supported IR version: 13` (21 tests across `test_cross_backend`, `test_learned_filter_audit`, `test_profile`, `test_bisect_model_quality` and the PTQ round trip). The same mismatch would make any model built in-repo with onnx 1.23 unloadable by the runtime the fork ships. Reproduced in isolation: the same Relu model is IR 14 and rejected with onnx 1.23.0, IR 13 and accepted with 1.22.0. **FIXED**: `onnx>=1.22.0,<1.23` in `ai/pyproject.toml` and `tools/ensemble-training-kit/pyproject.toml`, with a comment tying the cap to onnxruntime's IR limit. | — | `fix/onnx-ir-cap` | 2026-09-18 | closed | | **T-NO-ASM-SIMD-TEST-WARNINGS-2026-09-18** | **A `-Denable_asm=false` build emitted ~30 `-Wunused-function` / `-Wunused-const-variable` / `-Wunused-variable` warnings in SIMD parity-test files.** Ten files under `core/test/` (`test_vif_simd.c`, `test_ssimulacra2_simd.c`, `test_speed_simd.c`, `test_psnr_hvs_simd.c`, `test_ms_ssim_decimate.c`, `test_motion_v2_simd.c`, `test_iqa_convolve.c`, `test_integer_ssim_simd.c`, `test_cambi_simd.c`, `test_cambi.c`) defined scalar-reference helpers, fixture builders, and lookup tables/variables unconditionally while every caller sat under an ISA guard (`ARCH_X86`, `HAVE_AVX512`, `ARCH_AARCH64`, or the `ARCH_X86 \|\| ARCH_AARCH64` union `run_tests()` already uses). Measured on an i686 build; reproduces on x86-64 with asm off too. **FIXED**: each helper moved under the exact union of ISA conditions its callers use, walking the full dependency chain so the warning does not relocate to a `pick_*` dispatcher or `ref_*` scalar reference one level down (`test_ssimulacra2_simd.c` needed one guard spanning its whole pick/ref/test helper section for this reason). Nothing deleted, no `(void)`-cast or `[[maybe_unused]]` suppression. Verified zero warnings and an identical test count on `-Denable_asm=false` x86-64, default x86-64, and an aarch64 cross build under `qemu-aarch64-static`. The aarch64 build's two `vif_neon.c` `-Wunused-but-set-variable` warnings (dead `i_dst_stride` counters in the VIF statistic kernels) are fixed on the same branch, and `vif_neon.c` is split into helpers that clear its 36 clang-tidy findings with NEON output byte-identical under qemu (68 of 68 runs). | none | `fix/no-asm-test-warnings` | 2026-09-18 | closed | -| **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for four months after ADR-0728 removed it.** ADR-0728 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The other ADR-0728 removals the same merge restored are audited separately. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0728](adr/0728-native-build-sunset.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for almost four months after ADR-0691 removed it.** ADR-0691 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row, and ADR-0728 (#52, `bfd4c436b`) restated the removal in docs without touching a workflow. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The rest of the matrix is recorded as it runs in ADR-1259. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0691](adr/0691-vmafx-drop-legacy-build-paths.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | -| **T-CI-I686-LANE-RESURRECTED-2026-09-18** | **The `Ubuntu i686 gcc` lane ran for almost four months after ADR-0691 removed it.** ADR-0691 (#1564, `9aa008e70`) retired 32-bit x86 and its lane; the same day, the `libvmaf/` → `core/` rename merge `384d97d03` ("merge-resolves 42 master commits") restored the row, and ADR-0728 (#52, `bfd4c436b`) restated the removal in docs without touching a workflow. It then ran on every PR, compile-only with `-Denable_asm=false`, and ADR-1234 built preflight's `m32` stage on it. It tested nothing: run natively, i686 fails 2 tests without asm and 7 with it (x87 excess precision), and 3 x86-64-only intrinsics sat behind its `-Denable_asm=false`. **FIXED**: the maintainer kept the fork 64-bit only, so the row, its dependency step and its `matrix.i686` conditions are removed, and preflight drops `m32` (ADR-1258). The rest of the matrix is recorded as it runs in ADR-1259. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md), [ADR-0691](adr/0691-vmafx-drop-legacy-build-paths.md), [ADR-1259](adr/1259-ci-build-matrix-as-it-runs.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-X86-64-ONLY-INTRINSICS-2026-09-18** | **The x86 SIMD sources did not compile for 32-bit x86 with asm enabled** (the cause of [Netflix#1481](https://github.com/Netflix/vmaf/issues/1481)). `adm_avx2.c` and `adm_avx512.c` called `_mm_extract_epi64` on 12 lines each, beside an `extract_epi64()` fallback that only covered the 256-bit form, and `psnr_avx2.c` called `_mm_cvtsi128_si64`. GCC declares both only on x86-64. **FIXED**: `extract_epi64_128()` (`_mm_extract_epi64` on x86-64, a 32-bit fallback otherwise) and `_mm_storel_epi64`. An i686 build with asm then compiles warning-free. x86-64 code is unchanged (fast suite 137/137; clang-tidy and cppcheck clean). 32-bit stays unsupported (ADR-0728, ADR-1258), so this is portability hygiene. | [ADR-1258](adr/1258-keep-64-bit-only-retire-i686-lane.md) | `ci/retire-i686-lane` | 2026-09-18 | closed | | **T-PREFLIGHT-M32-FALSE-FAILURES-2026-09-18** | **`scripts/dev/preflight.sh`'s 32-bit sweep failed on code the i686 lane never compiles.** It swept every changed C file with `gcc -m32`, including `arm64/`, `x86/` SIMD and GPU trees, which the `Ubuntu i686 gcc` lane skips because it configures `-Denable_asm=false` and no GPU backend. Its missing-header filter also knew only clang's `file not found` and the German gcc text, while the script runs under `LC_ALL=C`, so gcc's `No such file or directory` (e.g. `arm_neon.h`) counted as a failure. Seen on the `fix/adm-cm-simd-bitexact` branch. **FIXED**: the stage skips those trees and accepts the English message. The underlying i686+asm gap stays as Netflix#1481 / ADR-0151 describe it. | none | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | | **T-EXPORTED-SYMBOLS-ASAN-GNU-LD-2026-09-18** | **`check_exported_symbols` failed in a GNU-ld ASan build.** GNU ld exports the linker-defined `__start_asan_globals` / `__stop_asan_globals` bounds of ASan's metadata section from `libvmaf.so`; lld, which the CI sanitizer lane uses, hides them. They are sanitizer runtime artefacts, not API. **FIXED**: the checker treats `__start_` / `__stop_` bounds of the ASan, HWASan and SanitizerCoverage sections as runtime-owned; any other section bound still fails. Also cleared the file's three ruff findings. | [ADR-0379](adr/0379-libvmaf-symbol-visibility.md) | `fix/preflight-m32-asan-symbols` | 2026-09-18 | closed | From d7b90b12212719627bea4f7c65b5e3487be9131f Mon Sep 17 00:00:00 2001 From: Lusoris Date: Sat, 19 Sep 2026 13:35:46 +0200 Subject: [PATCH 15/15] docs(adr): regenerate the ADR index after the rebase The rebase left the rendered README carrying both the pre-correction ADR-1258 row and the corrected one, while _index_fragments/ held only the corrected fragment. Regenerated with concat-adr-index.sh and generate-adr-by-tag.sh, which the docs freshness gate compares against. --- docs/adr/README.md | 1 - docs/adr/by-tag/fork-local.md | 3 ++- docs/adr/by-tag/index.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/adr/README.md b/docs/adr/README.md index dc71a2bd4..f21d49863 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -1072,7 +1072,6 @@ ADRs may exist there for local session continuity, but the tracked | [ADR-1250](1250-eupl-fork-relicense.md) | Licence fork-authored code under EUPL-1.2 and leave every file that carries someone else's code on the terms it already has, with that code's copyright notice restored. 946 files declared `SPDX-License-Identifier: BSD-3-Clause-Plus-Patent`, which is not a real identifier, while fork code otherwise spread across five licences plus 223 prose-only grants and 147 files with no notice at all. Which files move is decided by provenance and applied by `scripts/dev/relicense_fork_files.py`: no counterpart path or name upstream, no copyright naming anyone else, no licence the fork has not used for its own work, no outside contributor in its history, and — recorded in a reviewed `relicense_provenance.toml` — it does not carry ported or copied code. That moved 1,522 files and left 269 on their terms as carriers of Netflix, NVIDIA, libjxl, Xiph or IQA code, plus 526 more the earlier vetoes catch. The 123 dual `... OR MIT` Go files become plain EUPL-1.2, since an MIT option would defeat the reciprocity. Because `libvmaf.so` links fork and inherited code together, the shipped library is effectively copyleft from here on: a breaking change for permissive consumers, who can use Netflix upstream instead. | Proposed | license, compliance, process, docs, breaking-change | | [ADR-1249](1249-praetor-governance-adoption.md) | Adopt the praetor / HISS-16 governance system before `1.0.0-rc.1` as scaffolding and a ratchet only: declarations, the six compiled agent-context projections, generated policy artefacts and one required CI gate land now, while the five pre-migration epic tasks (#1444–#1448) wait until after the release and no product code is refactored for them. `AGENTS.md` becomes a harness of about 250 lines in the engine's internal register, which `compile-context --verify` and `audit` lint from praetor `e4b35cb` on; its hard rules and rebase-sensitive invariants move to `docs/development/` pages that the harness imports, and the rules only the hand-written `CLAUDE.md` carried move with them. The reviewer personas gain canonical sources in `.agents/agents/`. Lefthook owns the `pre-commit` and `pre-push` hooks and delegates both to the pre-commit framework, while the ADR-1241 dispatchers keep `commit-msg` and `pre-rebase`, superseding only ADR-0924's conclusion that a second hook manager is unacceptable. `sync --remote` stays unused: the engine hardcodes `main` while this fork's default branch is `master`. | Proposed | ci, process, agents, tooling, governance, docs, workspace | | [ADR-1256](1256-cambi-spatial-mask-simd-dispatch.md) | CAMBI's spatial-mask row kernels (`compute_dp_row`, `compute_mask_row`, from upstream `86da14d03`) are dispatched per ISA only where they measurably beat scalar: dp row on AVX2 / AVX-512 / NEON, mask row on AVX2 / AVX-512. Upstream's AVX2 dp row is 0.64–0.74x of scalar under Clang and icx, so the fork keeps a single add on the loop-carried chain (1.8–2.3x AVX2, 2.6–3.2x AVX-512 at 1080p on Zen 5); the AVX2 mask row compares with a 2^31 bias so it is exact for every input. `compute_mask_row_neon` stays parity-tested but undispatched because GCC and Clang already compile the scalar loop to the same NEON code. Scores are byte-identical. | Accepted | simd, avx2, avx512, neon, cambi, perf, upstream-port, fork-local | -| [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0728 decided. The `Ubuntu i686 gcc` lane it removed had come back by accident in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151 is marked superseded by ADR-0728. | Accepted | build, ci, x86, netflix-upstream, fork-local | | [ADR-1258](1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only, as ADR-0691 decided and ADR-0728 restated in docs. The `Ubuntu i686 gcc` lane ADR-0691 removed (`9aa008e70`) had come back the same day in the `libvmaf/` → `core/` rename merge (`384d97d03`) and ran compile-only, with asm off and no tests. An i686 run showed 7 x87 test failures and 3 x86-64-only intrinsics it never exercised. The lane, its dependency step and preflight's `m32` stage (amending ADR-1234's stage list) are removed. The three intrinsic calls get 32-bit-safe forms as hygiene, with no support promise. ADR-0151, whose lane ran until now, is superseded by this ADR. | Accepted | build, ci, x86, netflix-upstream, fork-local | | [ADR-1259](1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs. ADR-0689 and ADR-0691 removed lanes that the layout rename merge `384d97d03` restored about an hour later; ADR-0710 added `build.yml` and `sanitizers.yml` but removed nothing; ADR-0728's commit touched no workflow. All four are superseded. A table lists every lane in `libvmaf-build-matrix.yml` and `build.yml` with its required/advisory status and owning ADR. Maintainer decisions: the fork stays 64-bit only (ADR-1258); `Ubuntu SYCL`, `Ubuntu SYCL+CUDA`, `Ubuntu gcc static`, `Ubuntu CUDA static` and `macOS clang+DNN` stay, not required; `Cppcheck` and the three `Sanitizers (...)` jobs stay required. Records that two jobs report the required name `Windows MSVC+CUDA`. | Accepted | ci, build, fork-local | | [ADR-1247](1247-scorecard-exact-head-gates.md) | Bind Scorecard gates to their measured source and scope | Accepted | ci, security, supply-chain | diff --git a/docs/adr/by-tag/fork-local.md b/docs/adr/by-tag/fork-local.md index 944a1b887..e6b686042 100644 --- a/docs/adr/by-tag/fork-local.md +++ b/docs/adr/by-tag/fork-local.md @@ -3,7 +3,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines to update. -406 ADR(s) carry this tag. +407 ADR(s) carry this tag. | ID | Title | |----|-------| @@ -411,5 +411,6 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh`. Edit ADR `Tags:` lines | [ADR-1244](../1244-merge-train-ownership-and-validation.md) | Guard merge-train ownership and exact-head validation | | [ADR-1251](../1251-renovate-draft-automerge-deadlock.md) | Renovate opens automerge-eligible and security bumps ready for review | | [ADR-1252](../1252-solo-maintainer-declared-bypass.md) | Declare the single maintainer's bypass actor | +| [ADR-1256](../1256-cambi-spatial-mask-simd-dispatch.md) | Dispatch CAMBI's spatial-mask row SIMD kernels only where they measurably beat scalar | | [ADR-1258](../1258-keep-64-bit-only-retire-i686-lane.md) | Keep the fork 64-bit only; retire the resurrected i686 lane | | [ADR-1259](../1259-ci-build-matrix-as-it-runs.md) | Record the CI build matrix as it actually runs | diff --git a/docs/adr/by-tag/index.md b/docs/adr/by-tag/index.md index 7f605a2d8..37cea908e 100644 --- a/docs/adr/by-tag/index.md +++ b/docs/adr/by-tag/index.md @@ -215,7 +215,7 @@ Auto-generated by `scripts/docs/generate-adr-by-tag.sh` from each ADR's `Tags:` | [float_adm](float_adm.md) | 1 | | [float_moment](float_moment.md) | 1 | | [fork-internal](fork-internal.md) | 2 | -| [fork-local](fork-local.md) | 406 | +| [fork-local](fork-local.md) | 407 | | [fork-policy](fork-policy.md) | 1 | | [fp64](fp64.md) | 1 | | [fr](fr.md) | 1 |