Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,14 @@ jobs:
with:
submodules: recursive
persist-credentials: false
# Full history + tags: core/include/meson.build derives VMAF_VERSION
# from `git describe --tags --match 'v*.*.*'`, which needs a reachable
# tag and the commit distance to it. The default fetch-depth of 1
# fetches neither, so the build silently fell back to the project
# version and test_output's test_vmaf_version had no real version to
# check. libvmaf-build-matrix.yml already checks out at depth 0 for
# the same reason; this keeps the two workflows consistent.
fetch-depth: 0

# ── ccache (Linux + macOS) ──────────────────────────────────────
- name: Restore ccache
Expand Down Expand Up @@ -461,6 +469,16 @@ jobs:
./test/test_hip_smoke

# ── Windows CPU tests ─────────────────────────────────────────────
# test_output joins this list deliberately: it is the only test that
# asserts VMAF_VERSION looks like a version, and it ran on no Windows
# leg at all, so this job could not catch a regression in the version
# string. The test already carries a GetTempPathA/GetTempFileNameA
# path for exactly this platform.
# `|| exit /b 1` is required, not decorative: GitHub runs `shell: cmd`
# as `%ComSpec% /D /E:ON /V:OFF /S /C "CALL ..."`, so the step result is
# the errorlevel of the *last* command alone — without it, a failing
# test anywhere but the end of the list is silently swallowed. /V:OFF
# also rules out `!ERRORLEVEL!`, so `||` is the portable check.
- name: Run CPU tests (Windows)
if: matrix.windows
shell: cmd
Expand All @@ -470,10 +488,10 @@ jobs:
test_dict test_ref test_log test_thread_pool test_cpu
test_feature test_cli_parse test_model test_predict
test_ciede test_cambi test_psnr test_luminance_tools
test_feature_collector test_framesync
test_feature_collector test_framesync test_output
) do (
echo === %%t ===
test\%%t.exe
test\%%t.exe || exit /b 1
)

# ── Artifact ──────────────────────────────────────────────────────
Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24897,6 +24897,14 @@ legs. (ADR-0603, triggered by Renovate PR #1402)
([ADR-0562](docs/adr/0562-local-explainer-hang-fix.md))


- Fixed `vmaf --version`, the JSON/XML `version` field and `vmaf_version()`
reporting a bare commit abbreviation (e.g. `abafdfc`) instead of a version
whenever the build tree could not reach a `v*.*.*` tag — a shallow CI
checkout, a tarball export, or a worktree whose `.git` is a file. The
version is now always a real version string: `v3.1.0-2417-gabcdef0` when a
tag is reachable, and the project version otherwise.


Fix three CERT MEM04-C realloc OOM defects in vendored libsvm (core/src/svm.cpp):
Cache::get_data, svm_group_classes, and svm_check_parameter all overwrote the source
pointer with the realloc return value; replaced with save-temp/check-NULL/abort idiom
Expand Down Expand Up @@ -25687,6 +25695,12 @@ Restores the VK-1 + VK-2 perf fix originally landed in PR #879.
Windows MSVC + oneAPI SYCL builds.


- Fixed the Windows CI test step silently passing when any test other than the
last one in its list failed. GitHub runs `shell: cmd` with `/V:OFF`, so the
step reported only the final executable's exit code; each test is now
checked with `|| exit /b 1`. `test_output` was also added to that list.


- **`y4m_convert_411_422jpeg` 1-byte heap-buffer-overflow on
4:1:1 streams whose destination chroma row reduces to a single
pixel (`dst_c_w == 1`).** The Daala-derived 4:1:1 → 4:2:2-jpeg
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,7 @@ lint-sh:
$(call require-tool,shellcheck,your package manager, e.g. pacman -S shellcheck)
shellcheck $$(git ls-files '*.sh')
@scripts/ci/check-default-model-single-source.sh
@scripts/ci/check-vcs-version-not-bare-sha.sh

# Markdown lint (ADR-0866). Default scope is the touched-file delta vs
# origin/master so the ~6.2k pre-existing-warning tail (ADR-0864) doesn't
Expand Down
6 changes: 6 additions & 0 deletions changelog.d/fixed/vcs-version-bare-sha.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- Fixed `vmaf --version`, the JSON/XML `version` field and `vmaf_version()`
reporting a bare commit abbreviation (e.g. `abafdfc`) instead of a version
whenever the build tree could not reach a `v*.*.*` tag — a shallow CI
checkout, a tarball export, or a worktree whose `.git` is a file. The
version is now always a real version string: `v3.1.0-2417-gabcdef0` when a
tag is reachable, and the project version otherwise.
4 changes: 4 additions & 0 deletions changelog.d/fixed/windows-ci-swallowed-test-failures.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
- Fixed the Windows CI test step silently passing when any test other than the
last one in its list failed. GitHub runs `shell: cmd` with `/V:OFF`, so the
step reported only the final executable's exit code; each test is now
checked with `|| exit /b 1`. `test_output` was also added to that list.
21 changes: 20 additions & 1 deletion core/include/meson.build
Original file line number Diff line number Diff line change
@@ -1,10 +1,29 @@
# Revision file (vcs_version.h) generation
vmaf_git_dir = join_paths(libvmaf_src_root + '/../', '.git')

# `--always` is deliberately NOT passed. With it, a checkout that cannot reach
# a `v*.*.*` tag — a shallow CI clone (`actions/checkout` defaults to
# fetch-depth 1, which fetches no tags), a tarball export, or a worktree whose
# `.git` is a file rather than a directory — still exits 0 and prints a bare
# abbreviated commit name. That name then becomes VMAF_VERSION verbatim, so
# `vmaf --version`, the JSON/XML `version` attribute and vmaf_version() all
# report something like "abafdfc" instead of a version. Worse, it is silent
# and intermittent: roughly one commit in a thousand abbreviates to seven hex
# characters that happen to be all letters, which is how it first surfaced
# (merge commit abafdfcc3c… -> "abafdfc", no digit at all).
#
# Without `--always`, git exits non-zero in exactly those cases and meson
# substitutes `fallback` instead — see mesonbuild/scripts/vcstagger.py, which
# catches any exception from the command and uses the fallback string. The
# fallback is spelled out rather than left implicit so the intent survives a
# meson upgrade. When a tag *is* reachable the `--long` form still embeds the
# commit (v3.2.1-42-gabafdfc), so no provenance is lost in the normal case.
rev_target = vcs_tag(command: [
'git', '--git-dir', vmaf_git_dir,
'describe', '--tags', '--long',
'--match', 'v*.*.*', '--always'
'--match', 'v*.*.*'
],
fallback: meson.project_version(),
input: 'vcs_version.h.in',
output: 'vcs_version.h'
)
Expand Down
15 changes: 15 additions & 0 deletions docs/rebase-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@
# Rebase notes

<!-- markdownlint-disable MD001 MD003 MD004 MD007 MD013 MD018 MD022 MD024 MD025 MD026 MD028 MD029 MD031 MD032 MD033 MD036 MD037 MD038 MD040 MD041 MD046 MD049 MD050 MD051 MD052 MD053 MD055 MD056 MD058 MD059 -->
## fix/vcs-version-bare-sha — VMAF_VERSION must never be a bare commit SHA (2026-09-03)

- `core/include/meson.build`: upstream Netflix/vmaf carries the same `vcs_tag()` call **with**
`--always`. The fork deliberately drops that flag and pins an explicit
`fallback: meson.project_version()`. On an upstream sync this file will conflict; keep the
fork's side. Restoring `--always` reintroduces the defect where a tagless or shallow checkout
yields a bare abbreviated object name as `VMAF_VERSION`.
`scripts/ci/check-vcs-version-not-bare-sha.sh` fails the build if the flag comes back, so a
careless conflict resolution is caught rather than shipped.
- `.github/workflows/build.yml`: fork-added workflow; no upstream counterpart. The
`fetch-depth: 0` on the checkout is load-bearing (git describe needs tags plus the commit
distance), as is `|| exit /b 1` in the Windows `for` loop (GitHub runs `shell: cmd` with
`/V:OFF`, so without it only the last executable's exit code reaches the step result).
- `scripts/ci/check-vcs-version-not-bare-sha.sh`, `changelog.d/fixed/*`: wholly fork-added.

## gap/hip-bucket-v2 — AMD ROCm HIP backend gap closure (ADR-1154) (2026-09-03)

- `core/src/feature/hip/` and `core/src/hip/`: all touched files (`ciede_hip.c`, `float_adm_hip.c`,
Expand Down
2 changes: 2 additions & 0 deletions docs/state.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<!-- markdownlint-disable MD013 MD037 MD038 MD041 MD060 -->
_Updated: 2026-09-03 (T-VCS-VERSION-BARE-SHA-2026-09-03 closed — `core/include/meson.build` passed `--always` to `git describe`, so a checkout that cannot reach a `v*.*.*` tag still exited 0 and yielded a bare abbreviated object name as `VMAF_VERSION`. `build.yml` checks out at the `actions/checkout` default `fetch-depth: 1` (no tags), so every build on that workflow stamped a commit abbreviation into `vmaf --version`, the JSON/XML `version` field and `vmaf_version()`. Silent until the seven-character abbreviation contains no ASCII digit — (6/16)^7, about one commit in a thousand — which is what `test_output.c::test_vmaf_version` asserts; #1223's merge commit `abafdfcc3c8e…` abbreviates to `abafdfc` and failed two legs of an unrelated PR. Fixed by dropping `--always` (git then exits non-zero and meson substitutes the now-explicit `fallback`) and moving `build.yml` to `fetch-depth: 0`. Master's own `build.yml` legs do run and were green, but only by luck: all 20 most recent master commits abbreviate with a digit, so the defect is latent there too and fires on whichever commit first abbreviates to all letters (PR merge commits give extra rolls). One real coverage gap closed alongside: the Windows leg's whitelist omitted `test_output` — now included — and its `cmd` loop gained `|| exit /b 1` because GitHub's `shell: cmd` (`/V:OFF`) reported only the last executable's errorlevel, discarding any earlier failure. New gate `scripts/ci/check-vcs-version-not-bare-sha.sh` keeps `--always` out across upstream syncs. No ADR (bug fix). Row added to Recently closed.)_
_Updated: 2026-09-03 (code-scanning-open-alerts-and-reaudit — audited 13 open GitHub code-scanning alerts and re-audited 5 warning-level security dismissals without suppressions: (1) fixed MCP unused import and cyclic import in test_smoke_e2e.py / server.py / http_transport.py (Alerts 919, 917, 918); (2) fixed trivial-switch, loop-variable-changed, and constant-comparison in feature_name.cpp / mkdirp.cpp / pdjson.c (Alerts 167, 164, 691); (3) verified and restored UNIX domain socket bind with 0o660 permissions and full test coverage in online_trainer.py (Alert 373); (4) upgraded golang.org/x/crypto to v0.56.0 in go.mod (Alert 4: GO-2026-6354, GO-2026-6355); (5) hardened 3 SHA-1 cache keys with usedforsecurity=False in decorator.py (Alerts 731-733); (6) reported-not-fixed without inoperative suppressions: exact float comparisons in feature_name.cpp / predict.c (Alerts 168, 927; ADR-0138 / ADR-0139), test text-includes in test_luminance_tools.cpp / test_feature.cpp (Alerts 908, 165), compiler probe file in build tree (Alert 932), unimported openpgp advisory GO-2026-5932 (Alert 4 sub-finding), and Scorecard CodeReviewID / CIIBestPracticesID blockers (Alerts 1, 3; ADR-0263). Research-2028.)_
_Updated: 2026-09-03 (T-UPSTREAM-1564-ADM-CM-GPU-BORDER-AND-ROUNDING-2026-09-03 closed — fixed two GPU numerical defects in integer ADM contrast masking kernels (adm_cm.cu and adm_cm.hip): (1) border row selection at i == 0 && top <= 0 by replacing running pointer offsets with explicit absolute row/col indexing {row_top, row_bot, col_l, col_r} and sampling csf_a at row 0 center; (2) distributed rounding shift by enforcing row-level accumulation across columns in 64-bit precision before shifting once per row, eliminating warp-distributed and thread-distributed rounding bias drift. ADR-1167 supersedes ADR-0539. Row added to Recently closed.)_
_Updated: 2026-09-03 (T-DEP-PR-DOC-GATE-EXEMPTION-2026-09-03 closed — exempt strictly dependency-only bot pull requests from Doc-Substance Gate (ADR-0100/0167) and Deep-Dive Deliverables Checklist (ADR-0108) via scripts/ci/classify-dependency-pr.sh; eliminates false-positive red badges on Renovate and Dependabot updates (#1206, #1207, #1212, #1214). ADR-1152. Row added to Recently closed.)_
Expand Down Expand Up @@ -339,6 +340,7 @@ landed fix yet._

## Recently closed

| **T-VCS-VERSION-BARE-SHA-2026-09-03** | `core/include/meson.build` passed `--always` to `git describe --tags --long --match 'v*.*.*'`, which makes git exit 0 and print a bare abbreviated object name when no matching tag is reachable. Meson writes that into `vcs_version.h` verbatim, so on any tagless or shallow checkout — `build.yml` used the `actions/checkout` default `fetch-depth: 1` — `vmaf --version`, the JSON/XML `version` field and `vmaf_version()` all reported a commit instead of a version. Invisible until the abbreviation happens to hold no ASCII digit (≈1 commit in 1000), the one condition `core/test/test_output.c::test_vmaf_version` detects: #1223's merge commit `abafdfcc3c8e…` → `abafdfc` reddened that unrelated PR's Intel LLVM and macOS legs. Dropped `--always` so git fails and meson substitutes the now-explicit `fallback: meson.project_version()`; `build.yml` moved to `fetch-depth: 0`. Master's `build.yml` legs do run and were green, but only because every recent master commit abbreviated with a digit — the defect is latent on master, not absent. Coverage gap closed alongside: the Windows leg omitted `test_output` from its whitelist while its `cmd` loop discarded every non-final test's exit code (`/V:OFF`, no `!ERRORLEVEL!`) — both fixed. Gate: `scripts/ci/check-vcs-version-not-bare-sha.sh`. | none | PR #1266 | 2026-09-03 | closed |
| **T-DEP-CLASSIFIER-TWO-DOT-DIFF-2026-09-03** | `scripts/ci/classify-dependency-pr.sh` diffed `base_sha..head_sha` with two dots. GitHub's `pull_request.base.sha` is the base tip at PR-creation time, so once master moved the range reported every file merged since the branch point on top of the PR's own change — a Renovate PR touching only `deploy/helm/vmafx/values.yaml` measured as 36 files including `core/src/feature/adm_avx2.c`. The classifier then withheld the dependency exemption and every such PR failed the doc gates, unfixable by re-run. Now resolves the fork point with `git merge-base`. The 19 existing tests all fed precomputed `--diff` files and never exercised the git path; case 20 builds a real moved-base repo and fails without the fix. | no ADR: bug fix in an existing gate | fix |
| **T-SYCL-TIDY-LTO-GOLD-PLUGIN-2026-09-03** | Uncovered immediately behind `T-SYCL-TIDY-ZE-LOADER-MISSING-2026-09-03`: with the loader installed, `meson setup build-sycl` succeeds (`Library ze_loader found: YES`, 206 targets) and `Clang-Tidy SYCL (Changed Files, Advisory)` reaches `meson compile` for the first time, where every test binary fails to link with `bfd plugin: LLVM gold plugin has failed to create LTO module: Unknown attribute kind (102) (Producer: 'Intel.oneAPI.DPCPP.Compiler_2026.1.1' Reader: 'LLVM 17.0.6')`. `core/meson.build:11` sets `b_lto=true` as a project `default_option`, so linking runs LTO through the stock `ubuntu-24.04` binutils gold plugin, which is LLVM 17.0.6 and cannot read oneAPI DPC++ bitcode. Pinning an older oneAPI would not help — the mismatch is against the system linker plugin. Both SYCL legs of `libvmaf-build-matrix.yml` already pass `-Db_lto=false`; this job now does the same, and needs no LTO since it only produces codegen outputs and a `compile_commands.json`. Observed on `workflow_dispatch` run 33782036874, job 100737804631. | none | PR #1234 | 2026-09-03 | closed |
| **T-SYCL-TIDY-ZE-LOADER-MISSING-2026-09-03** | `Clang-Tidy SYCL (Changed Files, Advisory)` in `.github/workflows/lint-and-format.yml` failed in `Generate SYCL compile_commands.json`: `meson setup build-sycl` aborted at `cc.find_library('ze_loader', required : true)` (`core/src/meson.build`) with `/usr/bin/ld: cannot find -lze_loader` → `ERROR: C shared or static library 'ze_loader' not found`, so the job died before `meson compile`, before `gen-sycl-compile-commands.py`, and before any TU reached clang-tidy. This is the next defect uncovered once #1227 let the job run its steps at all. The step installs `intel-oneapi-compiler-dpcpp-cpp` and sources `setvars.sh --force` first, but oneAPI ships no Level Zero loader and the stock `ubuntu-24.04` image has neither the library nor `level_zero/ze_api.h`. The runtime package `libze1` is not a fix: `cc.find_library` emits a literal `-lze_loader`, which `ld` resolves against the unversioned `libze_loader.so` symlink only, never the `libze_loader.so.1` SONAME. Fixed by installing `libze-dev` (`noble/universe`, source package `level-zero`), which provides the symlink, the headers and `libze_loader.pc`, and pulls `libze1` transitively. Observed on run 33777315019 job 100722550493 (PR #1223), on a GitHub-hosted `ubuntu-24.04` runner. | none | PR #1234 | 2026-09-03 | closed |
Expand Down
30 changes: 30 additions & 0 deletions scripts/ci/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,36 @@ until master is fixed.
| `classify-dependency-pr.sh` | `rule-enforcement.yml` — `deep-dive-checklist` and `doc-substance-check` jobs ([ADR-1152](../../docs/adr/1152-dependency-pr-gate-exemption.md)) | Reads `$PR_AUTHOR`, `$HEAD_REF`, `$BASE_SHA`, `$HEAD_SHA` from workflow env. The exemption is author-AND-path-gated and must never be widened to a path glob alone. Bot identity requires `renovate[bot]` / `dependabot[bot]` (or `app/renovate` / `app/dependabot`), or a `renovate/*` / `dependabot/*` branch, AND all changed paths must be in the explicit manifest/lockfile allowlist. Bot PRs touching source code must still satisfy both documentation gates. Test suite: `scripts/ci/test-classify-dependency-pr.sh`. |
| `test-classify-dependency-pr.sh` | (local-only fixture driver, not invoked by CI) | Run before pushing changes to `classify-dependency-pr.sh`; exercises the predicate space across dependency-only diffs, mixed source diffs, non-bot authors, and real PR fixtures (#1206, #1207, #1212, #1214). |

## `check-vcs-version-not-bare-sha.sh` invariants

`core/include/meson.build` builds `VMAF_VERSION` from `git describe`, and
upstream Netflix/vmaf spells that call with `--always`. The fork deliberately
does not. With `--always`, git exits 0 even with no reachable `v*.*.*` tag and
prints a bare abbreviated object name, which meson writes into
`vcs_version.h` verbatim — so `vmaf --version`, the JSON/XML `version` field
and `vmaf_version()` all report a commit instead of a version on any shallow
checkout, tarball export, or worktree whose `.git` is a file.

Three properties are load-bearing, and this gate enforces each:

| Property | Why it matters |
| --- | --- |
| No `--always` in the `vcs_tag` command | It is what suppresses the non-zero exit that the fallback path depends on. |
| An explicit `fallback:` | Meson would default it to `meson.project_version()`, but the fallback *is* the tagless path here; spelling it out keeps the intent across meson upgrades. |
| `--match 'v*.*.*'` retained | Without it any tag in the repository can supply the version. |

Two things make the defect easy to reintroduce and hard to notice. It conflicts
with upstream on every sync, so a mechanical "take theirs" resolution restores
`--always`; and it is invisible until the seven-character abbreviation happens
to contain no ASCII digit — about one commit in a thousand — which is the only
condition `core/test/test_output.c::test_vmaf_version` can detect. Assume any
version-string failure on one leg is environmental until you have checked
whether the checkout could reach a tag.

`.github/workflows/build.yml` must therefore keep `fetch-depth: 0` on its
checkout: `git describe --long` needs both the tag objects and the commit
distance to them, and the `actions/checkout` default of 1 supplies neither.

## Calibration table contract (ADR-0234)

`gpu_ulp_calibration.yaml` is the single source of truth for
Expand Down
Loading
Loading