Skip to content

Use merged FAAD3 for default benchmark scoring - #116

Merged
nschimme merged 6 commits into
masterfrom
faad3-default-faad2-ref
Oct 21, 2026
Merged

nschimme merged 6 commits into
masterfrom
faad3-default-faad2-ref

Conversation

@nschimme

@nschimme nschimme commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Use the FAAD3 binary from the sibling FAAC build for default scoring and gapless checks.
  • Keep the pinned FAAD2 2.11.4 build as an optional compare_codecs.py reference supplied through --faad-bin.
  • Document the existing DAB+ and DRC diagnostic probes.

Validation

  • git diff --check
  • bash -n scripts/build_faad2.sh
  • FAAD2 tag archive checksum verified against the downloaded 2.11.4 tag archive.
  • Test suite not run.

Summary by Sourcery

Make merged FAAD3 the default benchmark and gapless decoder while retaining pinned FAAD2 2.11.4 for explicit reference comparisons.

New Features:

  • Use the merged FAAD3 build as the default benchmark and gapless-check decoder, with configurable FAAC build locations.
  • Support optional FAAD2 2.11.4 reference decoding for explicit codec comparisons and FAAD3-versus-FAAD2 benchmark diagnostics.

Bug Fixes:

  • Prevent decoder-version differences from being treated as encoder regressions in FAAC benchmark gating.
  • Make consolidated benchmark summaries omit detailed decoder and transient diagnostics while retaining them in per-rate reports.

Enhancements:

  • Improve decoder comparison reports to identify both binaries and clearly distinguish exact PCM equality from quality evaluation.

Build:

  • Version and isolate the optional FAAD2 2.11.4 build artifact and update its archive source and checksum validation.

CI:

  • Update benchmark CI to build merged FAAD3 and optional FAAD2 reference binaries, and remove unnecessary FAAD2 setup from unrelated workflows.
  • Configure benchmark reporting and gates for informational cross-decoder comparisons and compact consolidated summaries.

Documentation:

  • Document merged FAAD3 default scoring, explicit FAAD2 reference usage, decoder-comparison interpretation, consolidated reporting, and DAB+ DRC diagnostic probes.

Tests:

  • Update decoder-resolution, decoder-diff, and end-to-end tests for FAAD3 defaults and the expanded comparison report format.

@sourcery-ai

sourcery-ai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Sorry @nschimme, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 3 days and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.

@sourcery-ai

sourcery-ai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Reviewer's Guide

Default benchmark scoring and gapless validation now use the merged FAAD3 binary from the sibling FAAC build, while the pinned FAAD2 2.11.4 build remains available only as an explicit compare_codecs.py reference; CI guidance, tests, and documentation are updated accordingly.

Flow diagram for default decoder selection

flowchart TD
    A[Benchmark or gapless check] --> B{Explicit decoder binary supplied?}
    B -->|FAAD3| C[Use supplied FAAD3]
    B -->|No| D["default_faad3_path()"]
    D --> E[$FAAC_BUILD_DIR/frontend/faad]
    E --> F[FAAD3 scoring and gapless validation]
    B -->|Other decoder| G[Use selected decoder]
Loading

File-Level Changes

Change Details Files
Switch default benchmark and gapless decoding from the pinned FAAD2 build to the merged FAAD3 binary produced by the sibling FAAC checkout.
  • Resolve the default decoder from FAAC_SRC/FAAC_BUILD_DIR and require an executable FAAD3 binary.
  • Retain supplied FAAD3 binaries for per-run scoring and remove automatic FAAD2 fallback.
  • Update gapless checks, benchmark help text, comments, and unit tests for FAAD3-based resolution.
codec_bench/decoders.py
gapless.py
phase2_mos.py
phase3_stereo.py
run_benchmark.py
scripts/score_transient.py
utils.py
tests/test_unit.py
Make the pinned FAAD2 2.11.4 build an explicit optional decoder reference rather than part of automatic scoring.
  • Rename and version the generated binary as faad2-2.11.4.
  • Download the tagged 2.11.4 archive and verify its updated checksum.
  • Document passing the build output explicitly to compare_codecs.py via --faad-bin.
scripts/build_faad2.sh
docs/usage.md
Align CI and documentation with FAAD3-based scoring and add diagnostic probe documentation.
  • Update CI and usage guidance to describe the sibling FAAC FAAD3 dependency and custom build-directory variables.
  • Clarify that decoder comparisons use corresponding supplied FAAD3 binaries.
  • Document the existing DAB+ and DRC diagnostic probes and revise gapless metric descriptions.
action.yml
docs/ci.md
docs/metrics.md
docs/scripts.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@nschimme
nschimme merged commit 6e3d546 into master Oct 21, 2026
7 checks passed
@nschimme
nschimme deleted the faad3-default-faad2-ref branch October 21, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant