Skip to content

Bazel: align compiler and Windows toolchain behavior with Make - #3705

Open
napetrov wants to merge 8 commits into
uxlfoundation:mainfrom
napetrov:issue3530-migration
Open

Bazel: align compiler and Windows toolchain behavior with Make#3705
napetrov wants to merge 8 commits into
uxlfoundation:mainfrom
napetrov:issue3530-migration

Conversation

@napetrov

@napetrov napetrov commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Description

This PR closes seven reproducible compiler/toolchain parity gaps found while auditing the Bazel path against the existing Make behavior for issue #3530.

The changes are intentionally kept as seven focused commits. Together they align x86-64 ISA selection across GCC, Clang, ICX, and the MSVC fallback; fix Windows CPU detection and CRT selection; derive the DPC++ archiver from the selected compiler; and make the Windows nightly BAZEL_SH requirement explicit and fail-fast.

This is progress toward #3530, but it does not claim that every Make-only platform or feature is ready for deprecation.

What changed

ISA parity

  • Use -march=skylake-avx512 for GCC SKX objects in both Make and Bazel.
  • Define the missing Clang CPU matrix:
    • SSE2: -march=nocona
    • AVX2: -march=haswell
    • AVX-512: -march=skylake-avx512
  • Propagate ISA flags to materialized MSVC fallback actions while retaining Make parity:
    • SSE2: no /arch option
    • AVX2: /arch:AVX2
    • SKX/AVX-512 dispatch objects: /arch:AVX2

Toolchain/runtime behavior

  • Derive llvm-ar from the selected DPC++ compiler path instead of slicing the unrelated host compiler path; fail configuration when the companion archiver is absent.
  • Build the Windows CPU detector with MSVC and use the .exe output name expected by Windows.
  • Select the Windows CRT through a toolchain feature:
    • optimized/non-debug actions: /MD
    • debug actions: /MDd
  • Remove the former unconditional -MD from the shared Windows ICX flags.

Windows nightly reproducibility

  • Set step-scoped BAZEL_SH to Git for Windows Bash.
  • Fail before Bazel with a clear diagnostic if that executable is missing.
  • Document that Windows Bazel build, test, and analysis commands require BAZEL_SH.

Commit structure

  1. build: enable AVX-512 for GCC skx targets
  2. bazel: define Clang CPU ISA flags
  3. bazel: derive DPC++ archiver from compiler path
  4. bazel: compile CPU detector with MSVC on Windows
  5. bazel: select Windows CRT by build mode
  6. bazel: add MSVC CPU ISA flags
  7. ci: require Bazel shell on Windows nightly

The CRT removal/addition is atomic in commit 5; the workflow and documentation change are paired in commit 7.

Validation

Linux — exact clean PR head

Validated on clean head f7ab546fb85dd1c41098f4132c01060ecd7fbced with Bazel 9.2.0.

  • GCC 15.2, Clang 21.1.8, and ICX 2026.0 ISA action matrices passed.
  • SSE2/AVX2/AVX-512 actions materialized the expected compiler options.
  • GCC, ICX, and ICPX archive actions selected:
    • /opt/intel/oneapi/compiler/2026.0/bin/compiler/llvm-ar
  • Full host release:
PATH=/opt/intel/oneapi/compiler/2026.0/bin:$PATH \
CC=icx bazelisk \
  --output_base=<isolated-output-base> \
  build //:release --release_dpc=false --cpu=all --jobs=2

Result: rc 0, 3,188 actions, 17m50s.

Produced-release gates also passed:

  • ISA coverage: 592 dispatch symbols for each SSE2/AVX2/AVX-512 tier
  • release structure and SONAME checks
  • package metadata: 9 passed, 0 failed
  • MKL linkage probe: no bundled MKL archive objects or exported mkl_ symbols

Windows — cumulative source-equivalent patch

Validated on Windows Server 2025 with:

  • MSVC 19.44.35227
  • Intel oneAPI DPC++/C++ Compiler 2026.0.0
  • Bazel 9.2.0
  • Git for Windows Bash 5.3.9

Source integrity checks passed for the exact eight-file cumulative diff: git diff --check, reverse patch application, file inventory, and SHA-256 preservation.

Materialized MSVC parameters:

Dispatch tier params expected /arch violations
SSE2 390 none 0
AVX2 390 /arch:AVX2 0
SKX/AVX-512 390 /arch:AVX2 0

Materialized CRT parameters:

mode /MD /MDd
opt 1,533 0
dbg 0 1,533

Additional Windows gates:

  • generated CPU detector compiled and executed successfully; output: avx512
  • valid BAZEL_SH guard: rc 0
  • deliberately missing BAZEL_SH guard: rc 1 with the expected diagnostic
  • broad MSVC host build of //examples/daal/cpp:library_version_info_host: rc 0, 1,933 actions
  • full ICX/DPC release:
bazelisk --output_user_root=<isolated-root> build //:release \
  --config=release-dpc --cpu=avx2 --jobs=2 \
  --disk_cache= --remote_upload_local_results=false --verbose_failures

Result: rc 0, 3,294 actions, 49m43s.

Earlier Windows LNK1180/server-reset attempts were traced to a user-level C:\b-cache upload duplicating the multi-gigabyte onedal_core.lib archive until the disk reached zero free bytes. The same source passed after removing only that disposable cache and explicitly disabling cache upload. No compiler/linker source failure remained.

Current main applicability

All seven commits were cherry-picked, in order and without conflicts, onto main@9d920262384e681331fca9c3c4ed481e477ae81b in a disposable checkout.

  • clean status
  • unchanged cumulative diff: 8 files, 69 insertions, 15 deletions
  • workflow YAML assertions passed
  • //:release --release_dpc=false --cpu=all --nobuild passed with ICX/Bazel 9.2.0: rc 0, 4,641 configured targets

Independent review

Two independent passes reviewed the cumulative diff:

  • compiler/toolchain semantics: approve-with-nits; critical 0, should-fix 0
  • CI/integration: approve-with-nits for opening one PR; no code-level blocker

The recommendation was to keep this as one PR with the seven logical commits rather than squash or split it.

Draft / merge gates

This PR intentionally starts as a draft.

Before marking ready/merging:

  • Run and inspect the actual GitHub-hosted Windows Bazel workflow for this pushed head.
  • Confirm all required CI checks are green.
  • Resolve commit sign-off/DCO requirements if reported by CI (no DCO failure was reported).
  • Optional follow-up: an additional exact-head Windows opt/debug runtime smoke was not rerun manually after the validation VM account lock; exact-head hosted Windows CI passed.

Scope boundary

This series does not by itself complete global Make deprecation. Remaining broader items include non-x86 platforms, macOS, OpenRNG and other Make-only switches, real DPC CPU/GPU execution coverage, sanitizer execution, downstream-consumer gates, ABI/release-comparison hardening, performance/build-time/hermeticity gates, and Bazel-backed nupkg/documentation deliverables.


Checklist

Completeness and readability

  • Hard-to-understand behavior is commented, including the intentional MSVC SKX-to-/arch:AVX2 mapping.
  • Windows Bazel setup documentation is updated.
  • No commit sign-off/DCO failure was reported by CI.
  • The series applies to the current base without conflicts.

Testing

  • The changes were tested extensively on Linux and Windows as described above.
  • Required hosted CI passed on the exact PR head.
  • Validation covers materialized compiler parameters, full host/release builds, CPU detection, workflow guards, package structure, and metadata.

Performance

  • No algorithm, numerical path, or dispatch policy is changed; this PR corrects compiler/toolchain selection for existing dispatch tiers. Runtime benchmark changes are not expected.
  • No new measurable algorithm functionality is introduced, so no benchmark-suite extension is required.

@napetrov
napetrov marked this pull request as ready for review July 23, 2026 14:17
Copilot AI review requested due to automatic review settings July 23, 2026 14:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Aligns Bazel’s compiler/toolchain behavior with the existing Make-based contract to reduce parity gaps during the ongoing Make→Bazel migration work (issue #3530), with a focus on x86-64 ISA flag matrices, Windows toolchain/runtime behavior, and Windows CI reproducibility.

Changes:

  • Updates GCC/Clang ISA flag selection in Bazel (and GCC SKX in Make) to ensure distinct SSE2/AVX2/AVX-512 compilation targets (e.g., -march=skylake-avx512 for AVX-512).
  • Improves toolchain correctness: derives the DPC++ archiver from the DPC++ compiler path (fail-fast if missing), fixes Windows CPU detector compilation/output naming, and selects Windows CRT flags (-MD vs -MDd) via a dedicated toolchain feature.
  • Ensures Windows Bazel nightly builds are reproducible by explicitly setting and validating BAZEL_SH, and updates Windows Bazel setup documentation accordingly.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
dev/make/compiler_definitions/gnu.32e.mk Updates GCC SKX target to -march=skylake-avx512 to match the Bazel ISA matrix.
dev/bazel/toolchains/cc_toolchain_lnx.bzl Derives llvm-ar from the selected DPC++ compiler location and fails configuration when absent.
dev/bazel/toolchains/cc_toolchain_config_win.bzl Adds a toolchain feature to select /MD vs /MDd behavior (via -MD/-MDd) based on build mode.
dev/bazel/README.md Clarifies that Windows Bazel build/test/analysis require BAZEL_SH (not just bazel test).
dev/bazel/flags.bzl Aligns GCC/Clang ISA flags (including AVX-512) and removes unconditional Windows -MD from shared ICX flags.
dev/bazel/config/config.bzl Builds the CPU detector with MSVC on Windows and uses the expected .exe output name.
dev/bazel/cc/compile.bzl Propagates MSVC /arch settings to CPU-tier compilation actions to match Make’s dispatch-object behavior.
.github/workflows/nightly-build.yml Sets and validates BAZEL_SH explicitly for the Windows nightly Bazel release step (fail-fast if missing).

Comment thread dev/bazel/config/config.bzl Outdated
Signed-off-by: Ubuntu <ubuntu@ip-172-31-3-43.ec2.internal>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants