Bazel: add AArch64/RISC-V64 cross-compile and OpenRNG backend support - #3719
Open
napetrov wants to merge 5 commits into
Open
Bazel: add AArch64/RISC-V64 cross-compile and OpenRNG backend support#3719napetrov wants to merge 5 commits into
napetrov wants to merge 5 commits into
Conversation
Extends the oneDAL Bazel build (#3530 Phase D/P1) to Linux AArch64 and RISC-V64, matching Make's existing ref-backend support, and wires the OpenRNG library as an alternative RNG backend on top of the ref/OpenBLAS math backend. - Detect host/target arch from CC/CXX cross-compiler triple prefixes (dev/bazel/toolchains/common.bzl, cc_toolchain.bzl) instead of the previously hardcoded intel64, and register arch-specific exec/target-compatible toolchains and platforms (@config//:linux_aarch64, @config//:linux_riscv64). - Extend cpu_info to allow "sve"/"rv64" as single-variant ISA values on ARM/RISC-V (no runtime CPU dispatch there, unlike x86's sse2/avx2/avx512), resolved from the repository-rule-detected arch since build_setting rules can't see --platforms. - Add -march=armv8-a+sve / -march=rv64gc_v1p0_zvl128b to every compile action on those arches (flags.bzl), and DAAL_CPU=sve/rv64 and __CPU_TAG_ARMV8SVE__/__CPU_TAG_RISCV64GC__ defines (daal.bzl, dal.bzl) matching the existing C++ dispatch macros. - Add dev/bazel/deps/openrng.bzl (mirrors openblas.bzl) and a rng_backend config flag; selecting openrng adds OPENRNG_BACKEND and links @openrng//:openrng alongside OpenBLAS, matching dev/make/deps.ref.mk's RNG_OPENRNG. Validated with real aarch64-linux-gnu-gcc/riscv64-linux-gnu-gcc cross toolchains and qemu-user execution (full //cpp/daal:core_dynamic compiles, links, and runs under qemu for both arches), and confirmed no regression on the native x86_64 MKL build.
Adds Bazel build lanes for the new cross-compile support: - .github/workflows/ci-aarch64.yml: bazel build job on the existing GitHub-hosted arm64 runner, both ref and OpenRNG rng_backend. - .ci/pipeline/ci.yml: new LinuxBazelGNU_OpenBLAS_rv64 job, cross- compiling with the apt riscv64-linux-gnu-gcc toolchain (mirrors LinuxMakeLLVM_OpenBLAS_rv64's OpenBLAS build, minus the clang-only sysroot dance which plain gcc cross packages don't need). - .ci/env/bazelisk.sh: Bazelisk itself runs on the CI exec host, so pick the amd64/arm64 asset matching `uname -m` instead of always downloading bazelisk-linux-amd64 (would silently exec-format-fail on the arm64 runner). Also fixes a real bug in dev/bazel/cc/compile.bzl surfaced while testing //cpp/oneapi/dal:dynamic on the new arches: `cpu_to_file_dict` only ever contains entries for the x86 ISA IDs sse2/avx2/avx512 (from the _nrh/_hsw/_skx filename-suffix map), so compiling any *_cpu module with per-ISA special files (e.g. algo/jaccard) for cpu="sve"/"rv64" raised "key not found in dictionary". Falls back to the general _cpu file list for arches with no such per-ISA source split.
Contributor
Author
Independent review (4 parallel agents: Bazel/Starlark correctness, Make-parity, CI/infra, security)Fixed: Checked and ruled out (no action needed):
Noted, not blocking:
|
get_cross_tool_prefix stripped only the trailing "-<toolname>" segment
from CC/CXX's basename, so a compiler set to a versioned Debian/Ubuntu
package binary (e.g. CC=aarch64-linux-gnu-gcc-13, installed when only
the versioned gcc-13-aarch64-linux-gnu package is present without the
unversioned gcc-aarch64-linux-gnu meta-package) left a mangled prefix
("aarch64-linux-gnu-gcc-" instead of "aarch64-linux-gnu-"), causing
_find_tools to search PATH for a nonexistent ar/strip
(aarch64-linux-gnu-gcc-ar) and fail the whole toolchain repo rule.
Found via team review of #3719.
The gnu riscv64 cross-compiler on the ubuntu-24.04 image is gcc-13, which lacks the RVV segment intrinsics (vfloat32m4x2_t, __riscv_vlseg2e32_v_f32m4x2) used by OpenBLAS' RISCV64_ZVL128B kernels, so it fails to build them. Switch the OpenBLAS build in LinuxBazelGNU_OpenBLAS_rv64 to clang + a riscv64 sysroot, matching the already-green LinuxMakeLLVM_OpenBLAS_rv64 job, and reuse the same cache key so both jobs share the prebuilt OpenBLAS. The oneDAL Bazel build itself still cross-compiles with riscv64-linux-gnu-gcc; OpenBLAS is consumed as a static C-ABI archive, so its build compiler is irrelevant.
napetrov
marked this pull request as ready for review
July 29, 2026 19:59
napetrov
requested review from
Alexandr-Solovev,
ahuber21,
avolkov-intel,
david-cortes-intel,
ethanglaser,
homksei and
icfaust
as code owners
July 29, 2026 19:59
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends oneDAL’s Bazel build to support Linux AArch64 and RISC-V64 (including cross-compilation) and introduces an OpenRNG-based RNG backend option to match Make’s ref-backend capabilities, plus CI coverage for the new paths.
Changes:
- Add Linux AArch64/RISC-V64 platform + toolchain detection/registration and single-variant ISA handling (SVE/RV64).
- Introduce
--rng_backend=with an OpenRNG prebuilt dependency and wire it into DAAL builds. - Extend CI (GitHub Actions AArch64 Bazel lane + Azure rv64 cross-build) and fix Bazelisk host-arch asset selection.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| MODULE.bazel | Registers a new prebuilt @openrng repo rule. |
| dev/bazel/toolchains/extra_toolchian_lnx.tpl.BUILD | Makes the extra-tools toolchain usable across non-x86 targets (arch-independent tool). |
| dev/bazel/toolchains/common.bzl | Adds host/target arch detection and cross-tool prefix parsing. |
| dev/bazel/toolchains/cc_toolchain.bzl | Plumbs detected host/target arch IDs into toolchain requirements. |
| dev/bazel/toolchains/cc_toolchain_lnx.tpl.BUILD | Parameterizes toolchain exec/target CPU constraints for cross builds. |
| dev/bazel/toolchains/cc_toolchain_lnx.bzl | Uses cross-prefix binutils and avoids -B injection for GNU cross compilers. |
| dev/bazel/README.md | Documents Bazel cross-compilation for AArch64/RISC-V and --rng_backend. |
| dev/bazel/flags.bzl | Adds single-variant -march handling and ISA IDs for ARM/RISC-V. |
| dev/bazel/deps/openrng.tpl.BUILD | Adds BUILD template for a prebuilt OpenRNG static library + headers. |
| dev/bazel/deps/openrng.bzl | Adds repo rule wrapper for prebuilt OpenRNG dependency. |
| dev/bazel/dal.bzl | Adds oneAPI __CPU_TAG__ defines for SVE/RV64. |
| dev/bazel/daal.bzl | Adds DAAL CPU defines for SVE/RV64 and an OpenRNG backend define. |
| dev/bazel/config/config.tpl.BUILD | Adds rng_backend flag and AArch64/RISC-V target platform definitions. |
| dev/bazel/config/config.bzl | Extends cpu_info to support ARM/RISC-V single-variant ISA selection via an arch attribute. |
| dev/bazel/cc/compile.bzl | Fixes CPU-specialized source selection to avoid KeyError on non-x86 ISA IDs. |
| cpp/daal/BUILD | Wires OpenRNG dependency/headers into DAAL builds under the new flag. |
| .github/workflows/ci-aarch64.yml | Adds a Bazel build/test job on GitHub-hosted AArch64 runner (ref + openrng). |
| .ci/pipeline/ci.yml | Adds an Azure job for rv64 cross-compiling and ELF validation. |
| .ci/env/bazelisk.sh | Downloads the Bazelisk binary matching the CI host architecture (amd64 vs arm64). |
| .bazelrc | Adds --rng_backend flag alias. |
Comments suppressed due to low confidence (1)
cpp/daal/BUILD:90
- Similarly to the core deps,
@openrng//:headersis added whenever--rng_backend=openrngis set, regardless of backend. If OpenRNG is ref-only, gate this header dependency under@config//:backend_refto keep MKL builds independent of OPENRNGROOT.
}) + select({
"@config//:rng_backend_openrng": [ "@openrng//:headers" ],
"//conditions:default": [],
}),
Comment on lines
+70
to
+76
| # oneDAL's arch IDs, matching ARCH_DIR_ONEDAL in dev/make/function_definitions/*.mk. | ||
| _ARCH_ID_BY_TRIPLE_PREFIX = { | ||
| "aarch64": "arm", | ||
| "riscv64": "riscv64", | ||
| "x86_64": "intel64", | ||
| "amd64": "intel64", | ||
| } |
Comment on lines
+109
to
+112
| # `-march` applies to every compile action, not only the | ||
| # `_cpu`-suffixed ones that go through get_cpu_flags(). Matches | ||
| # COMPILER.all.gnu / COMPILER.lnx.clang in | ||
| # dev/make/compiler_definitions/{gnu,clang}.ref.{arm,riscv64}.mk. |
Comment on lines
+72
to
80
| }) + select({ | ||
| # Matches dev/make/deps.ref.mk RNG_OPENRNG: swaps the ref RNG's | ||
| # <random>-based shim for the OpenRNG (VSL-ABI-compatible) | ||
| # backend in src/externals/service_rng_ref.h. | ||
| "@config//:rng_backend_openrng": [ | ||
| "OPENRNG_BACKEND", | ||
| ], | ||
| "//conditions:default": [], | ||
| }), |
Comment on lines
42
to
50
| deps = select({ | ||
| "@config//:backend_ref": [ "@openblas//:openblas", | ||
| ], | ||
| "//conditions:default": [ "@mkl//:mkl_core", | ||
| ], | ||
| }) + select({ | ||
| "@config//:rng_backend_openrng": [ "@openrng//:openrng" ], | ||
| "//conditions:default": [], | ||
| }), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CC/CXXcross-compiler triple prefixes instead of the hardcodedintel64, and registers arch-specifictoolchain()s and@config//:linux_aarch64/@config//:linux_riscv64target platforms.cpu_infoto allowsve/rv64as single-variant ISA values on ARM/RISC-V (no runtime CPU dispatch there, unlike x86's sse2/avx2/avx512), and applies-march=armv8-a+sve/-march=rv64gc_v1p0_zvl128bplus the matchingDAAL_CPU/__CPU_TAG__defines, mirroring the existing Makedev/make/compiler_definitions/{gnu,clang}.ref.{arm,riscv64}.mk.dev/bazel/deps/openrng.bzl(mirrorsopenblas.bzl) and arng_backendconfig flag;--rng_backend=openrngaddsOPENRNG_BACKENDand links@openrng//:openrngalongside OpenBLAS, matchingdev/make/deps.ref.mk'sRNG_OPENRNG.dev/bazel/cc/compile.bzlsurfaced by//cpp/oneapi/dal:dynamicon the new arches: the per-ISA source-file split (cpu_to_file_dict, e.g.algo/jaccard's_skx-suffixed file) only ever has x86 ISA keys, socpu="sve"/"rv64"raised a dict KeyError. Now falls back to the general_cpufile list on arches with no such split.ci-aarch64.yml, bothrefandopenrngrng backends) and a newLinuxBazelGNU_OpenBLAS_rv64job cross-compiling withriscv64-linux-gnu-gccin.ci/pipeline/ci.yml. Fixed.ci/env/bazelisk.shto pick the Bazelisk asset matching the CI host arch instead of always downloading the amd64 binary (would exec-format-fail on the arm64 runner).Only the
refbackend is supported on ARM/RISC-V, same as Make.Test plan
aarch64-linux-gnu-gcc/riscv64-linux-gnu-gcccross toolchains +qemu-user-staticviaapt-get download(no root) and validated the full toolchain (compile, link, run under qemu) independent of Bazel first.bazel build --platforms=@config//:linux_aarch64 --backend_config=ref //cpp/daal:core_dynamic //cpp/oneapi/dal:dynamicwithCC=aarch64-linux-gnu-gcc— compiles and links; produced.sos are valid ARM aarch64 ELF (confirmed viaqemu-aarch64-staticandreadelf).CC=riscv64-linux-gnu-gccandqemu-riscv64-static.--rng_backend=openrngwith a stub VSL-ABI-compatibleopenrng.h/libopenrng.a— confirms the repo/include/define/link wiring end-to-end (compiles and linkscore_dynamic).//cpp/daal:core_dynamic,//cpp/oneapi/dal:dynamic) still builds cleanly — no regression on the existing path, including the x86-specific_skxper-ISA source split that thecompile.bzlfix touches.ci-aarch64.yml's bazel lane,LinuxBazelGNU_OpenBLAS_rv64) haven't run in this repo's actual CI yet — need a maintainer to trigger/review them before merge, since I can't run GitHub Actions/Azure Pipelines from this sandbox.openrng-build-and-test-equivalent Bazel lane inci-aarch64.ymluses.ci/env/openrng.shto build the real thing, but that hasn't executed yet.🤖 Generated with Claude Code