Skip to content

Python 3.12.13, massive update of python modules#10718

Open
smuzaffar wants to merge 2 commits into
IB/CMSSW_20_1_X/masterfrom
python3.12.13
Open

Python 3.12.13, massive update of python modules#10718
smuzaffar wants to merge 2 commits into
IB/CMSSW_20_1_X/masterfrom
python3.12.13

Conversation

@smuzaffar

@smuzaffar smuzaffar commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Python 3.12.13 along with massive update for python modules

  • xz 5.8.3 (which rebuilds python3 so we propose to update python version + python modules)
  • pip 26.1.2
  • setuptools 83.0.0

As this rebuild nearly all the externals (due to rebuild of python3) so I also propose to update the build configuration of externals to use CMAKE_BUILD_TYPE=%{cmake_build_type} which will allow us to configure and build a package in debug mode using cmsBuild --define cms_debug_packages=pkg1,pkg2

@cmsbuild

cmsbuild commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

A new Pull Request was created by @smuzaffar for branch IB/CMSSW_20_1_X/master.

@akritkbehera, @cmsbuild, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

@cmsbuild

cmsbuild commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

cms-bot internal usage

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild

cmsbuild commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@cmsbuild

cmsbuild commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54645/summary.html
COMMIT: 9bd62e7
CMSSW: CMSSW_20_1_X_2026-07-09-1100/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10718/54645/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed External Build

I found compilation error when building:

rm -rf /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211/pkgtools-pkg-src-move2git /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211/cmsdist-tmp; mkdir -p /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211/cmsdist-tmp; CMSBUILD_PACKAGE=py3-cryptography;  export _CMSBUILD_BUILD_ENV_=1; export TMPDIR=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211/cmsdist-tmp;  source /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc13/rpm-env.sh; source /data/cmsbld/jenkins/workspace/ib-run-pr-tests/pkgtools/build-system-env.sh 16;  rpmbuild  --noclean --buildroot /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/BUILDROOT/3abea73aac4048d3e5e6e8d664809211 -bp --define '_smp_build_nthreads 16' --define '_topdir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir' --define "compiling_processes 16"  --define '_builddir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211' --define "cmsdist_directory /data/cmsbld/jenkins/workspace/ib-run-pr-tests/cmsdist" --define "compilerv 1340" --define "cmscompilerv 13" --define "cmsos el9_amd64" --define "__bootstrap ~bootstrap" --define "almalinux_ver 9" --define "almalinux 9" --define "centos_ver 9" --define "centos 9" --define "rhel 9" --define "dist %{!?distprefix0:%{?distprefix}}%{expand:%{lua:for i=0,9999 do print("%{?distprefix" .. i .."}") end}}.el9%{?with_bootstrap:~bootstrap}" --define "el9 1" --define "dist_vendor         AlmaLinux OS Foundation" --define "dist_name           AlmaLinux" --define "dist_home_url       https://almalinux.org/" --define "dist_bug_report_url https://bugs.almalinux.org/" --define "package_vectorization x86-64-v2" --define "cmsswdata_version_link 1" --define "archfirst yes" --define "cmsBuild_bootstrap 1"   /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/SPECS/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211/spec >>/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_amd64_gcc13/external/py3-cryptography/49.0.0-3abea73aac4048d3e5e6e8d664809211/log 2>&1
Checking package dependencies: external+py3-uv-build+0.11.28-1a35419cece5f4de86902a3019fcdf29
Done checking package dependencies: external+py3-uv-build+0.11.28-1a35419cece5f4de86902a3019fcdf29
Checking local path dependency for rpm package external+py3-uv-build+0.11.28-1a35419cece5f4de86902a3019fcdf29 just build.
RPM installation stderr py3-uv-build:
error: Failed dependencies:
	libbz2.so.1()(64bit) is needed by external+py3-uv-build+0.11.28-1a35419cece5f4de86902a3019fcdf29-1-1.x86_64

Failed to install RPM for py3-uv-build
install-external+py3-uv-build+0.11.28-1a35419cece5f4de86902a3019fcdf29 failed.
Traceback (most recent call last):


@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test for el9_aarch64_gcc13

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test for el9_amd64_gcc13

@cmsbuild

Copy link
Copy Markdown
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54689/summary.html
COMMIT: d5597ed
CMSSW: CMSSW_20_1_X_2026-07-10-2300/el9_aarch64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10718/54689/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed External Build

I found compilation error when building:

[1422/1548] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/bin/nvcc -forward-unknown-to-host-compiler -DCOMPILE_HOPPER_TMA_GEMMS -DCPUINFO_SUPPORTED -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DENABLE_BF16 -DENABLE_CUDA_NHWC_OPS -DENABLE_DLPACK -DENABLE_FP4 -DENABLE_FP8 -DEXCLUDE_SM_110 -DEXCLUDE_SM_86 -DHAS_SM80_OR_LATER -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DORT_ENABLE_STREAM -DPLATFORM_POSIX -DPROTOBUF_USE_DLLS -DUSE_CUDA=1 -DUSE_DX_INTEROP=0 -DUSE_FLASH_ATTENTION=1 -DUSE_FP8_KV_CACHE=1 -DUSE_MEMORY_EFFICIENT_ATTENTION=1 -D_GNU_SOURCE -Donnxruntime_providers_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime/core/session -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/pytorch_cpuinfo-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/gsl-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/abseil_cpp-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/date-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/3.21.9-32fda8c6a1d12e87a866ecb905948ce2/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/flatbuffers-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/examples -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cudnn_frontend-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/eigen3-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/safeint-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include/cccl -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cudnn/9.23.0.39-daa375a8a35f76fb049a8ce1994fb144/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/mp11-src/include -DTHRUST_IGNORE_DEPRECATED_API -DCUB_IGNORE_DEPRECATED_API -Wno-deprecated-gpu-targets --static-global-template-stub=false -cudart shared -Xfatbin=-compress-all -compress-mode=size --expt-relaxed-constexpr --Werror default-stream-launch -Xcudafe "--diag_suppress=bad_friend_decl" -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -Xcudafe "--diag_suppress=expr_has_no_effect" -O3 -DNDEBUG -std=c++20 "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_89,code=[sm_89]" "--generate-code=arch=compute_90a,code=[sm_90a]" "--generate-code=arch=compute_100a,code=[sm_100a]" "--generate-code=arch=compute_120a,code=[sm_120a]" -Xcompiler=-fPIC -Wno-deprecated-declarations -Xcudafe --diag_suppress=conversion_function_not_usable --compiler-options -Wall --compiler-options -Wno-deprecated-copy --compiler-options -Wno-nonnull-compare --compiler-options -Wno-interference-size -Xcompiler -Wno-nonnull-compare -Xcompiler -Wno-interference-size --threads 1 --diag-suppress=177 --static-global-template-stub=false --diag-suppress=221 -Xcompiler -Wno-attributes -Xcompiler -Wno-reorder -Xcompiler -Wno-error=sign-compare -Xptxas=-w -Werror all-warnings -MD -MT CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim128_fp16_sm80.cu.o -MF CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim128_fp16_sm80.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim128_fp16_sm80.cu -o CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim128_fp16_sm80.cu.o
[1423/1548] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/bin/nvcc -forward-unknown-to-host-compiler -DCOMPILE_HOPPER_TMA_GEMMS -DCPUINFO_SUPPORTED -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DENABLE_BF16 -DENABLE_CUDA_NHWC_OPS -DENABLE_DLPACK -DENABLE_FP4 -DENABLE_FP8 -DEXCLUDE_SM_110 -DEXCLUDE_SM_86 -DHAS_SM80_OR_LATER -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DORT_ENABLE_STREAM -DPLATFORM_POSIX -DPROTOBUF_USE_DLLS -DUSE_CUDA=1 -DUSE_DX_INTEROP=0 -DUSE_FLASH_ATTENTION=1 -DUSE_FP8_KV_CACHE=1 -DUSE_MEMORY_EFFICIENT_ATTENTION=1 -D_GNU_SOURCE -Donnxruntime_providers_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime/core/session -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/pytorch_cpuinfo-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/gsl-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/abseil_cpp-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/date-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/3.21.9-32fda8c6a1d12e87a866ecb905948ce2/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/flatbuffers-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/examples -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cudnn_frontend-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/eigen3-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/safeint-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include/cccl -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cudnn/9.23.0.39-daa375a8a35f76fb049a8ce1994fb144/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/mp11-src/include -DTHRUST_IGNORE_DEPRECATED_API -DCUB_IGNORE_DEPRECATED_API -Wno-deprecated-gpu-targets --static-global-template-stub=false -cudart shared -Xfatbin=-compress-all -compress-mode=size --expt-relaxed-constexpr --Werror default-stream-launch -Xcudafe "--diag_suppress=bad_friend_decl" -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -Xcudafe "--diag_suppress=expr_has_no_effect" -O3 -DNDEBUG -std=c++20 "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_89,code=[sm_89]" "--generate-code=arch=compute_90a,code=[sm_90a]" "--generate-code=arch=compute_100a,code=[sm_100a]" "--generate-code=arch=compute_120a,code=[sm_120a]" -Xcompiler=-fPIC -Wno-deprecated-declarations -Xcudafe --diag_suppress=conversion_function_not_usable --compiler-options -Wall --compiler-options -Wno-deprecated-copy --compiler-options -Wno-nonnull-compare --compiler-options -Wno-interference-size -Xcompiler -Wno-nonnull-compare -Xcompiler -Wno-interference-size --threads 1 --diag-suppress=177 --static-global-template-stub=false --diag-suppress=221 -Xcompiler -Wno-attributes -Xcompiler -Wno-reorder -Xcompiler -Wno-error=sign-compare -Xptxas=-w -Werror all-warnings -MD -MT CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim256_fp16_sm80.cu.o -MF CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim256_fp16_sm80.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim256_fp16_sm80.cu -o CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim256_fp16_sm80.cu.o
[1424/1548] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/bin/nvcc -forward-unknown-to-host-compiler -DCOMPILE_HOPPER_TMA_GEMMS -DCPUINFO_SUPPORTED -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DENABLE_BF16 -DENABLE_CUDA_NHWC_OPS -DENABLE_DLPACK -DENABLE_FP4 -DENABLE_FP8 -DEXCLUDE_SM_110 -DEXCLUDE_SM_86 -DHAS_SM80_OR_LATER -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DORT_ENABLE_STREAM -DPLATFORM_POSIX -DPROTOBUF_USE_DLLS -DUSE_CUDA=1 -DUSE_DX_INTEROP=0 -DUSE_FLASH_ATTENTION=1 -DUSE_FP8_KV_CACHE=1 -DUSE_MEMORY_EFFICIENT_ATTENTION=1 -D_GNU_SOURCE -Donnxruntime_providers_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime/core/session -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/pytorch_cpuinfo-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/gsl-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/abseil_cpp-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/date-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/3.21.9-32fda8c6a1d12e87a866ecb905948ce2/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/flatbuffers-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/examples -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cudnn_frontend-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/eigen3-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/safeint-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include/cccl -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cudnn/9.23.0.39-daa375a8a35f76fb049a8ce1994fb144/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/mp11-src/include -DTHRUST_IGNORE_DEPRECATED_API -DCUB_IGNORE_DEPRECATED_API -Wno-deprecated-gpu-targets --static-global-template-stub=false -cudart shared -Xfatbin=-compress-all -compress-mode=size --expt-relaxed-constexpr --Werror default-stream-launch -Xcudafe "--diag_suppress=bad_friend_decl" -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -Xcudafe "--diag_suppress=expr_has_no_effect" -O3 -DNDEBUG -std=c++20 "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_89,code=[sm_89]" "--generate-code=arch=compute_90a,code=[sm_90a]" "--generate-code=arch=compute_100a,code=[sm_100a]" "--generate-code=arch=compute_120a,code=[sm_120a]" -Xcompiler=-fPIC -Wno-deprecated-declarations -Xcudafe --diag_suppress=conversion_function_not_usable --compiler-options -Wall --compiler-options -Wno-deprecated-copy --compiler-options -Wno-nonnull-compare --compiler-options -Wno-interference-size -Xcompiler -Wno-nonnull-compare -Xcompiler -Wno-interference-size --threads 1 --diag-suppress=177 --static-global-template-stub=false --diag-suppress=221 -Xcompiler -Wno-attributes -Xcompiler -Wno-reorder -Xcompiler -Wno-error=sign-compare -Xptxas=-w -Werror all-warnings -MD -MT CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_bf16_sm80.cu.o -MF CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_bf16_sm80.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_bf16_sm80.cu -o CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_bf16_sm80.cu.o
[1425/1548] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/bin/nvcc -forward-unknown-to-host-compiler -DCOMPILE_HOPPER_TMA_GEMMS -DCPUINFO_SUPPORTED -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DENABLE_BF16 -DENABLE_CUDA_NHWC_OPS -DENABLE_DLPACK -DENABLE_FP4 -DENABLE_FP8 -DEXCLUDE_SM_110 -DEXCLUDE_SM_86 -DHAS_SM80_OR_LATER -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DORT_ENABLE_STREAM -DPLATFORM_POSIX -DPROTOBUF_USE_DLLS -DUSE_CUDA=1 -DUSE_DX_INTEROP=0 -DUSE_FLASH_ATTENTION=1 -DUSE_FP8_KV_CACHE=1 -DUSE_MEMORY_EFFICIENT_ATTENTION=1 -D_GNU_SOURCE -Donnxruntime_providers_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/include/onnxruntime/core/session -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/pytorch_cpuinfo-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/gsl-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/abseil_cpp-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/date-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/onnx-build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/protobuf/3.21.9-32fda8c6a1d12e87a866ecb905948ce2/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/flatbuffers-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/examples -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cutlass-src/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/cudnn_frontend-src/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/eigen3-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/safeint-src -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cuda/13.3.1-42e59f38775163fbe81e7a5c898846b9/include/cccl -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_aarch64_gcc13/external/cudnn/9.23.0.39-daa375a8a35f76fb049a8ce1994fb144/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/build/_deps/mp11-src/include -DTHRUST_IGNORE_DEPRECATED_API -DCUB_IGNORE_DEPRECATED_API -Wno-deprecated-gpu-targets --static-global-template-stub=false -cudart shared -Xfatbin=-compress-all -compress-mode=size --expt-relaxed-constexpr --Werror default-stream-launch -Xcudafe "--diag_suppress=bad_friend_decl" -Xcudafe "--diag_suppress=unsigned_compare_with_zero" -Xcudafe "--diag_suppress=expr_has_no_effect" -O3 -DNDEBUG -std=c++20 "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_89,code=[sm_89]" "--generate-code=arch=compute_90a,code=[sm_90a]" "--generate-code=arch=compute_100a,code=[sm_100a]" "--generate-code=arch=compute_120a,code=[sm_120a]" -Xcompiler=-fPIC -Wno-deprecated-declarations -Xcudafe --diag_suppress=conversion_function_not_usable --compiler-options -Wall --compiler-options -Wno-deprecated-copy --compiler-options -Wno-nonnull-compare --compiler-options -Wno-interference-size -Xcompiler -Wno-nonnull-compare -Xcompiler -Wno-interference-size --threads 1 --diag-suppress=177 --static-global-template-stub=false --diag-suppress=221 -Xcompiler -Wno-attributes -Xcompiler -Wno-reorder -Xcompiler -Wno-error=sign-compare -Xptxas=-w -Werror all-warnings -MD -MT CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_fp16_sm80.cu.o -MF CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_fp16_sm80.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_fp16_sm80.cu -o CMakeFiles/onnxruntime_providers_cuda.dir/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el9_aarch64_gcc13/external/onnxruntime/1.26.0-69a91b24ecf49d5b34a43cb225791c58/onnxruntime-1.26.0/onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_fwd_split_hdim64_fp16_sm80.cu.o
ninja: build stopped: subcommand failed.
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.mCjv5y (%build)

RPM build warnings:
Macro expanded in comment on line 527: %{pkginstroot}/${PYTHON3_LIB_SITE_PACKAGES}




@cmsbuild

Copy link
Copy Markdown
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54688/summary.html
COMMIT: d5597ed
CMSSW: CMSSW_20_1_X_2026-07-10-2300/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10718/54688/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54688/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54688/git-merge-result

Comparison Summary

Summary:

  • You potentially removed 7 lines from the logs
  • Reco comparison results: 43599 differences found in the comparisons
  • DQMHistoTests: Total files compared: 45
  • DQMHistoTests: Total histograms compared: 3548186
  • DQMHistoTests: Total failures: 425267
  • DQMHistoTests: Total nulls: 312
  • DQMHistoTests: Total successes: 3122589
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 3.908 KiB( 44 files compared)
  • DQMHistoSizes: changed ( 8.0 ): 3.908 KiB SiStrip/MechanicalView
  • Checked 195 log files, 163 edm output root files, 45 DQM output files
  • TriggerResults: found differences in 12 / 44 workflows

Max Memory Comparisons exceeding threshold

@cms-sw/core-l2 , I found 5 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 16834.0_TTbar_14TeV+2025 step5 max memory diff 301.9 exceeds +/- 30.0 MiB
  • Error: Workflow 34434.911_TTbar_14TeV+Run4D121_DD4hep step5 max memory diff 57.2 exceeds +/- 30.0 MiB
  • Error: Workflow 34434.911_TTbar_14TeV+Run4D121_DD4hep step3 max memory diff 32.5 exceeds +/- 30.0 MiB
  • Error: Workflow 34634.0_TTbar_14TeV+Run4D121PU step2 max memory diff -95.8 exceeds +/- 30.0 MiB
  • Error: Workflow 34634.0_TTbar_14TeV+Run4D121PU step3 max memory diff -397.8 exceeds +/- 30.0 MiB

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild

Copy link
Copy Markdown
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54737/summary.html
COMMIT: d5597ed
CMSSW: CMSSW_20_1_X_2026-07-13-2300/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/10718/54737/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54737/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a95fd1/54737/git-merge-result

Comparison Summary

Summary:

  • You potentially added 8 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 45
  • DQMHistoTests: Total histograms compared: 3548186
  • DQMHistoTests: Total failures: 1610
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3546558
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 54.768 KiB( 44 files compared)
  • DQMHistoSizes: changed ( 34434.0,... ): 1.786 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltHpsPFTau150LooseTauWPDeepTau__2Dvspteta
  • DQMHistoSizes: changed ( 34434.0,... ): 1.774 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltL1SingleNNTau150__2Dvspteta
  • DQMHistoSizes: changed ( 34434.0,... ): 1.771 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.0,... ): 1.761 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__GEN__2Dvspt__eta
  • DQMHistoSizes: changed ( 34434.0,... ): 0.211 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltHpsPFTau150LooseTauWPDeepTau__vspt
  • DQMHistoSizes: changed ( 34434.0,... ): 0.199 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltL1SingleNNTau150__vspt
  • DQMHistoSizes: changed ( 34434.0,... ): 0.196 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltHpsPFTauTrack__vspt
  • DQMHistoSizes: changed ( 34434.0,... ): 0.184 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__GEN__vspt
  • DQMHistoSizes: changed ( 34434.0,... ): 0.157 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltHpsPFTau150LooseTauWPDeepTau__vseta
  • DQMHistoSizes: changed ( 34434.0,... ): 0.146 KiB HLTGenVal/tauHAD__HLT_LooseDeepTauPFTauHPS150_L2NN_eta2p1__hltL1SingleNNTau150__vseta
  • DQMHistoSizes: changed ( 34434.0 ): ...
  • Checked 195 log files, 163 edm output root files, 45 DQM output files
  • TriggerResults: no differences found

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

1 similar comment
@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

@smuzaffar

Copy link
Copy Markdown
Contributor Author

please test

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #10718 was updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants