Skip to content

Python Build - #292

Open
johannes-spies wants to merge 1 commit into
metatomic-corefrom
metatomic-core-python-build
Open

Python Build#292
johannes-spies wants to merge 1 commit into
metatomic-corefrom
metatomic-core-python-build

Conversation

@johannes-spies

@johannes-spies johannes-spies commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

This PR adds the infrastructure needed to build the metatomic_core Python wheel. Essentially, it's a port of the same functionality in metatensor (setup.py, CMakeLists.txt, and package-core.sh). The only difference is that we import metatensor in cmake_ext's run method to get the cmake_prefix_path which is required to build the native library.

The PR also makes an important change to the CI setup: now, containers for testing are built with Rust, similar to what metatensor does.

📚 Download documentation for this pull-request

⚙️ Download Python wheels for this pull-request (you can install these with pip)

@johannes-spies
johannes-spies changed the base branch from main to metatomic-core July 21, 2026 09:17
@johannes-spies
johannes-spies requested a review from Luthaf July 22, 2026 09:02

@Luthaf Luthaf left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, I'm not convinced we should switch to uv in CI, and we should for sure do it in a separate PR anyway

Comment thread .github/workflows/build-wheels.yml Outdated
Comment thread .github/workflows/torch-tests.yml Outdated
Comment thread .github/workflows/torch-tests.yml Outdated
Comment thread python/metatomic_ase/setup.py Outdated
Comment thread python/metatomic_core/CMakeLists.txt Outdated
Comment thread python/metatomic_torch/setup.py Outdated
Comment thread python/metatomic_torchsim/setup.py Outdated
Comment thread python/scripts/gcc11-manylinux_2_28_aarch64/Dockerfile
Comment thread scripts/git-version-info.py
@johannes-spies

Copy link
Copy Markdown
Contributor Author

Thank you for the review! You're right, the switch to uv was not necessary and I will revert it. I just tried to make the CI as similar as possible to track down the failure in the windows tests.

@johannes-spies
johannes-spies requested a review from Luthaf July 22, 2026 23:09

@Luthaf Luthaf left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wheel (as created by python -m build python/metatomic_core) contains configuration files to find nlohmann::json, which I don't think we want to make available through the to external users here.

[...]
adding 'metatomic/share/cmake/nlohmann_json/nlohmann_jsonConfig.cmake'
adding 'metatomic/share/cmake/nlohmann_json/nlohmann_jsonConfigVersion.cmake'
adding 'metatomic/share/cmake/nlohmann_json/nlohmann_jsonTargets.cmake'
adding 'metatomic/share/pkgconfig/nlohmann_json.pc'
[...]

Ideally they should not even be installed by the main metatomic CMakeLists.txt (this is not specific to Python) when we fetch the code from github ourself.

Can you have a look?

Comment thread .github/workflows/build-wheels.yml Outdated
Comment thread .github/workflows/build-wheels.yml Outdated
Comment thread .github/workflows/build-wheels.yml Outdated
@johannes-spies

Copy link
Copy Markdown
Contributor Author

Yes, you're right. 27675fd sets JSON_Install to OFF.

@Luthaf

Luthaf commented Jul 27, 2026

Copy link
Copy Markdown
Member

We still need to install the headers, just not the cmake config / .pc files

@johannes-spies
johannes-spies force-pushed the metatomic-core-python-build branch 2 times, most recently from 765ae6c to 8dc3972 Compare July 28, 2026 16:49
@johannes-spies

johannes-spies commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

I added the headers in 8dc3972. Also, I tried making the build-wheels.yml pipelines as close as possible to metatensor's pipeline! I hope everything is now very similar, minus uv.

@johannes-spies
johannes-spies requested a review from Luthaf July 31, 2026 10:48
Comment on lines +465 to +474
if (NOT nlohmann_json_FOUND)
# nlohmann_json was fetched and vendored by us: install its headers (used
# by metatomic's own public headers), but remove the CMake package config
# and pkg-config files it installs, since we don't want to advertise a
# system-wide nlohmann_json package to external users.
install(CODE "
file(REMOVE_RECURSE \"\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATADIR}/cmake/nlohmann_json\")
file(REMOVE \"\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATADIR}/pkgconfig/nlohmann_json.pc\")
")
endif()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the back and forth here, I did not realized we were using the cmake files from our own metatomic-config.cmake …

Maybe the best solution here would be to install these to a different directory, i.e. to CMAKE_INSTALL_PREFIX/lib/metatomic/external instead of CMAKE_INSTALL_PREFIX? This way we can find the headers with normal cmake configuration, and also hide them from most users by default.

@RMeli what do you think here?

@Luthaf
Luthaf force-pushed the metatomic-core branch 2 times, most recently from a13f4cc to 173345c Compare August 28, 2026 12:18
@Luthaf
Luthaf force-pushed the metatomic-core-python-build branch from c1498bd to 034a829 Compare August 28, 2026 12:23
@Luthaf
Luthaf force-pushed the metatomic-core branch 2 times, most recently from 0c4521a to 1637cfb Compare August 28, 2026 13:06
@Luthaf
Luthaf force-pushed the metatomic-core-python-build branch from 034a829 to 7577f1b Compare August 31, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants