Add ALIGNN 2.0 benchmark

[jae-hlee]

Benchmark submission: ALIGNN 2.0

Brief description of the algorithm

ALIGNN 2.0 (arXiv:2609.19487) is a pure-PyTorch reimplementation of the Atomistic Line Graph Neural Network. This submission covers the nine structure-based tasks of matbench_v0.1 with one fixed configuration across all tasks (hidden size 768, k-nearest-neighbour graph with 12 neighbours within 8 Å). Only the epoch budget varies, set in advance by dataset size. Relative to the original ALIGNN entry it improves dielectric, perovskites, log_kvrh, log_gvrh, mp_e_form and mp_gap, is within the fold-to-fold spread on jdft2d and phonons, and is lower on mp_is_metal. The four composition-only tasks are not applicable, since ALIGNN requires a crystal structure as input.

Included files

• results.json.gz – predictions for all 5 folds of the 9 tasks, recorded with task.record and validated with MatbenchBenchmark.is_valid
• info.json – algorithm description, references, requirements and notes (including a cross-cluster reproducibility check)
• make_matbench_dataset.py – pulls each fold through the matbench API and writes the training dataset and per-fold config
• collect_matbench.py – maps each fold's predictions back to matbench test order and records them
• config_mb_base.json, config_mb_classification_example.json – the training configuration (regression, and the mp_is_metal variant)
• mb_train.sbatch – the Slurm script that trains one fold

Could a maintainer please add the new_benchmark label?

[jae-hlee]

Hello,

It looks like both failures are coming from the CI environment rather than from this submission.

• test (3.9) fails automatically by GitHub because the workflow uses the deprecated actions/upload-artifact@v2.
• benchmark_submission crashes while importing matbench, before any benchmark folder is read: the environment installs monty==2022.4.26 together with pymatgen==2024.8.9, which requires monty>=2024.7.29, and the import fails with TypeError: deprecated() got an unexpected keyword argument 'deadline'.

I ran the assertions from scripts/test_submission.py locally against this folder using the repository's own matbench code: info.json fields present, results.json.gz loads and is_valid is True (9 of 13 tasks recorded), source files present, no file over 10 MB, and the algorithm name is unique among all entries.

Please let me know if there is anything that needs to be done on my side.
Thank you!