Skip to content

Add packaging metadata, CI, tests and a PyPI release workflow - #1

Merged
KalinNonchev merged 2 commits into
mainfrom
chore/packaging-ci-release
Jul 27, 2026
Merged

Add packaging metadata, CI, tests and a PyPI release workflow#1
KalinNonchev merged 2 commits into
mainfrom
chore/packaging-ci-release

Conversation

@KalinNonchev

Copy link
Copy Markdown
Member

What

Prepares deepspotm for a PyPI release and adds the automation to keep it working. No behaviour change to the model.

Packaging

  • requires-python moves >=3.9 to >=3.10. This one is a real bug, not a tidy-up: torch, transformers, peft and lightning all declare >=3.10 in their current releases, so a 3.9 user could never resolve a working dependency set. We were advertising support that cannot exist.
  • PEP 639 license fields. PolyForm-Noncommercial-1.0.0 is a valid SPDX identifier, so it now emits License-Expression: in the metadata, and LICENSE, WEIGHTS_LICENSE.md and THIRD_PARTY_LICENSES.md all ship in the distribution. This needs setuptools>=77, which the build-system requirement now pins.
  • Classifiers, keywords, paper/weights URLs, and wsi / test / dev extras.

Verified locally: python -m build succeeds and twine check --strict passes on both the wheel and the sdist. Resulting metadata is Metadata-Version: 2.4 with License-Expression: PolyForm-Noncommercial-1.0.0, three License-File entries, and all four asset files present in the wheel.

Tests

44 tests, none of which need the gated weights. The checkpoint is gated on the Hub, so CI can never load a real model. Everything here runs against the packaged assets or a randomly-initialised module, which means the suite passes on a fork with no secrets configured.

  • The decoder's (expression, gene_features, attn_weights) return contract, and that gene_indices both subsets and orders the output columns. This is precisely what downstream integrations unpack, including the TIAToolbox wrapper.
  • tokens.csv still carries the 19,338-gene panel the model reports, with dense token ids.
  • The packaged assets resolve from an installed wheel rather than only from a source tree.
  • check_state_dict_load still raises on unexplained key drift, which is the guard that stops a mismatched checkpoint from silently predicting from uninitialised weights.

CI

  • Install and test on 3.10 to 3.13, using the CPU torch wheel so the runner does not pull the ~2.5 GB CUDA build.
  • An import-check job installs the built wheel outside the source tree. A missing package-data entry would otherwise only surface for users, since it works fine from a checkout.
  • twine check --strict on every build.
  • Lint is deliberately scoped to tests/. Running ruff over src/ and examples/ currently reports 4 findings and would reformat 8 files; widening it is worth doing but belongs in its own PR rather than buried in this one.

Release

Tag-triggered publish through PyPI Trusted Publishing (OIDC), so no API token is stored in the repo. There is a TestPyPI path via workflow_dispatch for a dry run, and a guard that the tag matches the version in pyproject.toml. Dependabot watches pip and github-actions, grouped to one PR per week per ecosystem.

Before the first release you need to register the workflow at https://pypi.org/manage/account/publishing/ with owner ratschlab, repo DeepSpotM, workflow release.yml, environment pypi. I would run the TestPyPI path first, since version 1.0.0 can never be re-uploaded once it is published.

Also

  • config.py now uses importlib.resources.files() instead of path(), which is deprecated since 3.11 and returned a context-managed path that was already released by the time it was stored.
  • examples/predict_tcga_skcm.ipynb told users to run pip install deepspotm, which fails today because the package is not on PyPI. It now points at the git install, matching the README.

Kalin Nonchev added 2 commits July 27, 2026 16:56
Prepares the package for a PyPI release and adds the automation to keep it
working.

Packaging:
- requires-python moves >=3.9 -> >=3.10. torch, transformers, peft and
  lightning all require >=3.10 in their current releases, so a 3.9 install
  could never resolve a working dependency set.
- Adopt the PEP 639 license fields. The SPDX id PolyForm-Noncommercial-1.0.0
  now appears as License-Expression, and LICENSE, WEIGHTS_LICENSE.md and
  THIRD_PARTY_LICENSES.md all ship in the distribution.
- Add classifiers, keywords, and the paper/weights URLs so the PyPI page is
  usable, plus `wsi`, `test` and `dev` extras.

Tests (44, none of which need the gated weights):
- The checkpoint is gated on the Hugging Face Hub, so CI can never load a real
  model. Every test runs against the packaged assets or a randomly-initialised
  module instead, which keeps the suite runnable on a fork with no secrets.
- Cover the decoder's (expression, gene_features, attn_weights) contract and
  gene_indices subsetting/ordering, which is what downstream integrations such
  as TIAToolbox unpack.
- Assert tokens.csv still carries the 19338-gene panel the model reports, and
  that the assets resolve from an installed wheel rather than a source tree.

CI:
- Install and test on 3.10-3.13 using the CPU torch wheel.
- An import-check job installs the built wheel outside the source tree, so a
  missing package-data entry fails loudly instead of at user runtime.
- twine check --strict on every build.
- Lint is scoped to tests/ for now; running ruff over src/ and examples/
  reports 4 findings and would reformat 8 files, which belongs in its own
  change rather than buried here.

Release:
- Tag-triggered publish via PyPI Trusted Publishing (OIDC), so no API token is
  stored in the repository, with a TestPyPI path for dry runs and a check that
  the tag matches the project version.
- Dependabot for pip and github-actions, grouped to one PR per week.

Also:
- config.py uses importlib.resources.files() instead of the deprecated path(),
  which returned a context-managed path already released on exit.
- The TCGA example notebook said `pip install deepspotm`, which fails because
  the package is not on PyPI yet; it now points at the git install like the
  README does.
timm and lightning pull torchvision from PyPI, which does not match the CPU
torch wheel installed from the pytorch index. The mismatched pair fails at
import with 'operator torchvision::nms does not exist', so both have to come
from the same index.
@KalinNonchev

Copy link
Copy Markdown
Member Author

Exact Trusted Publishing setup, for the record.

deepspotm does not exist on PyPI yet, so this needs the pending publisher form rather than the per-project one. Signed in as nonchev, go to https://pypi.org/manage/account/publishing/, scroll to "Add a new pending publisher", and enter:

Field Value
PyPI Project Name deepspotm
Owner ratschlab
Repository name DeepSpotM
Workflow name release.yml
Environment name pypi

The environment name has to match the environment: name: pypi block in the workflow, otherwise the OIDC exchange is rejected.

Suggested order once this is merged:

  1. Add the pending publisher above, and a second one for TestPyPI at https://test.pypi.org/manage/account/publishing/ with environment testpypi.
  2. Run the release workflow via workflow_dispatch with target testpypi, then pip install -i https://test.pypi.org/simple/ deepspotm in a clean venv to confirm the wheel installs and imports.
  3. Only then git tag v1.0.0 && git push --tags. Version 1.0.0 can never be re-uploaded once it is on PyPI, so the dry run is worth it.

Unrelated but worth knowing: the name deepspot is also still unclaimed on PyPI.

@KalinNonchev
KalinNonchev merged commit 08533b8 into main Jul 27, 2026
7 checks passed
@KalinNonchev
KalinNonchev deleted the chore/packaging-ci-release branch July 27, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant