feat: teach find_python_dependencies to scan pyproject.toml and uv.lock by irfanuddinahmad · Pull Request #735 · openedx/repo-tools

irfanuddinahmad · 2026-07-21T16:01:01Z

What

find_python_dependencies previously only knew how to scan pip-compile style requirements.txt files (via requirements-parser) for dependencies whose PyPI-registered homepage belongs to a "second party" org (edx-unsupported, mitodl, overhangio, open-craft, eduNEXT, raccoongang) rather than openedx itself, so they can be flagged for review/migration.

Repos migrating from pip-compile to pyproject.toml + uv per the org-wide Python packaging modernization (openedx/public-engineering#506) have no flat requirements file left for it to scan. This forced openedx-platform to disable its Check Python Dependencies workflow entirely as part of its own migration (openedx/openedx-platform#38915), rather than have it silently no-op after requirements/edx/{base,testing}.txt were removed.

This closes #725.

How

iter_requirement_names() now branches on the input file name:

uv.lock: reads the fully-resolved [[package]] list — this matches the same direct+transitive dependency closure a pip-compile'd requirements.txt used to represent, so it's the most faithful drop-in replacement for the old --req-file requirements/edx/base.txt usage.
anything else: falls back to the existing requirements.txt parsing, unchanged.

pyproject.toml scanning was deliberately left out (see review discussion below): its [project.dependencies]/[dependency-groups] only list direct dependencies, not the resolved transitive closure, so scanning it directly would be misleading about what this tool actually checks.

main() also now dedupes names-per-file before hitting the PyPI API.

Verification

tests/test_find_python_dependencies.py covers both input formats (requirements.txt, uv.lock) plus two main()-level tests (one that should flag a second-party dependency, one that should respect --ignore).
Ran the new tests and the full existing suite: 95 passed.
Sanity-checked iter_requirement_names() directly against openedx-platform's real uv.lock (445 names) to confirm it doesn't choke on a real-world file.

Follow-up

Once this is released, openedx-platform's check_python_dependencies.yml can point --req-file at uv.lock and be re-enabled (currently workflow_dispatch-only + if: false, per the comment there referencing this issue).

openedx-webhooks · 2026-07-21T16:01:08Z

Thanks for the pull request, @irfanuddinahmad!

This repository is currently maintained by @openedx/axim-engineering.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

openedx/repo-tools#725 (find_python_dependencies needs to scan pyproject.toml/uv.lock) now has a proposed fix at openedx/repo-tools#735 -- linking it from the disabled-check comment so re-enabling this workflow is easy to track once it merges/releases.

mphilbrick211 · 2026-07-27T21:45:24Z

@openedx/axim-engineering hi there! Would someone be able to take a look at this?

feanil

@irfanuddinahmad this looks good, I don't think it's valuable to parse the pyproject dependencies and also I think it may be a bit misleading. Let's drop that and bump the version and then I think this is good to merge and release.

feanil · 2026-07-29T12:51:13Z

+            yield from _dependency_group_names(all_groups.get(included, []), all_groups, seen)
+
+
+def _names_from_pyproject_toml(data):


I'm not sure how valuable this is to have. These aren't the resolved full dependencies and so if someone ran the tool against this, they might be mislead into thinking that this tool resolves all the sub-dependencies if we pass the pyproject file instead of a lock file of some sort.

At the very least, we should document that we don't resolve dependencies via the find_python_dependencies tool. But I think it might be better to only support compiled file types like the uv.lock and *.txt files. That way the intent is clear and since we're moving everything to uv.lock, we don't need this at the moment anyway so let's not add more code than we need.

Done — dropped _names_from_pyproject_toml/_dependency_group_names and the pyproject.toml branch in iter_requirement_names() entirely, along with the now-unused packaging dependency (its only import was in the removed code) and the two pyproject.toml-specific tests. uv.lock/requirements.txt support is unchanged. Bumped __version__ to 4.1.0. Full suite still passes (95 tests).

Per review (feanil, PR openedx#735): scanning pyproject.toml's [project.dependencies]/[dependency-groups] only surfaces direct dependencies, not the resolved transitive closure -- unlike uv.lock or a pip-compile'd requirements.txt, both of which represent a fully resolved dependency graph. Supporting pyproject.toml directly would be misleading (callers could assume this tool resolves dependencies when it doesn't) and isn't needed anyway, since repos migrating off pip-compile land on uv.lock as their fully-resolved file. Removed _names_from_pyproject_toml/_dependency_group_names and the pyproject.toml branch in iter_requirement_names(), the now-unused `packaging` dependency (only import was in the removed code) and its two uv.lock entries, and the two tests exercising pyproject.toml support. uv.lock and requirements.txt support is unchanged. Bumped __version__ to 4.1.0 (new capability: uv.lock support) per review request, ready to merge and release.

irfanuddinahmad · 2026-08-06T10:24:26Z

Pushed — dropped pyproject.toml scanning entirely (kept uv.lock + the existing requirements.txt fallback), removed the now-unused packaging dep and its two pyproject.toml-only tests, and bumped __version__ to 4.1.0. PR description updated to match. Ready for another look.

find_python_dependencies previously only understood pip-compile style requirements.txt files (via requirements-parser). Repos migrating from pip-compile to pyproject.toml + uv (openedx/public-engineering#506) have no flat requirements file left for it to scan, which forced openedx-platform to disable its Check Python Dependencies workflow entirely (openedx/openedx-platform#38915, tracked in openedx#725). iter_requirement_names() now detects the input file by name: - uv.lock: reads the fully-resolved [[package]] list, matching the same direct+transitive closure a pip-compile'd requirements.txt used to represent. - pyproject.toml: reads [project.dependencies], [project.optional-dependencies], and [dependency-groups], resolving {include-group = "..."} references. - anything else: falls back to the existing requirements.txt parsing, unchanged. Verified against openedx-platform's actual pyproject.toml (202 names) and uv.lock (445 names). Closes openedx#725

Per review (feanil, PR openedx#735): scanning pyproject.toml's [project.dependencies]/[dependency-groups] only surfaces direct dependencies, not the resolved transitive closure -- unlike uv.lock or a pip-compile'd requirements.txt, both of which represent a fully resolved dependency graph. Supporting pyproject.toml directly would be misleading (callers could assume this tool resolves dependencies when it doesn't) and isn't needed anyway, since repos migrating off pip-compile land on uv.lock as their fully-resolved file. Removed _names_from_pyproject_toml/_dependency_group_names and the pyproject.toml branch in iter_requirement_names(), the now-unused `packaging` dependency (only import was in the removed code) and its two uv.lock entries, and the two tests exercising pyproject.toml support. uv.lock and requirements.txt support is unchanged. Bumped __version__ to 4.1.0 (new capability: uv.lock support) per review request, ready to merge and release.

openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels Jul 21, 2026

openedx-webhooks added this to Contributions Jul 21, 2026

github-project-automation Bot moved this to Needs Triage in Contributions Jul 21, 2026

irfanuddinahmad mentioned this pull request Jul 22, 2026

[DEPR]: openedx-platform pip-compile compatibility exports (requirements/edx/*.txt and equivalents) openedx/public-engineering#552

Open

6 tasks

mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Jul 27, 2026

feanil self-requested a review July 29, 2026 12:38

feanil requested changes Jul 29, 2026

View reviewed changes

This was referenced Aug 4, 2026

find_python_dependencies: support pyproject.toml / uv.lock, not just requirements/*.txt #725

Open

feat: consolidated pip-compile -> uv migration (all 5 stacked PRs, rebased onto master) openedx/openedx-platform#38915

Open

Irfan Ahmad added 2 commits August 6, 2026 15:30

irfanuddinahmad force-pushed the irfanuddinahmad/find-python-dependencies-pyproject-uv branch from b93e6b3 to 2609cb9 Compare August 6, 2026 10:31

Navigation Menu

Uh oh!

feat: teach find_python_dependencies to scan pyproject.toml and uv.lock - #735

feat: teach find_python_dependencies to scan pyproject.toml and uv.lock#735
irfanuddinahmad wants to merge 2 commits into
openedx:masterfrom
irfanuddinahmad:irfanuddinahmad/find-python-dependencies-pyproject-uv

irfanuddinahmad commented Jul 21, 2026 •

edited

Loading

Uh oh!

openedx-webhooks commented Jul 21, 2026

Uh oh!

mphilbrick211 commented Jul 27, 2026

Uh oh!

feanil left a comment

Uh oh!

feanil Jul 29, 2026

Uh oh!

irfanuddinahmad Aug 6, 2026

Uh oh!

irfanuddinahmad commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		yield from _dependency_group_names(all_groups.get(included, []), all_groups, seen)


		def _names_from_pyproject_toml(data):

Uh oh!

Conversation

irfanuddinahmad commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Verification

Follow-up

Uh oh!

openedx-webhooks commented Jul 21, 2026

Uh oh!

mphilbrick211 commented Jul 27, 2026

Uh oh!

feanil left a comment

Choose a reason for hiding this comment

Uh oh!

feanil Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

irfanuddinahmad commented Jul 21, 2026 •

edited

Loading