Set Partial Clone flags on sparse-checkout#4044
Conversation
![buildsworth-bk-app[bot]](https://avatars.githubusercontent.com/in/3010600?s=60&v=4){kind=small}
There was a problem hiding this comment.
I found one checkout-path issue that looks like it prevents the new auto-filter behavior from meeting the sparse-checkout intent when users haven't also supplied clone flags such as --sparse.
Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2623, then answer my questions about the findings.
…n sparse paths are present
|
@buildsworth-bk , please re-review |
buildsworth-bk-app
Bot
dismissed
their stale review
Previous concerns have been addressed.
There was a problem hiding this comment.
The previous clone-path concern is fixed and I dismissed that stale review. I found one remaining issue where the new fetch-path filter can still override a user-supplied clone filter.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2898, then answer my questions about the findings.
There was a problem hiding this comment.
The previous checkout concern is addressed. I left one non-blocking note from this pass; this touches core checkout behavior, so I'm leaving a comment review rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2897, then answer my questions about the findings.
There was a problem hiding this comment.
The clone-path concern is resolved, but the fetch-filter concern in the open thread is still unresolved. I don’t have any new findings beyond that.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2899, then answer my questions about the findings.
There was a problem hiding this comment.
The stale test note is addressed, but the fetch-filter concern in the open thread is still unresolved. I don't have any new findings beyond that.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2909, then answer my questions about the findings.
Previous concerns have been addressed.
There was a problem hiding this comment.
The fetch-filter concern is fixed, and I don't have any new findings. This still touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2930, then answer my questions about the findings.
There was a problem hiding this comment.
I don't have any new findings. This touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 3189, then answer my questions about the findings.
Co-authored-by: Josh Deprez <jd@buildkite.com>
There was a problem hiding this comment.
I found one compile blocker in the helper update.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 3803, then answer my questions about the findings.
Previous concerns have been addressed.
There was a problem hiding this comment.
The previous import blocker is fixed. I found one remaining lint blocker from the sparse helper being left unused.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 4416, then answer my questions about the findings.
Previous concerns have been addressed.
There was a problem hiding this comment.
The previous lint blocker is fixed, and I don't have any new findings. This still touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 4417, then answer my questions about the findings.
There was a problem hiding this comment.
I don't have any new findings. This still touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 4418, then answer my questions about the findings.
isaacsu
left a comment
isaacsu
left a comment
There was a problem hiding this comment.
Just one more, then I think we should be good to go.
…adding blobless filter flag
…t_WithGitMirrors and update code descriptions for clearer contexts
…ithSparseCheckoutFallsBackOnOldGit
There was a problem hiding this comment.
I don't have any new findings. This still touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 4972, then answer my questions about the findings.
There was a problem hiding this comment.
I found one failing integration-test expectation from the version-check order change.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 5024, then answer my questions about the findings.
Previous concerns have been addressed.
There was a problem hiding this comment.
The previous test-order blocker is fixed, and I don't have any new findings. This still touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 5085, then answer my questions about the findings.
There was a problem hiding this comment.
I don't have any new findings. This still touches core checkout behavior, so I'm leaving a comment rather than approving.
Want to dig deeper?
The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 5188, then answer my questions about the findings.
5 participants
Description
When
BUILDKITE_GIT_SPARSE_CHECKOUT_PATHSis set, the agent now injects--sparseand--filter=blob:noneinto git clone and--filter=blob:noneinto git fetch, so only the paths listed inBUILDKITE_GIT_SPARSE_CHECKOUT_PATHSare materialised and blobs outside the sparse set aren't downloaded up front.Each auto-injected flag respects a user-supplied equivalent — the agent won't add
--sparseif the user's clone flags already contain it, and won't add--filter=blob:noneif the user has--filter=…in eitherBUILDKITE_GIT_CLONE_FLAGSorBUILDKITE_GIT_FETCH_FLAGS(git takes the last--filteron the command line, so stacking ours after would silently override theirs; on fetch,git clone --filter=Xalso stores the filter inremote.origin.partialclonefilterand subsequent fetches inherit it automatically, so re-passing our own filter would override the user's choice for that fetch).The whole flow is gated on the local git actually supporting
sparse-checkout set --cone(>= 2.27). On older git, the agent falls back to a plain full checkout — no partial clone, no--sparse, no--coneattempt.The sparse-checkout decision is resolved once at the top of
defaultCheckoutPhaseviaplanSparseCheckout, which runs and validatesgit --versionat most once per checkout and only when sparse paths are configured.Changes
sparseCheckoutPlanstruct andplanSparseCheckoutfunction to perform git version check only once before clone startssingle-call sparse-checkout set --conelanded in 2.27; 2.26 only accepted--coneoninit).defaultCheckoutPhase, whensparsePlan.supported:--sparsetogitCloneFlagsunless the user already supplied it.--filter=blob:nonetogitCloneFlagsunless the user alreadysupplied a
--filter(checked viahasPartialCloneFilter).fetchSourcenow takes anautoAddBloblessFilter booland prepends--filter=blob:noneto fetch flags when set. The caller computes this assparsePlan.supported&& no--filterin clone flags && no--filterin fetch flags, so a user-supplied filter on either side is respected.hasPartialCloneFilterandhasSparseCheckoutFlaghelpers ininternal/job/git.go.TestCheckingOutLocalGitProjectWithSparseCheckoutAutoAddsBlobNoneFilter— verifies both auto-adds happen when the user supplies neither.
TestCheckingOutLocalGitProjectWithSparseCheckoutPreservesUserFilter— verifies a user-supplied
--filter=tree:0on clone is preserved and no--filter=blob:noneis prepended to fetch.TestCheckingOutLocalGitProjectWithSparseCheckoutFallsBackOnOldGit— verifies the git < 2.27 fallback: no auto-added flags, no
sparse-checkout set --cone, warning appears.Testing
go test ./...). Buildkite employees may check this if the pipeline has run automatically.go tool gofumpt -extra -w .)Disclosures / Credits
Claude code drafted all the tests and helped diagnose build failures. I applied and reviewed all changes.