Skip to content

teuthology/repo_utils: optimize repository cloning via bare repositories and worktrees - #2207

Merged
zmc merged 2 commits into
ceph:mainfrom
batrick:clone-once
May 27, 2026
Merged

teuthology/repo_utils: optimize repository cloning via bare repositories and worktrees#2207
zmc merged 2 commits into
ceph:mainfrom
batrick:clone-once

Conversation

@batrick

@batrick batrick commented May 27, 2026

Copy link
Copy Markdown
Member

Overview

This patchset optimizes repository management in teuthology by introducing a shared bare repository and git worktree strategy, while also fixing a directory naming bug where trailing repository URL characters were incorrectly stripped. Together, these changes reduce network overhead, speed up testing cycles, and lower disk/IO utilization.

Changes

1. Repository Cloning Optimization via Worktrees

  • Initializes a single bare clone of a target repository instead of performing repeated full clones for different commits or branches.
  • Utilizes git worktree to provision lightweight, isolated workspaces for requested commits, reducing redundant network fetches.
  • Checks if a target commit is already locally present in the bare repository before attempting a remote fetch.
  • Implements robust worktree pruning and error handling, falling back to standard directory removal if git worktree cleanup fails.

2. Precise Directory Naming Fix

  • Replaces string.rstrip('.git') with an exact substring check (endswith).
  • Prevents the unintended truncation of legitimate repository name characters that happen to be present in the word ".git".

batrick added 2 commits May 27, 2026 11:38
@batrick
teuthology/repo_utils: strip only .git suffix
4383c30 
Previous code with strip any trailing letters in .git.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
@batrick
teuthology/repo_utils: Optimize repository cloning via bare repositor…
b57d834 
…ies and worktrees

Repeatedly performing full clones of the QA suite and other repositories
for different commits or branches slows down testing iterations and
consumes excess disk space and I/O.

This commit introduces an optimization that initializes a single bare
clone of a target repository. It subsequently utilizes git worktree to
provision lightweight, isolated workspaces for requested commits. This
reduces redundant network fetches and significantly accelerates
repository checkout operations.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
@batrick
batrick requested a review from a team as a code owner May 27, 2026 15:47
@batrick
batrick requested review from amathuria and zmc and removed request for a team May 27, 2026 15:47
@batrick

batrick commented May 27, 2026

Copy link
Copy Markdown
Member Author

I split off this change from #2012

@batrick

batrick commented May 27, 2026

Copy link
Copy Markdown
Member Author

@zmc

Before:

(teuthology) pdonnell@soko04 ~$ time teuthology-suite --dry-run --ceph main --ceph-repo https://github.com/ceph/ceph -S e33ecece82ee6e7822fa77deeef68a08b30fd121 --suite-repo https://github.com/batrick/ceph --suite-branch wip-pdonnell-testing --suite smoke 
2026-05-27 18:08:37,786.786 INFO:teuthology.suite:Using random seed=9886
2026-05-27 18:08:37,787.787 INFO:teuthology.suite.run:Checking for expiration (None)
2026-05-27 18:08:37,787.787 INFO:teuthology.suite.run:kernel sha1: distro
2026-05-27 18:08:38,129.129 INFO:teuthology.suite.run:ceph sha1 explicitly supplied
2026-05-27 18:08:38,129.129 INFO:teuthology.suite.run:ceph sha1: e33ecece82ee6e7822fa77deeef68a08b30fd121
2026-05-27 18:08:38,337.337 INFO:teuthology.suite.run:ceph branch: wip-pdonnell-testing 40c89c4e457f23e3832f4739133a9fbf998abebb
2026-05-27 18:08:38,340.340 INFO:teuthology.repo_utils:Cloning https://github.com/batrick/ceph wip-pdonnell-testing from upstream
2026-05-27 18:09:37,093.093 INFO:teuthology.suite.run:teuthology branch: main 456e338aa0e568d90c775bdad1af84945d891e03
2026-05-27 18:09:37,204.204 INFO:teuthology.suite.run:Suite smoke in /tank/home/pdonnell/src/github.com_batrick_ceph_40c89c4e457f23e3832f4739133a9fbf998abebb/qa/suites/smoke generated 23 jobs (not yet filtered or merged)
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/cfuse_workunit_suites_blogbench}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/cfuse_workunit_suites_fsstress}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/cfuse_workunit_suites_iozone}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_pjd}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/kclient_workunit_direct_io}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_dbench}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_fsstress}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/kclient_workunit_suites_pjd}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/libcephfs_interface_tests}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/mon_thrash}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rados_api_tests}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rados_bench}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rados_cls_all}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rados_ec_snaps}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rados_python}}
2026-05-27 18:09:38,259.259 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rados_workunit_loadgen_mix}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rbd_api_tests}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rbd_cli_import_export}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rbd_fsx}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rbd_python_api_tests}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rbd_workunit_suites_iozone}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rgw_ec_s3tests}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rgw_s3tests}}
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Suite smoke in /tank/home/pdonnell/src/github.com_batrick_ceph_40c89c4e457f23e3832f4739133a9fbf998abebb/qa/suites/smoke scheduled 23 jobs.
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:0/23 jobs were filtered out.
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Scheduled 23 jobs in total.
2026-05-27 18:09:38,260.260 INFO:teuthology.suite.run:Test results viewable at https://pulpito-ng.ceph.com/pdonnell-2026-05-27_18:08:37-smoke-main-distro-default-trial/

real    1m2.391s
user    3m46.911s
sys     0m21.075s

After:

(teuthology) pdonnell@soko04 ~$ time teuthology-suite --dry-run --ceph main --ceph-repo https://github.com/ceph/ceph -S e33ecece82ee6e7822fa77deeef68a08b30fd121 --suite-repo https://github.com/batrick/ceph --suite-branch wip-pdonnell-testing --suite smoke 
2026-05-27 18:07:35,708.708 INFO:teuthology.suite:Using random seed=7291
2026-05-27 18:07:35,709.709 INFO:teuthology.suite.run:Checking for expiration (None)
2026-05-27 18:07:35,709.709 INFO:teuthology.suite.run:kernel sha1: distro
2026-05-27 18:07:35,789.789 INFO:teuthology.suite.run:ceph sha1 explicitly supplied
2026-05-27 18:07:35,790.790 INFO:teuthology.suite.run:ceph sha1: e33ecece82ee6e7822fa77deeef68a08b30fd121
2026-05-27 18:07:35,985.985 INFO:teuthology.suite.run:ceph branch: wip-pdonnell-testing d758a2eada1206ca4498e95f4ba74d4b610f8d39
2026-05-27 18:07:35,988.988 INFO:teuthology.repo_utils:enforce_repo_state dest_clone=/tank/home/pdonnell/src/github.com_batrick_ceph dest_path=/tank/home/pdonnell/src/github.com_batrick_ceph_d758a2eada1206ca4498e95f4ba74d4b610f8d39 repo_url=https://github.com/batrick/ceph branch=wip-pdonnell-testing commit=d758a2eada1206ca4498e95f4ba74d4b610f8d39
2026-05-27 18:07:35,988.988 INFO:teuthology.repo_utils:Fetching into bare repo /tank/home/pdonnell/src/github.com_batrick_ceph from https://github.com/batrick/ceph (branch: wip-pdonnell-testing, commit: d758a2eada1206ca4498e95f4ba74d4b610f8d39)
2026-05-27 18:07:36,395.395 INFO:teuthology.repo_utils:Setting up worktree at /tank/home/pdonnell/src/github.com_batrick_ceph_d758a2eada1206ca4498e95f4ba74d4b610f8d39 from bare repo /tank/home/pdonnell/src/github.com_batrick_ceph using ref d758a2eada1206ca4498e95f4ba74d4b610f8d39
2026-05-27 18:07:46,761.761 INFO:teuthology.suite.run:teuthology branch: main 456e338aa0e568d90c775bdad1af84945d891e03
2026-05-27 18:07:46,830.830 INFO:teuthology.suite.run:Suite smoke in /tank/home/pdonnell/src/github.com_batrick_ceph_d758a2eada1206ca4498e95f4ba74d4b610f8d39/qa/suites/smoke generated 23 jobs (not yet filtered or merged)
2026-05-27 18:07:47,894.894 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/cfuse_workunit_suites_blogbench}}
2026-05-27 18:07:47,894.894 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_fsstress}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/cfuse_workunit_suites_iozone}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/cfuse_workunit_suites_pjd}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/kclient_workunit_direct_io}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_dbench}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/kclient_workunit_suites_fsstress}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/kclient_workunit_suites_pjd}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/libcephfs_interface_tests}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/mon_thrash}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rados_api_tests}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rados_bench}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rados_cls_all}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rados_ec_snaps}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rados_python}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rados_workunit_loadgen_mix}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rbd_api_tests}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{rpm_latest} tasks/{0-install test/rbd_cli_import_export}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rbd_fsx}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rbd_python_api_tests}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{centos_9.stream} tasks/{0-install test/rbd_workunit_suites_iozone}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rgw_ec_s3tests}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduling smoke/basic/{clusters/{fixed-3-cephfs} objectstore/bluestore-bitmap supported-random-distro/{ubuntu_latest} tasks/{0-install test/rgw_s3tests}}
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Suite smoke in /tank/home/pdonnell/src/github.com_batrick_ceph_d758a2eada1206ca4498e95f4ba74d4b610f8d39/qa/suites/smoke scheduled 23 jobs.
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:0/23 jobs were filtered out.
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Scheduled 23 jobs in total.
2026-05-27 18:07:47,895.895 INFO:teuthology.suite.run:Test results viewable at https://pulpito-ng.ceph.com/pdonnell-2026-05-27_18:07:35-smoke-main-distro-default-trial/

real    0m14.098s
user    0m4.504s
sys     0m1.311s

zmc
zmc approved these changes May 27, 2026
View reviewed changes

@zmc zmc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on!

@zmc
zmc merged commit 89ff969 into ceph:main May 27, 2026
10 checks passed
@zmc zmc mentioned this pull request May 27, 2026
Closed
@batrick
batrick deleted the clone-once branch May 27, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zmc zmc zmc approved these changes
@amathuria amathuria Awaiting requested review from amathuria amathuria is a code owner automatically assigned from ceph/teuthology

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@batrick @zmc