Skip to content

Pipelines | Name the agent image parameter poolImage everywhere and require it - #4626

Merged
paulmedynski merged 4 commits into
mainfrom
dev/paul/pool-image
Sep 10, 2026
Merged

Pipelines | Name the agent image parameter poolImage everywhere and require it#4626
paulmedynski merged 4 commits into
mainfrom
dev/paul/pool-image

Conversation

@paulmedynski

@paulmedynski paulmedynski commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

The agent image parameter was spelled five different ways across the pipeline templates, and ten of them carried a default, so a template could silently select an agent image that its caller never mentioned. This makes the parameter uniform and makes every image an explicit choice at the call site.

  • Rename to poolImage. All 21 declarations, references and call-site keys, replacing vmImage (16), platformImage (2), image (1), agentImage (1) and imageOverride (1). This pairs with the poolName parameter it always accompanies.
  • Remove the defaults. Ten declarations carried one. The nine that live in templates with callers lose theirs, and the 9 call sites that had been inheriting a value now pass it explicitly.
  • Keep one default, on the CI package pipeline's top-level parameter. It has no caller to supply a value: it exists so developers can pick the pool image manually when queueing a run, and it defaults to ADO-UB24 for automated runs.

Not renamed

These keep their names because they are Azure Pipelines' own vocabulary, not ours:

  • the vmImage: key inside a pool: block (4 sites)
  • the imageOverride demand capability (22 sites)

Also untouched: the image field of the PR pipeline's platforms objects and the CI matrix's images map, which are data structures rather than template parameters; and eng/pipelines/onebranch/, which governs its own pools.

Call sites that gained an explicit image

call site template image
dotnet-sqlclient-ci-core.yml compute-versions-ci-stage ADO-UB24
dotnet-sqlclient-ci-core.yml generate-secrets-ci-stage ADO-UB24
dotnet-sqlclient-ci-core.yml verify-nuget-packages-ci-stage ADO-Win25
dotnet-sqlclient-ci-core.yml ci-code-coverage-job ADO-UB24
build-abstractions-package-ci-stage.yml pack-abstractions-package-ci-job ADO-UB24
build-azure-package-ci-stage.yml pack-azure-package-ci-job ADO-UB24
build-logging-package-ci-stage.yml pack-logging-package-ci-job ADO-UB24
build-sqlserver-package-ci-stage.yml pack-sqlserver-package-ci-job ADO-UB24
build-sqlclient-package-ci-stage.yml ci-build-nugets-job ADO-Win25

Validation

This is a pure refactor - no job changes the image it runs on.

check result
YAML parses across eng/pipelines/** 92 files, 0 failures
Leftover old parameter names 0
poolImage declarations 21, exactly 1 with a default
Call sites missing a required poolImage 0
Effective image per call site, before vs after 41 call sites, 0 mismatches

The last check resolves every call site to the image it actually uses - explicit value, or the default it inherited - at both the parent commit and this branch, and confirms all 41 land on the identical image.

These changes have not been executed against Azure DevOps.

Pipeline runs

Base URL for the Public project: https://dev.azure.com/SqlClientDrivers/public/_build/results?buildId=

Public

pipeline trigger run result
PR-SqlClient-Project (2197) automatic on PR push 173337 partially succeeded - no failed stage
sqlclient-pr (2281) automatic on PR push 173338 succeeded
sqlclient-ci-package (2300) queued manually on refs/pull/4626/merge 173343 succeeded
CI-SqlClient (1879) queued manually on refs/pull/4626/merge 173344 partially succeeded - no failed stage after retry

CI-SqlClient-Package (1917), PR-SqlClient-Package (2198) and sqlclient-ci-stress (2250) are affected but disabled, so there is nothing to run.

ADO.Net

These build the internal dotnet-sqlclient repo, so no GitHub PR can reach them. They are queued manually against a branch carrying this change. sqlclient-ci-kerberos and sqlclient-ci-managed-instance matter most - they are the only affected pipelines whose templates a GitHub PR cannot exercise at all.

All runs below are on refs/heads/dev/paul/pool-image in the internal repo.

pipeline run result
sqlclient-ci-kerberos (2306) 20260908.4 succeeded
sqlclient-ci-managed-instance (2308) 25102 succeeded
sqlclient-ci-package (2277) 25102 succeeded
sqlclient-pr (2271) 25101 succeeded
MDS Main CI (1825) 25104 partially succeeded - no failed stage after retry
MDS Main CI-Package (1933) 25104 failed - only MacOSLatest_Sql25, see note below

A poolImage mismatch is a template-expansion error, which Azure DevOps reports at queue time before any agent is allocated - so a run that compiles is already proof for these, even if it is cancelled before the tests finish.

sqlclient-ci-kerberos and sqlclient-ci-managed-instance both passed end to end, so the two templates that no GitHub PR can reach are verified by real runs rather than by compilation alone.

Failures and retries

Every failure observed was an infrastructure flake on agents this change does not touch. Across all 10 runs, MacOSLatest_Sql25 and Win11_Azure_Sql_ARM64 were the only stages that ever failed - no stage that consumes a renamed poolImage parameter failed in any run.

stage failure relation to this PR
MacOSLatest_Sql25 Configure SQL Server [macOS] exits 1, then Publish Test Artifacts finds no TestResults. On the retry in MDS Main CI-Package the job instead hit the 90 minute timeout on hosted agent Azure Pipelines 6. none - these jobs select their agent with pool: vmImage: macos-latest, one of the 4 Azure Pipelines keys deliberately left unrenamed
Win11_Azure_Sql_ARM64 Download SqlServer Package Artifacts returns Too Many Requests; passed on retry none - artifact download throttling

Retries cleared the macOS stage in PR-SqlClient-Project, CI-SqlClient and MDS Main CI, and the ARM64 stage in MDS Main CI-Package. Only the macOS stage in MDS Main CI-Package is still red, for the timeout above.

The partially succeeded results are the normal outcome for these pipelines: the quarantined flaky-test steps report succeededWithIssues, which downgrades the run without any task failing.

Not affected

OneBranch (2240/2241), perf (2305/2311/2313), github-sync (2263), the sni and ctaip pipelines, and the docs pipelines.

Copilot AI balanced review requested due to automatic review settings September 2, 2026 16:50
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Sep 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The refactor preserves image selections, and all 38 call sites supply the 19 newly required template parameters.

Pull request overview

Standardizes agent-image template parameters as poolImage across CI/PR pipelines while preserving the image selections introduced by #4515.

Changes:

  • Renames all custom image parameters to poolImage.
  • Removes template defaults and explicitly supplies images at every call site.
  • Retains the standalone package pipeline’s queue-time default.
File summaries
File Description
eng/pipelines/stages/verify-nuget-packages-ci-stage.yml Requires poolImage for verification.
eng/pipelines/stages/generate-secrets-ci-stage.yml Requires poolImage for secret generation.
eng/pipelines/stages/compute-versions-ci-stage.yml Requires poolImage for version computation.
eng/pipelines/stages/build-sqlserver-package-ci-stage.yml Supplies the pack-job image.
eng/pipelines/stages/build-sqlclient-package-ci-stage.yml Supplies the NuGet build image.
eng/pipelines/stages/build-logging-package-ci-stage.yml Supplies the logging pack image.
eng/pipelines/stages/build-azure-package-ci-stage.yml Renames test images and supplies the pack image.
eng/pipelines/stages/build-abstractions-package-ci-stage.yml Renames test images and supplies the pack image.
eng/pipelines/pr/stages/test-stages.yml Passes platform images using poolImage.
eng/pipelines/pr/stages/pack-stage.yml Renames the required pack-stage image parameter.
eng/pipelines/pr/stages/generate-secrets-stage.yml Renames the required secrets-stage image parameter.
eng/pipelines/pr/stages/collect-coverage-stage.yml Renames the required coverage image parameter.
eng/pipelines/pr/sqlclient-pr-pipeline.yml Updates PR stage call sites.
eng/pipelines/pr/jobs/test-sqlclientmanual-job.yml Standardizes the manual-test image parameter.
eng/pipelines/pr/jobs/test-buildproj-job.yml Standardizes the build-project image parameter.
eng/pipelines/jobs/test-azure-package-ci-job.yml Standardizes Azure test image references.
eng/pipelines/jobs/test-abstractions-package-ci-job.yml Standardizes Abstractions test image references.
eng/pipelines/jobs/pack-sqlserver-package-ci-job.yml Requires an explicit pack image.
eng/pipelines/jobs/pack-logging-package-ci-job.yml Requires an explicit pack image.
eng/pipelines/jobs/pack-azure-package-ci-job.yml Requires an explicit pack image.
eng/pipelines/jobs/pack-abstractions-package-ci-job.yml Requires an explicit pack image.
eng/pipelines/dotnet-sqlclient-ci-core.yml Supplies explicit images to core stages and jobs.
eng/pipelines/common/templates/stages/ci-run-tests-stage.yml Passes matrix images as poolImage.
eng/pipelines/common/templates/jobs/ci-run-tests-job.yml Standardizes test-job image handling.
eng/pipelines/common/templates/jobs/ci-code-coverage-job.yml Requires an explicit coverage image.
eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml Standardizes the NuGet build image parameter.
eng/pipelines/ci/stress/sqlclient-ci-stress-stage.yml Updates stress-job image arguments.
eng/pipelines/ci/stress/sqlclient-ci-stress-pipeline.yml Updates the secrets-stage image argument.
eng/pipelines/ci/stress/sqlclient-ci-stress-job.yml Standardizes stress-job image handling.
eng/pipelines/ci/package/sqlclient-ci-package-pipeline.yml Renames the queue-time image parameter while retaining its default.
eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-stage.yml Updates managed-instance job arguments.
eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-job.yml Standardizes the managed-instance image parameter.
eng/pipelines/ci/kerberos/sqlclient-ci-kerberos-pipeline.yml Updates the coverage-job image argument.
Review details
  • Files reviewed: 33/33 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@paulmedynski paulmedynski added the Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems. label Sep 2, 2026
@paulmedynski paulmedynski added this to the 7.1.0 milestone Sep 2, 2026
@paulmedynski paulmedynski moved this from To triage to In progress in SqlClient Board Sep 2, 2026
cheenamalhotra
cheenamalhotra previously approved these changes Sep 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved conflict markers invalidate the Kerberos YAML, and managed-instance callers still use the removed parameter name.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 32/32 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread eng/pipelines/ci/kerberos/sqlclient-ci-kerberos-pipeline.yml Outdated
Copilot AI review requested due to automatic review settings September 8, 2026 12:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Managed-instance callers are now incompatible with the renamed required parameter, and the Kerberos templates were omitted from the stated repository-wide rename.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-job.yml:75

  • All four callers in sqlclient-ci-managed-instance-stages.yml still pass vmImage (lines 56, 70, 82, and 100). Because this renamed parameter has no default, Azure template expansion will receive an unexpected vmImage key and no required poolImage, so the managed-instance pipeline cannot compile. Rename those caller keys to poolImage.
  - name: poolImage
  • Files reviewed: 31/31 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread eng/pipelines/ci/package/sqlclient-ci-package-pipeline.yml
paulmedynski added a commit that referenced this pull request Sep 8, 2026
… templates

Complete the rename started in PR #4626 so template expansion matches the
renamed declarations:

- sqlclient-ci-kerberos-job.yml: rename the vmImage parameter to poolImage
  and update the ImageOverride demand.
- sqlclient-ci-kerberos-stages.yml: rename all four call-site keys.
- sqlclient-ci-managed-instance-stages.yml: rename all four call-site keys
  to match the already-renamed poolImage parameter.

Addresses review feedback on PR #4626.
Copilot AI review requested due to automatic review settings September 8, 2026 12:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

All required template call sites supply poolImage; only the non-blocking validation counts in the description need correction.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

eng/pipelines/ci/package/sqlclient-ci-package-pipeline.yml:46

  • The validation counts are stale: the current tree has 21 poolImage declarations (20 required template parameters plus this queue-time parameter) and 41 textual template call sites, rather than 20 and 38. Please update the PR description and ensure the effective-image comparison covers all 41 call sites so its validation evidence matches the final diff.
  • Files reviewed: 34/34 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.04%. Comparing base (ce9f01b) to head (3d441a2).
⚠️ Report is 6 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (ce9f01b) and HEAD (3d441a2). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (ce9f01b) HEAD (3d441a2)
CI-SqlClient 1 0
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4626      +/-   ##
==========================================
- Coverage   71.88%   64.04%   -7.84%     
==========================================
  Files         290      284       -6     
  Lines       45056    68006   +22950     
==========================================
+ Hits        32389    43557   +11168     
- Misses      12667    24449   +11782     
Flag Coverage Δ
CI-SqlClient ?
PR-SqlClient-Project 64.04% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@paulmedynski
paulmedynski marked this pull request as ready for review September 8, 2026 17:53
@paulmedynski
paulmedynski requested a review from a team as a code owner September 8, 2026 17:53
@paulmedynski paulmedynski moved this from In progress to In review in SqlClient Board Sep 8, 2026
Base automatically changed from dev/automation/ci-retire-hosted-agents to main September 9, 2026 00:06
@mdaigle
mdaigle force-pushed the dev/paul/pool-image branch from 9d91494 to 1102240 Compare September 9, 2026 00:06
paulmedynski added a commit that referenced this pull request Sep 9, 2026
… templates

Complete the rename started in PR #4626 so template expansion matches the
renamed declarations:

- sqlclient-ci-kerberos-job.yml: rename the vmImage parameter to poolImage
  and update the ImageOverride demand.
- sqlclient-ci-kerberos-stages.yml: rename all four call-site keys.
- sqlclient-ci-managed-instance-stages.yml: rename all four call-site keys
  to match the already-renamed poolImage parameter.

Addresses review feedback on PR #4626.
Copilot AI review requested due to automatic review settings September 9, 2026 10:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Template declarations, references, and callers are consistently aligned, with successful pipeline runs confirming expansion behavior.

Review details
  • Files reviewed: 34/34 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

The image parameter was spelled five different ways across the pipeline
templates - vmImage, image, agentImage, imageOverride and platformImage - so a
reader had to check each template to learn what to pass.  Rename all twenty
declarations, references and call-site keys to 'poolImage', alongside the
'poolName' parameter it accompanies.

Remove the image defaults from the nineteen templates that have callers, and
pass the value explicitly at the nine call sites that had been inheriting one.
A default meant a template silently picked an agent image that its caller never
mentioned, which is easy to miss when auditing which images a pool must
provide.  Every call site now names its image.

The queue-time parameter on the CI package pipeline keeps its default, since it
has no caller to supply one, and the default lets a manual run pick a Windows
agent when that is what we want to validate.

This does not change which image any job runs on.  All 38 call sites resolve to
the same image as before, whether they previously passed a value or inherited a
default.

Note that ADO's own 'vmImage' pool keyword and the 'imageOverride' demand
capability keep their names; only our parameters are renamed.
… templates

Complete the rename started in PR #4626 so template expansion matches the
renamed declarations:

- sqlclient-ci-kerberos-job.yml: rename the vmImage parameter to poolImage
  and update the ImageOverride demand.
- sqlclient-ci-kerberos-stages.yml: rename all four call-site keys.
- sqlclient-ci-managed-instance-stages.yml: rename all four call-site keys
  to match the already-renamed poolImage parameter.

Addresses review feedback on PR #4626.
Copilot AI review requested due to automatic review settings September 9, 2026 11:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

All renamed templates have explicit matching call-site values, and effective agent selection is preserved.

Review details
  • Files reviewed: 34/34 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

cheenamalhotra
cheenamalhotra previously approved these changes Sep 9, 2026
Copilot AI review requested due to automatic review settings September 9, 2026 17:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

All 41 affected call sites supply the required parameter, preserve their prior images, and have corroborating pipeline validation.

Review details
  • Files reviewed: 34/34 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@priyankatiwari08 priyankatiwari08 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified every caller of the renamed templates passes poolImage; no stale vmImage/platformImage/imageOverride parameter references remain. One concern:

  • sqlclient-ci-package-pipeline.yml: agentImage is a queue-time parameter, so renaming it breaks saved run configurations and any API/CLI trigger that passes agentImage.

Comment thread eng/pipelines/ci/package/sqlclient-ci-package-pipeline.yml
@paulmedynski
paulmedynski merged commit 8725663 into main Sep 10, 2026
212 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in SqlClient Board Sep 10, 2026
@paulmedynski
paulmedynski deleted the dev/paul/pool-image branch September 10, 2026 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants