Install the macOS docker CLI from a Homebrew bottle instead of source-building it - #4661
Merged
Conversation
…-building it Homebrew ships no Intel macOS bottle for the current docker formula, so 'brew install docker' compiles the CLI and builds Go in order to do it. That takes minutes and can exhaust the step budget. The macOS agents in the Azure Pipelines hosted pool are Intel, so this affects every macOS test job. Install-DockerCli.macos.ps1 installs the newest docker version that is bottled for Intel macOS, read from Homebrew's own OCI registry on ghcr.io. Bottles are content-addressed, so the download is verified against the digest the registry advertises rather than a checksum pinned here. Nothing is fetched from outside Homebrew, and the result is what 'brew install' would have produced. This replaces the brew install rather than acting as a fallback to it, so the path runs on every macOS job and a break surfaces immediately instead of only during the next Homebrew outage. colima still installs through brew: its bottle carries payloads outside bin/ and it depends on lima at runtime, so it cannot be installed by lifting a single binary. Also bounds the setup step with timeoutInMinutes, which it previously lacked entirely - it was limited only by the 90 minute job timeout.
paulmedynski
added
the
Area\Engineering
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The fixed 15-version search window will eventually exclude the last available Intel bottle.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Replaces source-building Docker CLI on Intel macOS agents with direct installation from a verified Homebrew bottle.
Changes:
- Adds OCI bottle discovery, verification, and extraction.
- Integrates the installer with a 30-minute setup timeout.
- Adds Pester coverage and test documentation.
File summaries
| File | Description |
|---|---|
eng/pipelines/scripts/Install-DockerCli.macos.ps1 |
Implements bottle installation. |
eng/pipelines/scripts/tests/Install-DockerCli.macos.Tests.ps1 |
Tests selection and integrity rules. |
eng/pipelines/scripts/tests/README.md |
Documents the new tests. |
eng/pipelines/common/templates/steps/configure-sql-server-macos-step.yml |
Uses the installer and adds a timeout. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The microsoft/mssql-release tap pins an openssl@3 formula that has no Intel macOS bottle, so 'brew install mssql-tools18' compiled openssl from source: 7,630 files, built in 12 minutes. That was the single largest cost in the macOS SQL setup step and, together with the Colima boot and the SQL image pull, pushed the step past its timeout. go-sqlcmd is a single bottled Go binary in homebrew-core with no openssl dependency. Verified against v1.10.0 that it parses the flags this step already uses (-S, -U, -P, -Q, -C, -No, -l), so the sqlcmd invocations are unchanged. Removing the tap also removes 'brew tap' and 'brew trust', and with them 'brew update', which the in-line comment justified solely as a prerequisite for making 'brew trust' available. The readiness check deliberately stays on the host rather than moving into the container: the tests connect from the host, so the probe has to traverse the Colima port-forward and the container port mapping to prove what the tests actually depend on. Raises the step timeout to 40 minutes. Excluding the openssl build, the observed run still needed ~18 minutes before 'docker run', and SQL readiness polling can add up to 6 more.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The bottle search will eventually age out, and the sqlcmd and timeout changes contradict the stated scope.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
eng/pipelines/scripts/Install-DockerCli.macos.ps1:49
- This fixed 15-version window makes the installer expire if Homebrew continues publishing Docker releases without Intel bottles: after 15 newer releases, the last usable bottle still exists in the registry but is never inspected, so every macOS setup fails. Scan version tags until a matching Intel bottle is found (or use another durable lookup that cannot age out) rather than relying on the usable version always remaining in the newest few.
# Each version costs a registry round-trip and the answer is always in the
# newest few.
$MaxVersionsScanned = 15
- Files reviewed: 4/4 changed files
- Comments generated: 2
- Review effort level: Balanced
Addresses review feedback on PR #4661. The 15-version scan window would have expired the workaround. Homebrew has stopped publishing Intel macOS bottles, so ARM-only tags accumulate at the head of the version list and the newest usable version sinks further down with every docker release. Once it passed index 15 the script would have failed even though a usable bottle remained. Verified against the live registry: homebrew/core/docker has 113 version tags, and the newest Intel-bottled one is already at index 1 (29.7.2-1) because 29.8.0 publishes only arm64 macOS and linux bottles. At the observed release cadence - 33 tags between 29.0.0 and 29.8.0 - the window would have closed in roughly three months. A manifest request costs ~0.2s, so walking the whole list is ~25s, and that only happens when no Intel bottle exists at all, which is already a broken state. The realistic degraded case costs a few seconds. Adds a test covering a long run of arm64-only versions ahead of a usable one. Confirmed load-bearing by reintroducing the cap: that test, and only that test, fails.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Registry pagination can construct invalid URLs, and the SQLCMD and timeout changes contradict the declared scope.
Review details
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
eng/pipelines/scripts/Install-DockerCli.macos.ps1:82
- The pagination target is always prefixed with
$Registry, but a registryLinktarget may already be absolute. In that case this produces a URL such ashttps://ghcr.iohttps://ghcr.io/..., so pagination fails and older bottled versions are never considered. Resolve the target against the current request URI so both relative and absolute links work; the existing tests currently provide noLinkheader and do not catch this path.
eng/pipelines/common/templates/steps/configure-sql-server-macos-step.yml:53
- This also replaces
mssql-tools18with the Go implementation, while the PR description explicitly saysmssql-tools18remains installed through Homebrew and that the rest of setup is untouched. That is a separate toolchain change with compatibility implications; either keep the existing SQLCMD installation or update the stated scope and document/validate the migration.
brew install sqlcmd
eng/pipelines/common/templates/steps/configure-sql-server-macos-step.yml:251
- The PR description says this step will be limited to 30 minutes, but the implementation configures 40 minutes. Please align the configured timeout with the documented behavior (or update the description if 40 minutes is intentional).
timeoutInMinutes: 40
- Files reviewed: 4/4 changed files
- Comments generated: 0 new
- Review effort level: Balanced
paulmedynski
marked this pull request as ready for review
September 8, 2026 13:29
paulmedynski
enabled auto-merge (squash)
September 8, 2026 13:29
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4661 +/- ##
==========================================
- Coverage 65.92% 64.45% -1.47%
==========================================
Files 290 284 -6
Lines 44987 67980 +22993
==========================================
+ Hits 29656 43817 +14161
- Misses 15331 24163 +8832
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
cheenamalhotra
approved these changes
Sep 8, 2026
Member
|
There's a failure in pipelines on macOS that may need attending to. |
Contributor
Author
|
The CI-SqlClient macOS failure is from SQL Server crashing inside the container. I've seen this a few times, so something to look into separately. The Ubuntu enclave failures (cancellations) are due to the requested image not existing in the AE pool - already fixed in #4515. These can be ignored here. |
mdaigle
approved these changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Homebrew has stopped publishing Intel macOS bottles, and the Azure Pipelines hosted macOS agents are Intel. Any formula whose current version has lost its Intel bottle is therefore compiled from source on our CI agents. This PR removes two such source builds from the macOS SQL Server setup step and bounds the step properly.
1. Docker CLI — installed from a bottle instead of source-built
brew install dockercompiles the CLI, and builds Go in order to do it. The siblingmicrosoft/mssql-rsrepo measured this across 147 runs of the equivalent step: bottled path 29s median, 1% failure; source-build path 441s median (774s max), 36% failure. See microsoft/mssql-rs#500, which this borrows the approach from.eng/pipelines/scripts/Install-DockerCli.macos.ps1(new) installs the newest docker version that is bottled for Intel macOS, read from Homebrew's own OCI registry on ghcr.io:sequoia/sonoma/venturabottle.docker/<version>/bin/docker), so nothing else from an anonymous download reaches disk.brew installwould have produced, with no hand-pinned version.This replaces the
brew installrather than acting as a fallback to it. A fallback that only runs during a Homebrew outage is untested precisely when it is needed; running it every time means a break surfaces immediately. The tradeoff is that we own this install path permanently.The version search walks the entire tag list newest-first rather than a fixed window.
homebrew/core/dockerhas 113 version tags and the newest Intel-bottled one is already at index 1 (29.7.2-1), because 29.8.0 ships onlyarm64_*andx86_64_linux. A manifest request costs ~0.22s, so a full walk is ~25s and only occurs when no Intel bottle exists at all.2. sqlcmd — go-sqlcmd instead of
mssql-tools18The first CI run on this branch timed out, and the docker CLI was not the reason — it installed in 4.7s. The dominant cost was
brew install mssql-tools18at 750s (12.5 min): themicrosoft/mssql-releasetap pins anopenssl@3formula with no Intel bottle, so Homebrew compiled openssl (7,630 files, built in 12 minutes). That is the same root cause as the docker problem, in the same step.mssql-tools18is replaced bysqlcmd(go-sqlcmd) from homebrew-core — a single bottled Go binary with no openssl dependency. This also removesbrew tap,brew trust, andbrew update, the last of which the in-line comment justified solely as a prerequisite forbrew trust.The readiness probe deliberately stays on the host rather than moving into the container via
docker exec: the tests connect from the host, so the probe must traverse the Colima port-forward and the container port mapping to prove what the tests actually depend on.3. Step timeout
The step previously had no
timeoutInMinutesat all and was bounded only by the 90-minute job timeout — which is why a slow install surfaced as an opaque job timeout with no diagnostics. It is now 40 minutes, sized from measurement: excluding the openssl build, the observed run still needed ~18 minutes to reachdocker run(Colima boot 5.6 min; SQL image pull 9.6 min, 7 of which is extraction inside the VM), and SQL readiness polling can add up to 6 more.Issues
None — proactive fix for a latent CI failure.
Testing
Ran CI-SqlClient to confirm macOS job succeeds.