Unveil environments setup-local: make the command visible - #5835
Merged
Conversation
This was referenced Jul 6, 2026
Collaborator
Integration test reportCommit: b6a88d8
9 interesting tests: 4 RECOVERED, 4 SKIP, 1 flaky
Top 14 slowest tests (at least 2 minutes):
|
This was referenced Jul 6, 2026
rugpanov
force-pushed
the
dbconnect/07-acceptance
branch
from
July 6, 2026 11:47
6c2fadc to
c45ca0c
Compare
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
July 6, 2026 11:47
3123548 to
8008a65
Compare
rugpanov
force-pushed
the
dbconnect/07-acceptance
branch
from
July 7, 2026 08:35
c45ca0c to
25d07fe
Compare
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
July 7, 2026 08:35
8008a65 to
27f8064
Compare
ZaSkittles
pushed a commit
to ZaSkittles/cli
that referenced
this pull request
Jul 7, 2026
databricks#5823) ## Why - The `local-env` feature needs a shared vocabulary before any behavior can be built: the result shape, the error taxonomy, and how a compute target maps to an environment key. - Landing these contract types first lets every later layer (resolve / fetch / merge / pipeline / command) depend on stable, reviewed definitions. - Kept deliberately minimal and dependency-free so it reviews on its own and stays `unused`/`deadcode`-clean with no consumers yet. ## What - **`result.go`** — the `--json` / `E_*` output contract: `Result`, `PipelineError`, `ErrorCode`, `PhaseName`, `PhaseStatus`, `Mode`, `TargetInfo`, `ResolvedInfo`, `Plan`, `Warning`; plus the command-path constants (`local-env` / `python` / `sync`) defined in one place. - **`envkey.go`** — `EnvKeyForServerless` / `EnvKeyForSparkVersion` / `NormalizeServerless`, and `PythonMinorFromRequires` (clause-aware: returns the effective highest lower bound of a `requires-python`). - No wiring into `cmd/`, so the CLI is unchanged. Filesystem/artifact constants and the phase-order slice deliberately live with their consumer (PR 5). ## Testing strategy - Unit tests for the error/type contract (`result_test.go`) and env-key mapping incl. multi-clause / strict-`>` / no-floor `requires-python` cases (`envkey_test.go`). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex across several rounds to convergence (all findings fixed or explicitly rejected as speculative). --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | **databricks#5823 ← you are here** | foundation: result types + env-key mapping | | 2 | databricks#5824 | compute-target resolution | | 3 | databricks#5826 | constraint fetch + offline cache | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.
ZaSkittles
pushed a commit
to ZaSkittles/cli
that referenced
this pull request
Jul 7, 2026
## Why - `local-env` must turn the user's compute selection into a single environment key before it can fetch anything, and the selection can come from several places with a defined precedence. - Isolating resolution behind a narrow seam keeps it testable without a live workspace and keeps SDK details out of the engine. ## What - **`target.go`** — `ResolveTarget` with ordered precedence `--cluster` → `--serverless` → `--job` → bundle target, producing a `TargetInfo` + env key. - Compute lookups go through the narrow `ComputeClient` interface (stubbable in tests). - `ValidateTargetFlags` rejects more than one target flag; `ResolveTarget` runs it up front so a non-Cobra caller can't silently resolve the wrong target. - Classic-compute jobs read the Spark version from the documented first return of `GetJobSparkVersion` (not the recorded-version third return). ## Testing strategy - Unit tests against a stub `ComputeClient` covering each precedence branch, the mutually-exclusive-flags error, and the job classic-compute contract (`target_test.go`). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex to a clean pass. --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | databricks#5823 | foundation: result types + env-key mapping | | 2 | **databricks#5824 ← you are here** | compute-target resolution | | 3 | databricks#5826 | constraint fetch + offline cache | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.
ZaSkittles
pushed a commit
to ZaSkittles/cli
that referenced
this pull request
Jul 7, 2026
…icks#5826) ## Why - Once a target resolves to an env key, `local-env` needs the pinned Python version, `databricks-connect` version, and dependency constraints published for that key. - The fetch must degrade gracefully offline and distinguish “this environment isn't published” from “the network is down,” because those call for different user action. - The artifact host must be a Databricks-owned, access-controlled location and must never default to a personal repo — whoever controls the host controls what the CLI installs. ## What - **`constraints.go`** — fetches the per-environment `pyproject.toml`, parses `requires-python`, the `databricks-connect` pin, and `[tool.uv]` `constraint-dependencies`, and caches it on disk. - **Host parameterization** — no host is hardcoded: `RepoConstraintBaseURL` reads the repo (`owner/name`) from the temporary `DATABRICKS_LOCALENV_CONSTRAINT_REPO` env var and builds a `raw.githubusercontent.com/<repo>/main` URL; the built-in default is empty. When unset it returns `""` and `FetchConstraints` reports the missing source as a fetch-phase `E_FETCH` error (so there is no untrusted default, and the failure flows through the normal phase/JSON reporting). Once `databricks/environments` can publish, that becomes the hardcoded default and the env var is no longer required. - Failure classification: **404** → `E_ENV_UNSUPPORTED` (no cache fallback — a distinct non-transient condition); **transport / non-404** → `E_FETCH` with fallback to the last-good cached copy. - Robustness: validate the body (parse + require `requires-python`) **before** caching so a bad 2xx can't poison the cache; atomic cache write (mkdir + temp-file + rename); dedicated `http.Client` with a 30s timeout; body read bounded by `io.LimitReader` at 1 MiB; `databricks-connect` matched by leading package name under PEP 503 normalization (so `Databricks_Connect` matches, `databricks-connectors` does not); cache filename = readable slug + sha256 suffix to prevent collisions. ## Testing strategy - Unit tests with an `httptest` server: 200-parse, 404 → `E_ENV_UNSUPPORTED`, transport failure + cache fallback, missing-`requires-python` rejection, PEP 503 name matching, cache-dir creation, collision-free filenames, oversized-body rejection (`constraints_test.go`). - Host resolution: `TestRepoConstraintBaseURL` (env var → URL, unset → `""`, whitespace treated as unset) and `TestFetchConstraintsNoSourceConfigured` (empty host → `E_FETCH` naming the env var). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex to a clean pass (several fetch/cache edge-case fixes landed from review). --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | databricks#5823 | foundation: result types + env-key mapping | | 2 | databricks#5824 | compute-target resolution | | 3 | **databricks#5826 ← you are here** | constraint fetch + offline cache | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.
ZaSkittles
pushed a commit
to ZaSkittles/cli
that referenced
this pull request
Jul 7, 2026
…atabricks#5827) ## Why - `local-env` must apply the resolved Python version and constraints to the user's `pyproject.toml` without disturbing their own content — comments, ordering, formatting, and unrelated config must survive untouched. - Re-running must be safe and idempotent, and a greenfield project needs a sensible file created from scratch. - This is the most intricate logic in the feature, so it lands in its own PR for focused review. ## What - **`merge.go`** — a formatting-preserving merge that rewrites only the env-owned regions (`requires-python`, the `databricks-connect` entry in `[dependency-groups].dev`, and a marker-bracketed managed `[tool.uv]` block) and preserves every other byte incl. CRLF; idempotent. `RenderFreshPyproject` builds a complete managed file for a greenfield project. - Scoping/robustness: managed `constraint-dependencies` nests header-less inside an existing user `[tool.uv]` (never a duplicate header); single- vs multi-line array detection tracks real bracket depth outside strings/comments; the `databricks-connect` rewrite is confined to `dev` and leaves trailing comments alone; `requires-python`'s inline comment is preserved; table-header parsing tolerates inline comments and recognizes `[[array.of.tables]]`. ## Testing strategy - Unit tests that parse the merged output as TOML (not just substring checks), covering idempotency, CRLF preservation, user-key preservation, the duplicate-`[tool.uv]` case, bracket-in-element arrays, sibling-group/comment non-clobbering, and `[[tool.uv.index]]` children (`merge_test.go`). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex to a clean pass (multiple TOML-corruption edge cases were caught and fixed). --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | databricks#5823 | foundation: result types + env-key mapping | | 2 | databricks#5824 | compute-target resolution | | 3 | databricks#5826 | constraint fetch + offline cache | | 4 | **databricks#5827 ← you are here** | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.
rugpanov
force-pushed
the
dbconnect/07-acceptance
branch
from
July 14, 2026 11:51
25d07fe to
da5bf75
Compare
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
July 14, 2026 11:52
27f8064 to
8fb5ea2
Compare
rugpanov
force-pushed
the
dbconnect/07-acceptance
branch
from
July 14, 2026 12:50
da5bf75 to
b537940
Compare
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
July 14, 2026 12:50
8fb5ea2 to
d3c1357
Compare
rugpanov
force-pushed
the
dbconnect/07-acceptance
branch
from
July 16, 2026 11:31
c6d4042 to
58b80d9
Compare
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
July 16, 2026 11:32
3bea91d to
43f5af9
Compare
Contributor
Author
|
⛔ Do not merge until databricks/environments is publicly available and exposed. This PR unveils But the command fetches its Python version / Holding in draft until the environments repo is live. The underlying command already works via dogfooding (hidden) on main today; this is purely the public-exposure gate. |
rugpanov
marked this pull request as draft
July 16, 2026 12:47
swearyangupta
pushed a commit
to swearyangupta/cli
that referenced
this pull request
Jul 17, 2026
…ks#5832) ## Why - The engine needs a real `PackageManager` implementation (uv) and a CLI entry point so a user can actually run the feature. - Wiring it in while keeping it `Hidden` lets the command be dogfooded and exercised by acceptance tests without becoming user-visible until the stack is complete. - This is the first layer reachable from `main`, so it also makes the whole `libs/localenv` package live for the `deadcode` checker. ## What - **`libs/localenv/uv.go`** — the uv implementation of `PackageManager`: discover/install uv, install the Python minor, `uv sync`, seed pip into the venv, validate; plus the `pip.conf` → `UV_INDEX_URL` bridge for Databricks-managed machines. - **`cmd/localenv/`** — the command tree matching `local-env python sync`: a `local-env` group (`Hidden: true`), a `python` subgroup, and the `sync` verb. Parent nodes use `root.ReportUnknownSubcommand`; `sync` uses `cobra.NoArgs`, resolves flags/bundle target, builds the `Pipeline` with the uv manager, and renders text or `--json`. All Cobra `Use` values + the `--json` command field come from the `libs/localenv` constants. - **`cmd/cmd.go`** — registers the group. ## Testing strategy - Unit tests for uv helper logic (discovery, `pip.conf` index-url, arg builders, stderr surfacing) (`uv_test.go`). - Runtime smoke test of the hidden 3-level command: absent from top-level help; help works at each level; unknown subcommand exits non-zero; bare group shows help; flags + mutual-exclusion + `NoArgs` behave. - Gates: `go build ./...`, `go test`, `golangci-lint`, `deadcode` (whole tree, no pragmas), `gofmt` — all green. - `cmd/localenv/` unit tests are intentionally deferred to the acceptance PR (databricks#5833), per the repo convention that user-visible CLI output is covered by acceptance tests. --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch (this PR targets `main`), so its diff shows only that layer. Layers 1–5 have merged; the original layer-5 PR (databricks#5828) was split into 5a/5b/5c during review. | # | PR | What | Status | |---|----|------|--------| | 1 | databricks#5823 | foundation: result types + env-key mapping | merged | | 2 | databricks#5824 | compute-target resolution | merged | | 3 | databricks#5826 | constraint fetch + offline cache | merged | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | merged | | 5a | databricks#5850 | package-manager interface + detection | merged | | 5b | databricks#5851 | six-phase pipeline orchestrator | merged | | 5c | databricks#5854 | --check cache purity, greenfield name, dbc insertion | merged | | 6 | **databricks#5832 ← you are here** | uv backend + CLI command (registered hidden) | | | 7 | databricks#5833 | acceptance tests | | | 8 | databricks#5835 | unveil (unhide + help + changelog) | | This pull request and its description were written by Isaac.
swearyangupta
pushed a commit
to swearyangupta/cli
that referenced
this pull request
Jul 17, 2026
…ts (databricks#5936) Follow-up refinements from the [databricks#5832](databricks#5832) review (all were left as non-blocking there and deferred with a "will address in a follow-up" reply). The `local-env` command remains hidden until databricks#5835, so there is no user-visible changelog entry. ## Changes - **uv install consent** (`libs/localenv/uv.go`) — `EnsureAvailable` no longer runs the remote uv installer (`curl … | sh` / `irm … | iex`) silently. It now requires consent: a truthy `DATABRICKS_LOCALENV_AUTO_INSTALL_UV` opt-in for non-interactive runs (CI/IDE), or an interactive `y/N` prompt via `cmdio.AskYesOrNo`. A non-interactive run without the opt-in returns an actionable error instead of downloading and executing an installer. (The `--debug` log of the exact installer command from databricks#5832 is kept.) - **serverless job version** (`cmd/localenv/compute.go`) — `GetJobSparkVersion` now reads `Environments[0].Spec.EnvironmentVersion`, so a serverless `--job` resolves to its actual `serverless-vN` instead of always defaulting to v4. Empty still falls back to v4. - **double bundle load** (`cmd/localenv/sync.go`) — skip `bundleTarget` when an explicit `--cluster/--serverless/--job` flag is set. `ResolveTarget` only consults the bundle as a fallback, so the second `TryConfigureBundle` load (and its re-printed load-time diagnostics) was wasted for the explicit-flag case. - **robust `Validate` parse** (`libs/localenv/uv.go`) — the `uv run` probe now prints `PYVER:` / `DBCVER:` sentinels and parses by prefix instead of line position, so a stray stdout line from uv doesn't shift the parse. - **cleanup** (`libs/localenv/uv.go`) — fold single-caller `newUvManager` into `NewUvManager`; collapse the triplicated `resolveIndexURL` + conditional `WithEnv` into one `runUv` helper (the conditional stays so an already-set `UV_INDEX_URL` isn't clobbered). ## Tests Adds unit tests for the install consent gate (`TestConfirmUvInstall`) and the sentinel parse (`TestLineWithPrefix`). Full `libs/localenv` suite passes, including under a CI-like environment with `UV_INDEX_URL`/`PIP_INDEX_URL` set. This pull request and its description were written by Isaac.
swearyangupta
pushed a commit
to swearyangupta/cli
that referenced
this pull request
Jul 17, 2026
## Why - The command's user-visible behavior — text and `--json` output, and every error path — needs end-to-end coverage against the real CLI. - `cmd/localenv/` carries no unit tests by design, so acceptance tests are where that surface is verified (repo convention: user-visible CLI output is covered by acceptance tests). ## What - **`acceptance/localenv/`** — 9 scenarios driven through the (hidden) command against the in-process fake server: `help` (three-level tree), `no-target` (`E_NO_TARGET`), `flag-conflict` (Cobra mutual-exclusion), `manager-unsupported` (conda project → clean P1 exit), `env-unsupported` (404 → `E_ENV_UNSUPPORTED` at fetch), `json-error` (`--output json` error object), `serverless-check` (dry-run plan), `serverless-json` (`--json` plan), and `constraints-only`. - Scripts use `local-env python sync` and the `DATABRICKS_LOCALENV_CONSTRAINT_SOURCE` override; goldens show the `local-env python sync` command field and managed marker. No source changes. ## Testing strategy - Goldens generated with `-update` and verified **stable on a clean re-run** (no `-update`); all 9 subtests pass. - `musterr` guards the five expected-failure scenarios; `trace` shows the three output-producing ones. - Full acceptance suite run to confirm no regressions elsewhere (only pre-existing, environment-specific failures unrelated to this change). - Diff confined to `acceptance/localenv/`. - Independently verified by a review subagent (PASS — goldens, scripts, stubs, stale-refs, hygiene) and by codex (no issues). --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Layers 1–5 have merged (the original layer-5 PR databricks#5828 was split into 5a/5b/5c during review). | # | PR | What | Status | |---|----|------|--------| | 1 | databricks#5823 | foundation: result types + env-key mapping | merged | | 2 | databricks#5824 | compute-target resolution | merged | | 3 | databricks#5826 | constraint fetch + offline cache | merged | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | merged | | 5a | databricks#5850 | package-manager interface + detection | merged | | 5b | databricks#5851 | six-phase pipeline orchestrator | merged | | 5c | databricks#5854 | --check cache purity, greenfield name, dbc insertion | merged | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | | 7 | **databricks#5833 ← you are here** | acceptance tests | | | 8 | databricks#5835 | unveil (unhide + help + changelog) | | This pull request and its description were written by Isaac.
rugpanov
changed the title
tanishgupta-db
pushed a commit
to tanishgupta-db/cli
that referenced
this pull request
Jul 21, 2026
…correct package (databricks#5959) ## What Renames the local-environment command from `databricks local-env python sync` to **`databricks environments setup-local`** and moves it into the existing `environments` command group, per the updated `[P0] CLI Changes` spec. The `environments` group intentionally spans both server-side environment-resource APIs and local provisioning, so a local-install verb belongs there rather than in a standalone top-level group. ## Why here (package placement) `cmd/workspace/environments/environments.go` is generated (`DO NOT EDIT`), so the command is attached the same way `cmd/apps` extends the generated `apps` group: - **`cmd/environments/`** (hand-written, moved from `cmd/localenv/`) exposes a `Commands()` function returning the `setup-local` verb. - **`cmd/workspace/environments/overrides.go`** (new, non-generated) has an `init()` that appends to the generated group's `cmdOverrides` hook, attaching the command. - The standalone `cli.AddCommand(localenv.New())` is dropped from `cmd/cmd.go`. ## Changes - **Command tree:** `local-env python sync` → `environments setup-local`. The `python` subgroup is removed — P0 is Python-only with no language selector (a language axis like `setup-local python` would be additive later; nothing is reserved now). - **Constants:** `CommandGroup`/`CommandVerb`/`CommandName` updated in `libs/localenv/result.go`; JSON `command` field is now `"environments setup-local"`. - **Managed markers:** the `pyproject.toml` managed-block markers now derive from `CommandName` (they previously hard-coded the old name and are written into user files), so the command name lives in exactly one place. - **Still hidden:** the command remains `Hidden` until the environment constraints repository is public — unchanged behavior from before the rename. - Regenerated acceptance goldens + help output. ## Out of scope (deliberate) - **Flag renames** (`--cluster` → `--cluster-id`, `--serverless` → `--serverless-version`, `--job` → `--job-id`, `--check` → `--dry-run`, `--constraint-source` → `--constraint-source-url`) — the spec renames these too, but they are a separate follow-up to keep this PR to the command rename + package move. - The `libs/localenv` package name and `acceptance/localenv/` directory name are left as-is (internal, not user-visible; renaming is cosmetic churn). ## Interaction with the stack databricks#5835 (`[VPEX][8/8]`, held in draft until the constraints repo is public) unhides and documents this command under its **old** name. Whichever lands second must reconcile: the changelog fragment and the `Hidden` flag should reflect `environments setup-local`. ## Testing - `go build ./...`, lint (0 issues), `deadcode` clean - `libs/localenv` + `cmd/environments` unit tests pass - `acceptance/localenv` + `acceptance/help` regenerated and green - Verified `databricks environments setup-local` is runnable, appears under the `environments` group when unhidden, and stays out of help while `Hidden` This pull request and its description were written by Isaac.
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
August 4, 2026 08:18
43f5af9 to
85a944e
Compare
rugpanov
changed the title
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
August 4, 2026 08:24
85a944e to
b7115f7
Compare
yolocs
pushed a commit
to yolocs/dbcli
that referenced
this pull request
Aug 4, 2026
databricks#6136) ## Changes The environment constraint artifacts now live in the public `databricks/environments` repo, so the CLI no longer needs a configurable source: - Hardcode the default base URL `https://raw.githubusercontent.com/databricks/environments/main/python` (anchored at the `python/` subtree where the artifacts live). - Remove the hidden `--constraint-source-url` flag and the `DATABRICKS_LOCALENV_CONSTRAINT_REPO` (owner/name) env var. - Replace `RepoConstraintBaseURL` with `ConstraintBaseURL(ctx)`, which returns the hardcoded default and still honors a single full-URL override env var — renamed to `DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE` so it reads as test-only / power-user, not a supported knob. - Update the acceptance `test.toml` files to the renamed override var. The command stays `Hidden: true`; unhiding it (help, changelog, completion) is the separate unveil change (databricks#5835). ## Why The repo publishing its constraint artifacts was the precondition for a non-empty default. Now that it is public, the empty-default + owner/name-repo plumbing (needed only while the artifacts lived in a private/personal repo) is dead weight, and a real default is safe to ship. ## Tests - `go test ./libs/localenv/... ./cmd/environments/...` and the `localenv`/`help` acceptance suites pass (no golden changes — the override value is unchanged and the removed flag was already hidden). - `TestConstraintBaseURL` now covers the hardcoded default + override; verified the published `serverless-v5` artifact resolves (`HTTP 200`) at the hardcoded URL. - `golangci-lint` and `deadcode` clean; full `go build ./...` green. _This PR was written by Claude Code._
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
August 10, 2026 12:53
b7115f7 to
0f7db2c
Compare
rugpanov
marked this pull request as ready for review
August 10, 2026 13:46
The environment constraint artifacts are now published in the public databricks/environments repo and the command's source is hardcoded to it (#6136), so the command is ready for users. Remove Hidden:true so `databricks environments setup-local` appears in `environments --help` and completion, and add a changelog fragment announcing it. Adds an acceptance test (localenv/group-help) that asserts setup-local is listed under the generated environments group — the visibility this change unveils. It greps a single line rather than snapshotting the whole group help, which also lists generated API subcommands that churn on SDK regen. No existing golden changes: the top-level help lists command groups (environments is always shown), and the command's own --help golden was already captured while it was hidden. Co-authored-by: Isaac
rugpanov
force-pushed
the
dbconnect/08-unveil
branch
from
August 10, 2026 15:54
0f7db2c to
b6a88d8
Compare
anton-107
approved these changes
Aug 11, 2026
anton-107
left a comment
anton-107
left a comment
Contributor
There was a problem hiding this comment.
Reviewed at b6a88d8. Approving the change itself — it's clean and the new test is a real guard. One sequencing caveat below that I'd weigh before actually hitting merge; it's about something outside this PR, not a defect in it.
The acceptance test is genuinely load-bearing. I didn't want to assume trace ... | grep fails correctly, so I mutation-tested it: re-added Hidden: true, re-ran, and got the right failure —
Error: Not equal:
expected: "\n>>> [CLI] environments --help\n setup-local ... "
actual : "\n>>> [CLI] environments --help\n\nExit code: 1\n"
then restored and confirmed green. The framework runs scripts under bash -euo pipefail, so grep's exit 1 propagates and the golden picks up an Exit code: 1 line. Good call grepping one line instead of snapshotting group help that churns on SDK regen. Full TestAccept/(localenv|help) = 54 passed. out.test.toml matches every sibling directory under acceptance/localenv/, so it's conventional rather than stray.
Readiness checks all came back clean: no TODOs or experimental markers in the command path, no dogfooding references leaking into user-facing text, no debug-only flags, help text reads appropriately for a first-time user, and no stale Hidden-era comment left behind.
The caveat: one thing to fix upstream first
Unveiling promotes a reporting bug from dogfood-only to user-visible. libs/localenv/pipeline.go:551 versionFromPin derives the reported dbconnectVersion by taking everything from the first digit of the published pin string, so a pin of ~=17.3.0 reports 17.3.0 (correct today) but a bare-major pin like ~=17.0 would report "17.0" — not a version anything installs. In --dry-run nothing corrects it, because validate is stubbed at pipeline.go:203-211; on a real run validate overwrites it with the truth at pipeline.go:462-467. So dry-run and real runs would disagree, and the --output json contract the VS Code extension consumes carries the wrong value.
That's latent right now. It becomes live if the pending databricks/environments PR 15 (bare-major serverless pins) lands. I've asked for changes there and suggested hardening versionFromPin regardless — a pin string is not a version, and today nothing on either side of that cross-repo contract tests the coupling. My suggestion is to land that fix before flipping this on, so the first users to see this command don't see a wrong version.
Minor, non-blocking: merging this ahead of the setup-local telemetry PR means early adopters generate no adoption data — the exact metric that PR exists to capture. If the ordering is easy to swap, telemetry first is slightly better.
Reviewed with Claude Code; all claims above were verified against the branch.
Contributor
Author
|
@anton-107 thank you for the review. Very good finding with dry-run functionality. Reported an internal ticket for it and will work on it as a fast follow-up. |
Collaborator
Integration test reportCommit: 74a3042
25 interesting tests: 12 flaky, 6 FAIL, 4 RECOVERED, 2 SKIP, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
yolocs
pushed a commit
to yolocs/dbcli
that referenced
this pull request
Aug 11, 2026
… setup-local --dry-run (databricks#6218) ## Summary Found in code review of the unveil PR (databricks#5835). P1, not P0: it only affects `--dry-run`; real runs are correct. `versionFromPin` in `libs/localenv/pipeline.go` derives the reported `dbconnectVersion` by taking everything from the first digit of the published pin string. A pin string is not a version: - A point pin like `~=17.3.0` happens to return `17.3.0` (correct). - A major-only pin like `~=17.0` returns `"17.0"` — a `major.minor` floor that nothing installs. Serverless has pinned by major (`~=17.0`, not `~=17.3.0`) since databricks/environments#15 (merged 2026-08-11), so this is live now for serverless targets. On a real run `validate` overwrites `dbconnectVersion` with the actually-installed version, so the reported value is correct. Under `--dry-run`, `validate` is stubbed, so the fabricated value is what gets reported — dry-run and real runs disagree, and the `--output json` contract the VS Code extension consumes carries the wrong value in dry-run. No wrong install happens; the provisioned venv is still correct. ## Fix Gate the reported version behind a new `dbcVersionFromPin`, which emits a version only when the pin carries a full `major.minor.patch` (e.g. `~=17.3.0` → `17.3.0`) and returns `""` (omitted from JSON) for a range-only pin such as `~=17.0`. The key constraint: `versionFromPin` has two callers. Hardening it directly would break `dbcMajorFromPin`'s major extraction for `~=17.0` and regress *real* runs (validate would fail to determine the major) — worse than the dry-run bug. So the gate is applied only to the reporting path; `versionFromPin` is left untouched. ## Tests The ticket noted nothing tested this cross-repo coupling. Added: - `TestPipelineDryRunOmitsFabricatedDBConnectVersion` — end-to-end dry-run with a `~=17.0` pin (written red-first; reported `17.0` before the fix). - `TestDBCVersionFromPin` — table-driven, including the `~=17.0` bare-major case. - `TestDBCMajorFromPinHandlesMajorOnlyPin` — guards that the real-run major path still works for `~=17.0`. `go test ./libs/localenv` (170 pass) and related acceptance tests (9 pass) are green; the existing JSON goldens use a full `17.2.0` pin, so their output is unchanged. `go vet`, `golangci-lint`, and `gofmt` are clean. No changelog fragment: `setup-local` is still hidden pending the unveil (databricks#5835), matching the prior decision to drop a premature fragment for it. This pull request and its description were written by Isaac.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Unveils the
databricks environments setup-localcommand: removesHidden: truefromcmd/environments/sync.goso it appears inenvironments --helpand shell completion, and adds a changelog fragment announcing it.Why
The command was kept hidden while the feature landed across the VPEX stack and while its constraint-artifact repo was private.
databricks/environmentsis now public and the command's source is hardcoded to it (#6136), so the command works end-to-end for users and is ready to expose.Tests
go build+golangci-lintclean oncmd/environments../task check-changelogpasses.localenv+helpacceptance suites pass with no golden changes (-updateproduces no diff): the top-level help lists command groups —environmentsis a generated API group and is always shown — and the command's own--helpgolden (acceptance/localenv/help) was already captured while hidden.databricks environments --helpnow listssetup-local, and end-to-end provisioning against the public repo works (serverless dry-run + real provision, merge into an existing /bundle initproject).This PR was written by Claude Code.