Skip to content

fix(sandbox): build real arm64 sandbox images and ship arm64 Cube variants - #3897

Merged
lyingbug merged 1 commit into
Tencent:mainfrom
nullptr-error:fix/sandbox-multiarch
Sep 30, 2026
Merged

lyingbug merged 1 commit into
Tencent:mainfrom
nullptr-error:fix/sandbox-multiarch

Conversation

@nullptr-error

@nullptr-error nullptr-error commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Description

The arm64 entries of wechatopenai/weknora-sandbox:main, latest and main-desktop contain an amd64 rootfs. Registry manifests show the arm64 and amd64 entries share all 16 layer digests; only the config's architecture field differs. Inside the arm64 image, uname -m is x86_64, dpkg --print-architecture is amd64, and python3 is an x86-64 ELF. Real arm64 hosts without x86 binfmt fail with exec format error, and Apple Silicon runs the image entirely under emulation.

Root cause: docker/Dockerfile.sandbox declared ARG TARGETPLATFORM=linux/amd64 before the first FROM. A declared default overrides the value BuildKit injects per platform, so with platforms: linux/amd64,linux/arm64 both platforms pulled amd64 bases and ran every RUN on amd64, and BuildKit still labeled the second config arm64. The stage-level ARG TARGETARCH=amd64 in cube / desktop-cube had the same effect, so their amd64-only guard never fired.

Changes:

  • Declare TARGETPLATFORM without a default; FROM --platform=${TARGETPLATFORM:-linux/amd64} falls back to amd64 only when it is empty (legacy builder, which also rejects an empty --platform).
  • The runtime stage now fails the build if dpkg --print-architecture differs from TARGETARCH, so a relabeled image can no longer be published silently.
  • ghcr.io/tencentcloud/cubesandbox-base:2026.16 publishes a real linux/arm64 image (distinct layers, aarch64 envd), and Cube supports arm64 bare-metal KVM since v0.5.0; PVM remains x86_64-only. cube-base now follows TARGETPLATFORM, the amd64-only guard is replaced by an envd-vs-rootfs architecture check, and cube / desktop-cube are built for linux/amd64,linux/arm64 in CI and in scripts/build_images.sh.

Type of Change

  • 🐛 Bug fix
  • 🔧 Configuration / Build / CI

Related Issue

N/A

Testing

Run on Apple Silicon (colima, BuildKit v0.32.2):

  • Minimal repro: with ARG TARGETPLATFORM=linux/amd64, docker buildx build --platform linux/arm64 produces an x86_64 / amd64 rootfs; without the default it produces aarch64 / arm64.
  • sandbox, desktop, cube, desktop-cube each built for linux/amd64,linux/arm64 with a docker-container builder (same driver as setup-buildx-action).
  • Pushed sandbox, desktop and cube (multi-platform) to a local registry, then ran every platform entry by digest: layer sets differ between platforms, and amd64 → x86_64 / amd64 / x86-64 python3 ELF, arm64 → aarch64 / arm64 / aarch64 python3 ELF.
  • cube / desktop-cube per platform: amd64 has envd x86-64, arm64 has envd ARM aarch64; cube-entrypoint.sh present and executable.
  • Regression guards: re-adding the old ARG default makes an arm64 runtime build fail with runtime rootfs is amd64 but TARGETARCH=arm64; pinning cube-base back to amd64 makes an arm64 cube build fail with envd is x86-64 but rootfs is arm64.
  • Legacy builder (DOCKER_BUILDKIT=0) still builds with the amd64 fallback, and honours --build-arg TARGETPLATFORM.
  • docker buildx build --check: only the 4 intentional RedundantTargetPlatform warnings remain.

Not tested: building a Cube template from the arm64 image on an arm64 Cube cluster (none available).

Note: the cube / desktop-cube arm64 builds run under QEMU on the amd64 ubuntu-latest runner, so build-sandbox will take noticeably longer.

Checklist

  • git diff --check origin/main...HEAD passes
  • Changed source files are formatted
  • Targeted tests for the changed packages/components pass (no code paths changed; the Go change is a comment)
  • Diff-scoped lint passes where applicable
  • Full-repository checks were run, or any unrelated/environment-dependent failures are documented above
  • Self-reviewed the code
  • Added/updated tests covering the change
  • Updated related documentation (README, website-docs/, Swagger annotations, etc.)
  • Breaking changes are clearly called out in the description above

Copilot AI balanced review requested due to automatic review settings September 30, 2026 02:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…iants

The global `ARG TARGETPLATFORM=linux/amd64` in Dockerfile.sandbox overrode
the per-platform value BuildKit injects, so the multi-platform CI build ran
every platform on the amd64 rootfs and only relabeled the arm64 config.
The published arm64 entries of main, latest and main-desktop are byte-for-byte
the amd64 layers. The stage-level `ARG TARGETARCH=amd64` had the same effect,
so the cube "amd64 only" guard could never fire.

- Declare TARGETPLATFORM without a default and fall back to linux/amd64 in
  FROM only when it is empty (legacy builder).
- Fail the runtime stage when the rootfs architecture differs from
  TARGETARCH.
- cubesandbox-base:2026.16 publishes linux/arm64 as well, and Cube runs on
  arm64 bare-metal KVM (PVM stays x86_64-only). Follow TARGETPLATFORM for
  cube-base, replace the amd64-only guard with an envd-vs-rootfs check, and
  build cube / desktop-cube for linux/amd64,linux/arm64 in CI and
  build_images.sh.
@lyingbug
lyingbug merged commit 176aa0d into Tencent:main Sep 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants