ci(demo): registry mode=max build cache for ARC (fix ~3h cold builds) by evereq · Pull Request #9786 · ever-co/ever-gauzy

evereq · 2026-07-06T13:38:00Z

Why

The docker-build-publish-demo.yml jobs (gauzy-api, gauzy-webapp, gauzy-worker) on the self-hosted ARC runners were running ~3h and getting cancelled without finishing — vs ~30 min previously. Root cause, confirmed from the build log (run 28782300929):

The build imported the cache manifest but still ran yarn install fully cold — one stage logged #100 4296.6 (71 min) and the worker #100 6015.6 (100 min) in a single install, plus native recompiles (bcrypt, better-sqlite3, @sentry/profiling-node).
cache-to: type=inline only stores the final image stage, so the expensive multi-stage dependencies / proddependencies (yarn install) layers were never cached.
driver: docker (on the api + worker jobs) uses the classic builder, which can't import registry cache effectively.

What

For each of the 3 jobs:

Drop driver: docker → use the default docker-container buildx driver (required for mode=max).
cache-to: type=registry,ref=…:buildcache,mode=max — caches all stages (incl. the yarn-install layers) to a dedicated :buildcache tag.
cache-from now reads :buildcache (with :latest as fallback).
Move a GHCR login before the Build step so cache-to can push the cache.

Because the cache lives in GHCR, it's restored by every fresh ephemeral runner — this is the cross-runner cache persistence. First run after merge is still cold (it populates :buildcache); subsequent runs should drop back to minutes.

Notes

No change to the pushed image tags or target registries.
Set up QEMU is left as-is; it's unused (platforms: linux/amd64) and could be removed in a follow-up to shave ~2 min.
Infra side (separate, already applied): the ARC -8 runner pool now guarantees the build 8 vCPU / 16 GiB (the docker build runs in the dind sidecar, which previously had no resource limits).

🤖 Generated with Claude Code

Summary by cubic

Fix ARC demo Docker builds by enabling registry-backed Buildx cache (mode=max) and dropping the classic Docker driver. This stops cold yarn install stages and returns builds to minutes.

Bug Fixes
- Apply to gauzy-api, gauzy-webapp, and gauzy-worker demo jobs on ARC.
- Switch from driver: docker to Buildx docker-container.
- Use GHCR :buildcache with cache-to: type=registry,mode=max and cache-from (fallback to :latest).
- Log in to GHCR before build so cache push works.
- No change to image tags/registries; first run warms the cache.

^{Written for commit 869ac19. Summary will update on new commits.}

…ding cold The self-hosted (ARC) demo image builds were taking ~3h (never finishing) vs ~30m before, because every build reinstalled+recompiled all deps from scratch: - cache-to: type=inline only stores the FINAL image stage, so the expensive multi-stage yarn-install / native-module layers were never cached; - driver: docker (classic builder) can't import registry cache at all. Switch each job to the default docker-container buildx driver and push a mode=max layer cache to a dedicated :buildcache tag on GHCR. mode=max caches every stage (incl. the 30-100min yarn-install layers), and because the cache lives in the registry it is restored by every fresh ephemeral runner. A GHCR login is moved before the Build step so cache-to can push. First run after merge is still cold (populates :buildcache); subsequent runs reuse it and should drop back to minutes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

sonarqubecloud · 2026-07-06T13:39:59Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

coderabbitai · 2026-07-06T13:40:51Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: b335a87a-81ad-41a8-921d-154af6156939

📥 Commits

Reviewing files that changed from the base of the PR and between d4ed5dc and 869ac19.

📒 Files selected for processing (1)

.github/workflows/docker-build-publish-demo.yml

Summary by CodeRabbit

Chores
- Improved the Docker image build process for the API, web app, and worker services.
- Updated build caching to reuse previous images more effectively, which should help speed up future builds and reduce rebuild times.
- Kept the existing image publishing flow unchanged.

Walkthrough

This PR updates the demo Docker build-publish GitHub Actions workflow to use registry-based build caching. It adds a GHCR login step before each of the gauzy-api, gauzy-webapp, and gauzy-worker builds, and replaces inline caching with cache-from/cache-to entries referencing :buildcache and :latest image tags with mode=max.

Changes

Docker Build Cache Migration

Layer / File(s)	Summary
gauzy-api caching update `.github/workflows/docker-build-publish-demo.yml`	Adds a GHCR login step and switches cache-from/cache-to to registry-backed `:buildcache`/`:latest` tags with `mode=max`.
gauzy-webapp caching update `.github/workflows/docker-build-publish-demo.yml`	Adds a GHCR login step and switches cache-from/cache-to to registry-backed `:buildcache`/`:latest` tags with `mode=max`.
gauzy-worker caching update `.github/workflows/docker-build-publish-demo.yml`	Adds a GHCR login step and switches cache-from/cache-to to registry-backed `:buildcache`/`:latest` tags with `mode=max`.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

Three jobs, three logins, one shiny cache,
Buildx now hops at a swifter pace,
No more inline crumbs to store,
Just :buildcache tags, and nothing more,
A rabbit thumps in CI delight! 🐇⚡

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch ci/gauzy-demo-buildcache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

chatgpt-codex-connector · 2026-07-06T15:22:55Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

greptile-apps · 2026-07-06T15:25:38Z

Greptile Summary

This PR fixes ~3-hour cold builds on the ARC self-hosted runners by switching all three demo Docker jobs (gauzy-api, gauzy-webapp, gauzy-worker) from cache-to: type=inline to cache-to: type=registry,mode=max, and dropping the driver: docker override that blocked registry cache import. A pre-build GHCR login is added so cache-to can push the :buildcache manifest during the build step.

Replaces inline (final-stage only) cache with mode=max registry cache that captures all intermediate stages including the expensive multi-minute yarn install layers, restoring build times from ~3h back to minutes on ephemeral ARC runners.
Drops driver: docker on gauzy-api and gauzy-worker, switching both to the default docker-container buildx driver required for registry-backed mode=max cache.
Adds a GHCR login-action step before each build for cache-push auth, which duplicates the pre-existing GHCR login step later in each job (the credential store persists for the whole job, so one login is enough).

Confidence Score: 4/5

Safe to merge — the changes are scoped to the demo CI workflow and the core logic is correct; the only cleanup left is removing the now-redundant second GHCR login in each job.

The switch to docker-container driver with mode=max registry cache is technically correct and well-understood. The pre-build GHCR login is necessary for cache-to to push the cache manifest. The only notable issue is that each job now logs into GHCR twice with identical credentials — the new pre-build step and the pre-existing post-build step — which wastes a few seconds per run but causes no failures.

.github/workflows/docker-build-publish-demo.yml — verify the duplicate GHCR login steps are acceptable or consolidate them.

Important Files Changed

Filename	Overview
.github/workflows/docker-build-publish-demo.yml	Switches all three demo build jobs from inline cache to registry-backed mode=max cache in GHCR; drops driver:docker from api and worker jobs; adds a GHCR pre-build login for cache push auth — this duplicates the existing post-build GHCR login step in each job.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Runner as ARC Ephemeral Runner
    participant Buildx as Buildx (docker-container)
    participant GHCR as GHCR (:buildcache)
    participant DockerDaemon as Local Docker Daemon
    participant Registries as DockerHub / DO / GHCR:latest

    Runner->>Buildx: setup-buildx-action (docker-container driver)
    Runner->>GHCR: login-action (GITHUB_TOKEN) — auth for cache-to push
    Runner->>Buildx: "build-push-action (load=true)"
    Buildx->>GHCR: "cache-from type=registry :buildcache (restore)"
    Buildx->>GHCR: "cache-from type=registry :latest (fallback)"
    Note over Buildx: Builds all stages (yarn install layers cached)
    Buildx->>GHCR: "cache-to type=registry :buildcache mode=max (all stages)"
    Buildx->>DockerDaemon: "load=true (export final image)"
    Runner->>Registries: docker push (DockerHub / DO / GHCR:latest)

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Runner as ARC Ephemeral Runner
    participant Buildx as Buildx (docker-container)
    participant GHCR as GHCR (:buildcache)
    participant DockerDaemon as Local Docker Daemon
    participant Registries as DockerHub / DO / GHCR:latest

    Runner->>Buildx: setup-buildx-action (docker-container driver)
    Runner->>GHCR: login-action (GITHUB_TOKEN) — auth for cache-to push
    Runner->>Buildx: "build-push-action (load=true)"
    Buildx->>GHCR: "cache-from type=registry :buildcache (restore)"
    Buildx->>GHCR: "cache-from type=registry :latest (fallback)"
    Note over Buildx: Builds all stages (yarn install layers cached)
    Buildx->>GHCR: "cache-to type=registry :buildcache mode=max (all stages)"
    Buildx->>DockerDaemon: "load=true (export final image)"
    Runner->>Registries: docker push (DockerHub / DO / GHCR:latest)

_{Reviews (1): Last reviewed commit: "ci(demo): use registry mode=max build ca..." | Re-trigger Greptile}

greptile-apps · 2026-07-06T15:25:43Z

+      - name: Login to GitHub Container Registry (build cache)
+        uses: docker/login-action@v4
        with:
-          driver: docker
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}


Redundant GHCR login steps per job

Each of the three jobs now contains two identical docker/login-action@v4 calls for GHCR — the new one added before the build (lines 29–34 in gauzy-api, similar locations in the other two jobs) and the pre-existing one used before the final push (e.g., lines 92–97). The login-action writes credentials to Docker's credential store, which persists for the entire job, so the second call is a no-op. Since both use the same GITHUB_TOKEN, you can safely remove the later "Login to GitHub Container Registry" step in each job (or alternatively, drop the newly added one and move the existing one above the build step).

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

augmentcode · 2026-07-06T15:26:34Z

🤖 Augment PR Summary

Summary: Improves ARC demo Docker build times by switching to registry-backed Buildx caching so expensive multi-stage dependency installs are reused across ephemeral runners.

Changes:

Use the default Buildx docker-container driver (drop classic driver: docker).
Log into GHCR before the build so BuildKit can push cache layers.
Export/import cache to a dedicated :buildcache tag using mode=max (all stages cached).
Keep existing image tags/registries intact for gauzy-api, gauzy-webapp, and gauzy-worker.

Technical Notes: mode=max captures intermediate layers (e.g., yarn install) to GHCR so subsequent runs return to minutes instead of hours.

_{🤖 Was this summary useful? React with 👍 or 👎}

augmentcode

Review completed. 1 suggestion posted.

Comment augment review to trigger a new review at any time.

augmentcode · 2026-07-06T15:26:35Z

+          # supports inline (final-stage) cache, which is why the yarn-install stages were cold.
+          cache-from: |
+            type=registry,ref=ghcr.io/ever-co/gauzy-api-demo:buildcache
+            type=registry,ref=ghcr.io/ever-co/gauzy-api-demo:latest


In .github/workflows/docker-build-publish-demo.yml, cache-from: type=registry,ref=...:latest may stop being a useful fallback now that cache-to no longer exports type=inline, since the :latest image may not carry BuildKit cache metadata. If the :buildcache tag is missing/expired, this could lead to unexpectedly cold builds despite the "fallback" entry.

Severity: low

Other Locations

.github/workflows/docker-build-publish-demo.yml:154

.github/workflows/docker-build-publish-demo.yml:254

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

evereq marked this pull request as ready for review July 6, 2026 15:22

evereq merged commit 0395284 into develop Jul 6, 2026
14 of 18 checks passed

evereq deleted the ci/gauzy-demo-buildcache branch July 6, 2026 15:23

greptile-apps Bot reviewed Jul 6, 2026

View reviewed changes

augmentcode Bot reviewed Jul 6, 2026

View reviewed changes

evereq mentioned this pull request Jul 6, 2026

ci(demo): drop unused Set up QEMU steps #9787

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ci(demo): registry mode=max build cache for ARC (fix ~3h cold builds)#9786

ci(demo): registry mode=max build cache for ARC (fix ~3h cold builds)#9786
evereq merged 1 commit into
developfrom
ci/gauzy-demo-buildcache

evereq commented Jul 6, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

sonarqubecloud Bot commented Jul 6, 2026

Uh oh!

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading

Review failed

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Jul 6, 2026

Uh oh!

greptile-apps Bot commented Jul 6, 2026

Uh oh!

greptile-apps Bot Jul 6, 2026

Uh oh!

augmentcode Bot commented Jul 6, 2026

Uh oh!

augmentcode Bot left a comment

Uh oh!

augmentcode Bot Jul 6, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

evereq commented Jul 6, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Notes

Summary by cubic

Uh oh!

sonarqubecloud Bot commented Jul 6, 2026

Quality Gate passed

Uh oh!

coderabbitai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Jul 6, 2026

Uh oh!

greptile-apps Bot commented Jul 6, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot commented Jul 6, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

evereq commented Jul 6, 2026 •

edited by cubic-dev-ai Bot

Loading

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading

augmentcode Bot Jul 6, 2026 •

edited

Loading