Slim the nightly: 1.5 GB downloads should be ~400 MB plus opt-in content

[mhglover]

Slim the nightly: 1.5 GB downloads should be ~400 MB plus opt-in content

The problem

The nightly downloads are huge. Every platform payload is ~1.5 GB, nightly
testers auto-download all of it every night (the nightly channel ships full
nupkgs, no deltas), and every install carries ~1.4 GB of media and binaries
that many users will never use. That's not respectful of users' bandwidth or
disk space — and it's about to break: GitHub caps release assets at 2 GiB,
the win full.nupkg is already at 1.57 GB, and career venue packs are still
being added.

How it could be instead

We ship a slim nightly application file (~350–400 MB — Electron, Python
runtime, audio addon, ffmpeg, soundfont, starter songs). During initial
launch, the app asks two questions:

• "Do you want a simulated rig?" — yes starts downloading the amp/effect
  tones (per-platform VST pack) in the background. Users who fork audio
  through their own physical rig — the game silently monitoring their
  instrument — skip this and never carry a byte of it.
• "Do you want to play career mode?" — yes starts downloading the
  starter venue (119 MB); higher-tier venues can download all at once or as
  the user starts to approach needing them, depending on how conservative we
  want to be. Users who don't play career carry zero venue bytes.

Either download can also be started later from Settings. Nightly updates
shrink to the slim app; the big content updates only when it actually
changes.

Where the 1.5 GB comes from

The payload grew 6× in two weeks, in two discrete jumps (win-x64 artifact):

Night	Size	Event
07-03	0.79 GB	baseline
07-07	3.08 GB	rig_builder joined the bundle: 644 MB of VST3s, each a fat mac+win+linux bundle — all three OS slices ship to every platform
07-14	4.83 GB	club + arena venue packs committed into core (feedBack#961/#963): 678 MB of crowd MP4s, bundled on every platform

Composition per platform: 797 MB career venue packs (bar 119 / club 343 /
arena 335) + ~650 MB rig_builder VST3s + ~350–400 MB actual app. Each
artifact then carries the payload 2–3× (win: zip + full.nupkg + msi), so one
night's artifacts total ~12.5 GB.

The Docker nightly is unaffected — core's .dockerignore never ships
plugins/career/. (Side observation: the career plugin is therefore absent
from the Docker image entirely — intended?)

Most of the plumbing already exists

• Venue packs are deferrable today. The career plugin already has
  POST /packs/{venue_id}/download with background progress, a pack URL
  field in venues.json (currently null), downloaded-override →
  bundled-fallback resolution, and graceful no-pack degradation (crowd
  manifest cleared, per-venue "Download venue pack" buttons). Needed: host
  the packs (release assets), set the URLs, stop bundling.
• VST download needs a small amount of new code. VSTs load strictly from
  <plugin_dir>/vst today, but rig_builder already runtime-downloads NAM
  captures (tone3000, disk budget), so the download pattern exists
  in-plugin. Publish per-platform-sliced VST packs as release assets;
  download into a config-dir location and add it to the VST search path.
  Interim zero-code option: platform-slice the fat .vst3 bundles at build
  time (−430 MB/platform while still bundling).

Also worth doing

• Artifact retention 7d → 2d in nightly.yml (−~60 GB standing storage —
  artifacts only ferry files to the publish job; testers use the release).
• Stop committing venue packs to core git — the repo is ~1 GB
  server-side, every re-render re-adds its full weight to history forever,
  and the largest arena file is 89 MB against GitHub's 100 MB hard limit.
• Optionally re-encode the crowd MP4s — 55–93 MB per loop is generous; a
  lower bitrate could roughly halve pack sizes.

[mhglover]

Progress update — opt-in content packs underway

Landing this incrementally. Approach we've aligned on and started building:

Keep a bundled starter, make the rest opt-in. The Dive Bar (bar, ~119 MB) stays in the build so career works offline out of the box; the higher venues download on demand when you unlock them. This leans entirely on career's existing pack pipeline (POST /packs/{id}/download → sha256-verify → extract → atomic swap), which already degrades gracefully when a pack isn't present — so it's wiring, not new machinery.

Hosting = per-pack, versioned, immutable releases. Matching the convention already in use (venue-arena-v1 / arena-pack-v1.zip): tag venue-<id>-v<N>, asset <id>-pack-v<N>.zip. Immutable tags mean no clobbering and a clean cache story.

Reproducible pack builds → no manual sha juggling. The pack builder produces a byte-identical zip for identical media (fixed order/mtime/perms, stored). That means a pack's sha256/bytes can be committed to venues.json before anyone uploads the asset — the CI workflow or a maintainer just publishes, and the bytes match what's already committed. No round-trip of hash values.

Publishing is automated. A content-packs GitHub Action (workflow_dispatch with venues + version inputs) plus a small tools/content_packs.py build/publish/manifest script. Cutting a new venue pack is "run the workflow," not a person's checklist.

Existing progress is preserved. Player career state (stars, unlocks) lives in the config dir, untouched by the bundle change, and unlocked is computed from stars, not pack presence. A player who already reached the Arena upgrades into a build where that venue shows unlocked with a re-download button — never locked or broken.

Where the venues stand:

Venue	Unlock	Ships how	Release	Status
The Dive Bar	0★	bundled starter	—	✅ in build
Velvet Room	50★	opt-in download	venue-club-v1	manifest seeded; awaiting publish
Feedback Arena	150★	opt-in download	venue-arena-v1 ✅	published + validated end-to-end

Arena's live asset was pulled, run through career's actual pack validator, and its real hash committed — so that path is proven against a real download, not just a fixture.

Size impact: moving club + arena out of the bundle is ~678 MB off every platform's payload; the bundled starter keeps first-run playable offline. First install lands near the ~400 MB target this issue set.

Next: first-launch "play career? / build a rig?" prompts (so decliners carry zero opt-in bytes), then the same treatment for rig_builder's VST packs (per-platform sliced), which is the other big chunk. Further nightly-size work beyond content packs is tracked separately.

[mhglover]

I've got a small request to help me with for somebody with appropriate org permissions:

@byrongamatos published a release for venue-arena-v1 on the 14th. I need you to publish the venue-club-v1 release so Career can pull it on demand once we stop bundling the higher venues. I'm not an org member so I can't cut it myself. I think this is how it's done from a got-feedBack/feedBack checkout:

# 1. flat-zip the club media (matches the arena pack's layout)
cd plugins/career/venue-packs/club
zip -0 -X -j /tmp/club-pack-v1.zip *

# 2. create the release
gh release create venue-club-v1 --repo got-feedBack/feedBack \
  --title "Club venue pack v1" \
  --notes "Opt-in career venue pack. Not a code release." \
  --latest=false

# 3. upload the release
gh release upload venue-club-v1 /tmp/club-pack-v1.zip --repo got-feedBack/feedBack

This manual path is just to seed the one release. The first PR will include automation to keep these packs updated going forward and will include a helper script for publishers.

[mhglover]

Status update — all three phases now up as draft PRs

The slimming work is staged across the repos as draft PRs, cross-linked and stacked in merge order. Nothing's published yet (details at the bottom), but I have smoke-tested the download path against a real release asset and it held up.

PR 1 — venue packs off the bundle

• feat(career): host higher venues as opt-in content packs feedBack#1023 · build: stop bundling the higher career venue packs #123
• Club + arena (~678 MB) become on-demand downloads; the Dive Bar starter stays bundled so career works offline. Reuses career's existing pack pipeline. Reproducible pack builds + a content-packs workflow so publishing is automated, not a manual chore.

PR 2 — career prompts + achievements

• feat(career): first-launch milestone + download-size disclosure feedBack#1024
• Download-size disclosure on the venue buttons ("Download venue pack (~335 MB)"), read live from the manifest. Career becomes the first source plugin to contribute achievements (career_started, first_venue_gig). Verified that existing progress survives the upgrade — a previously-unlocked venue shows unlocked + downloadable, never broken.

PR 3 — rig_builder VSTs (~650 MB)

• feat(tools): per-platform VST3 slicing for rig content packs feedBack#1025 · feat: opt-in per-platform VST pack (download + slicing + first-run prompt) feedBack-plugin-rig-builder#55 · build: stop bundling rig_builder VST3 bundles (~650 MB) #124
• The fat .vst3 bundles are sliced per-platform (~200 MB each) and download on demand into a writable config-dir root the loader now searches (the bundled plugin dir is read-only in packaged builds). A first-run "Use a simulated rig?" prompt offers the download with its size; rig achievements reward the musician-learning arc (rig_simulated, rig_from_tone, rig_gearhead).

Testing so far

• Smoke-tested the career path against the real 351 MB arena-pack-v1.zip: downloaded from the URL in venues.json, through career's actual download code — stream → sha256 → extract → validate → install. Worked, but that's one asset on one machine, not a broad matrix.
• Added unit/e2e/JS tests across all three PRs; the rig download path had zero coverage before and now has an end-to-end test. Not yet exercised: a full desktop build, and the live publish workflow (needs org write).

Size impact: first install drops from ~1.5 GB toward the ~400 MB target — venue media (~678 MB) and VSTs (~650 MB) both move to opt-in, with a bundled starter keeping first-run playable offline.

Still needed (org write required): publish the pack releases, then the manifest hashes get filled from the live assets (arena's already done this way). That's venue-club-v1 and the three vst-<platform>-v1 packs. Until then those manifest entries are placeholders and their downloads no-op.

Heads-up for the feedBack-plugin-rig-builder maintainer: 3 pre-existing test failures (not from this PR)

While adding the VST-download tests I noticed three tests already failing on main in feedBack-plugin-rig-builder, independent of my changes (they fail with my diff stashed):

• tests/test_watcher_paths.py::test_persist_preset_chain_writes_relative_song_key_for_nested_basename
• tests/test_watcher_paths.py::test_materialization_watcher_preserves_nested_sloppak_relative_paths
• tests/test_manifest.py::test_screen_blocks_amp_button_while_mega_chain_active

The two watcher ones look like a relative-path-preservation regression — the nested song key loses its artist/ prefix ('song_p.psarc' where 'artist/song_p.psarc' is expected). Flagging so it's on the radar; my PR (#55) doesn't touch that code and leaves the count unchanged (25 passed / 3 failed before and after).

[mhglover]

Correction + a fix from a real build test

I ran the bundle against the actual branches (good news: it produces a ~200 MB tree — venue-packs down to the bundled bar starter, rig vst/ gone, no errors), and a runtime smoke of the endpoints caught something I got wrong above.

I said the unpublished pack entries would "no-op." They didn't — the committed manifests carry a placeholder (real-looking release URL, bytes: 0, all-zero sha) until a release exists, and the old code treated "has a pack object" as "downloadable." So the UI would have offered a download that can only fail (the placeholder URL 404s) — a broken button, not a no-op.

Fixed in both plugins: a pack is only offered once a publish has stamped its real size (bytes > 0). Until then the venue card shows "coming soon" and the download endpoint returns 404 — no dead button. Landed on the career PR and the rig plugin PR, with a regression test.

So the accurate state: arena (published) is downloadable today; club and the three vst-<platform>-v1 packs show "coming soon" until their releases are cut, then flip to downloadable automatically once the manifest hashes are filled from the live assets.