Pipeline: upload an environment-only page and desktop descriptor per version, then flag it as latest (OPE-433)

.gitignore

@@ -26,3 +26,8 @@ tests/perf/output/
 # Locally built Go binary (gen-maps uses `go run .`)
 map-generator/map-generator
 map-generator/map-generator.exe
+
+# Staging dir tests/RenderDesktopDescriptor.test.ts creates in the repo root
+# (it needs the CLI copy at the same relative depth as the real one). Cleaned
+# up after each case; ignored so a crashed run cannot dirty the tree.
+.tmp-desktop-cli-*

docs/MultiServer.md

@@ -330,3 +330,68 @@ wait for games to end), then delete its cluster entries and its
   append-only forever, numWorkers immutable while a letter has live games,
   and membership ≠ liveness (a flapping health check must never shrink the
   map — removal stays drain-then-delete).
+
+## Publish pipeline (v2)
+
+Written while PR #5365 ("Server list v2") was still open; fold this into
+roadmap item 4 of that section once it merges.
+
+Every deploy already uploads the build's hashed assets to R2 and a
+fully-rendered `index-<short>.html` replay shell next to them. `update.sh`
+now also publishes, per **site** and per **version**, the three objects the
+static Worker will serve. The site is `SITE_HOST` when the deployment sits
+behind a load balancer, else `<subdomain>.<domain>`; the version is the
+7-character prefix of `static/commit.txt`. All four uploads go through
+`PUT $R2_ENDPOINT/game_assets/upload/<urlencoded key>`, which prefixes
+`game_assets/`:
+
+| Object                                        | Rendered by                                    |
+| --------------------------------------------- | ---------------------------------------------- |
+| `sites/<site>/v/<short>/index.html`           | `RenderStaticIndex.ts --environment-only`      |
+| `sites/<site>/v/<short>/desktop/release.json` | `RenderDesktopDescriptor.ts`                   |
+| `sites/<site>/v/<short>/desktop/version.json` | `RenderDesktopDescriptor.ts --version-pointer` |
+
+Both renderers run inside the freshly built image with the live container's
+env file, exactly as the replay shell already does, so what is published is
+what that build's server would itself have produced.
+
+**The page carries no server.** `renderHtmlContent(path, { perServer: false })`
+omits `cluster`, `instanceLetter`, `instanceId`, `serverHost` and `siteHost`;
+`index.html` guards those lines the way it already guarded `serverHost`, so a
+render that supplies them is byte-for-byte what it always was. That is what
+lets one page be cached and served to every player of a version — the client
+asks the API which server to use. The legacy `index-<short>.html` upload keeps
+the server values until OPE-431 lands, because today's client throws without a
+worker-count source.
+
+**The descriptors move earlier, not elsewhere.** `release.json` and
+`version.json` are the same objects `/desktop/*.json` serves today, from the
+same `buildDescriptor`; publishing them per version lets the Worker answer for
+a site with no game server reachable, and makes a rollback a pointer flip.
+`release.json`'s `template.html` is the raw EJS template by design: the Steam
+shell renders it itself.
+
+**Flagging `latest`.** After the new container is running, `update.sh` calls
+`POST $R2_ENDPOINT/cluster/latest` with `{ site, version }` (version is the
+full sha). The API refuses a version no server has checked in for, so a `409`
+right after `docker run` is expected and is retried every 5s for up to 90s —
+servers register within ~10s of boot. Outcomes:
+
+- `200` / `204` — logged, done.
+- `404` — the API predates the registry; warn and continue.
+- `400` / `401` / `403` — a bad key or a malformed request. No retry can fix
+  it, so it is decided at once on the same terms as the row below: warn and
+  continue, or fail under `CLUSTER_STATE_SOURCE=api`.
+- `409` (or an unreachable API) after the retries — warn and continue, because
+  the page and its servers still come from `BOOTSTRAP_CONFIG` and nothing a
+  player sees has changed. **Unless** `CLUSTER_STATE_SOURCE=api` is in the
+  site's env file, which says its clients take the server list from the API: an
+  unflagged version then means no server is `open` and nobody can start a game,
+  so the deploy fails rather than reporting a success it did not achieve.
+  `deploy.sh` gains the passthrough for that variable separately (roadmap item
+  3); until it does, it is always absent, which is the lenient path above.
+
+Until the Worker exists nothing reads any of this, so the uploads are additive
+and prod is unaffected. The decision table above is unit-tested in
+`tests/UpdateFlagLatest.test.ts`, which extracts the real function out of
+`update.sh` and drives it with a scripted `curl`.

index.html

@@ -113,18 +113,30 @@
     <meta property="og:image" content="<%- gameplayScreenshotUrl %>" />
     <meta property="og:type" content="game" />
 
-    <!-- Injected from Server env -->
+    <!--
+      Injected from Server env.
+
+      Every per-server value (cluster, instanceLetter, instanceId, serverHost,
+      siteHost) is emitted through a guard so the whole line — indentation and
+      trailing comma included — disappears when the renderer does not supply
+      it. That is what lets one page be rendered per VERSION rather than per
+      server and uploaded to the CDN (docs/MultiServer.md, "Publish pipeline
+      (v2)"); a game server supplies them all and gets exactly the page it
+      always served.
+
+      Never add a placeholder the Steam shell does not supply: openfront-desktop
+      renders this same template itself and a missing local is a ReferenceError,
+      i.e. a blank window. Guarding an existing one is safe — the shell supplies
+      a value and the line still emits.
+    -->
     <script>
       window.BOOTSTRAP_CONFIG = {
         gitCommit: <%- gitCommit %>,
         assetManifest: <%- assetManifest %>,
         cdnBase: <%- cdnBase %>,
-        gameEnv: <%- gameEnv %>,
-        cluster: <%- cluster %>,
-        instanceLetter: <%- instanceLetter %>,
+        gameEnv: <%- gameEnv %>,<%- typeof cluster !== "undefined" && cluster ? "\n        cluster: " + cluster + "," : "" %><%- typeof instanceLetter !== "undefined" && instanceLetter ? "\n        instanceLetter: " + instanceLetter + "," : "" %>
         turnstileSiteKey: <%- turnstileSiteKey %>,
-        jwtAudience: <%- jwtAudience %>,
-        instanceId: <%- instanceId %>,<%- typeof serverHost !== "undefined" && serverHost ? "\n        serverHost: " + serverHost + "," : "" %><%- typeof siteHost !== "undefined" && siteHost ? "\n        siteHost: " + siteHost + "," : "" %>
+        jwtAudience: <%- jwtAudience %>,<%- typeof instanceId !== "undefined" && instanceId ? "\n        instanceId: " + instanceId + "," : "" %><%- typeof serverHost !== "undefined" && serverHost ? "\n        serverHost: " + serverHost + "," : "" %><%- typeof siteHost !== "undefined" && siteHost ? "\n        siteHost: " + siteHost + "," : "" %>
       };
       document.documentElement.style.setProperty(
         "--background-image-url",

src/server/RenderDesktopDescriptor.ts

@@ -0,0 +1,84 @@
+// Builds the desktop (Steam) release descriptor at DEPLOY time and writes it to
+// stdout, so update.sh can upload it alongside the version's page as
+// sites/<site>/v/<short>/desktop/release.json (and version.json with
+// --version-pointer).
+//
+// Today the game server builds this per request from its own static/ directory
+// (src/server/Master.ts, /desktop/*.json). Uploading it per version is what
+// lets the static Worker answer /desktop/*.json for a site without any game
+// server being reachable, and what makes a rollback a pointer flip rather than
+// a redeploy. Same buildDescriptor, same inputs, same env vars as the server —
+// this is the identical descriptor, computed one deploy earlier.
+//
+// Run inside the freshly built image with the live container's env file, the
+// same way RenderStaticIndex.ts is:
+//
+//   npx tsx src/server/RenderDesktopDescriptor.ts
+//   npx tsx src/server/RenderDesktopDescriptor.ts --version-pointer
+//
+// clientVersion is GIT_COMMIT (the full sha baked into the image) rather than
+// static/commit.txt, so it is the value the server would report for itself.
+import path from "path";
+import { fileURLToPath } from "url";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+
+const VERSION_POINTER = "--version-pointer";
+
+const args = process.argv.slice(2);
+const unknown = args.filter((a) => a !== VERSION_POINTER);
+if (unknown.length > 0) {
+  console.error(
+    `Unknown argument(s): ${unknown.join(" ")}. Usage: RenderDesktopDescriptor.ts [${VERSION_POINTER}]`,
+  );
+  process.exit(2);
+}
+const pointerOnly = args.includes(VERSION_POINTER);
+
+// stdout is this program's DATA channel: whatever lands there is uploaded
+// verbatim as release.json. Several things on the import path below write to
+// it as if it were a log — dotenv's "injected env" banner and Logger.ts's OTEL
+// line at module evaluation, and winston's Console transport (which defaults to
+// stdout for every level) if buildDescriptor warns about an empty cdnBase. Any
+// one of them prefixes the JSON with prose and publishes a descriptor no Steam
+// client can parse.
+//
+// So: send everything that thinks it is logging to stderr, and keep the real
+// stdout for the payload. This has to happen before DesktopRelease is loaded,
+// which is why that import is dynamic — a static one would be hoisted above
+// these statements, and prettier-plugin-organize-imports would reorder it
+// anyway.
+const writeOut = process.stdout.write.bind(process.stdout);
+process.stdout.write = process.stderr.write.bind(
+  process.stderr,
+) as typeof process.stdout.write;
+
+const [{ GameEnv }, { buildDescriptor }, { ServerEnv }] = await Promise.all([
+  import("../core/configuration/Config"),
+  import("./DesktopRelease"),
+  import("./ServerEnv"),
+]);
+
+try {
+  const descriptor = await buildDescriptor(
+    path.join(__dirname, "../../static"),
+    {
+      clientVersion: ServerEnv.gitCommit(),
+      cdnBase: ServerEnv.cdnBase(),
+      // Same rule as Master.ts's descriptorOpts: production must have a CDN, or
+      // the descriptor would send every Steam client to the app server for
+      // ~570MB of assets. Failing here fails the DEPLOY, which is the point.
+      requireCdnBase: ServerEnv.env() === GameEnv.Prod,
+    },
+  );
+  const out = pointerOnly
+    ? {
+        clientVersion: descriptor.clientVersion,
+        coreVersion: descriptor.coreVersion,
+      }
+    : descriptor;
+  writeOut(JSON.stringify(out));
+} catch (error) {
+  console.error("Failed to build desktop release descriptor:", error);
+  process.exit(1);
+}

src/server/RenderHtml.ts

@@ -11,11 +11,68 @@ const APP_SHELL_CACHE_CONTROL =
 
 const appShellContentCache = new Map<string, Promise<string>>();
 
-export async function renderHtmlContent(htmlPath: string): Promise<string> {
+export interface RenderHtmlOptions {
+  /**
+   * Inject the values that only make sense for ONE running server: `cluster`,
+   * `instanceLetter`, `instanceId`, `serverHost`, `siteHost`.
+   *
+   * True (the default) is what a game server serves and what the legacy
+   * `index-<short>.html` replay shell is rendered with — byte-for-byte what
+   * this function has always produced.
+   *
+   * False produces an environment-only page: everything that depends on the
+   * BUILD and the ENVIRONMENT (gitCommit, assetManifest, cdnBase, gameEnv,
+   * turnstileSiteKey, jwtAudience) and nothing that depends on which server
+   * happens to render it. That page is uploaded once per version to
+   * `sites/<site>/v/<short>/index.html` and served by the static Worker to
+   * every player of that version, which is only sound if it names no server —
+   * the client asks the API for the server list instead (see
+   * docs/MultiServer.md, "Server list v2").
+   *
+   * Rendering with perServer false also avoids reading CLUSTER_JSON at all, so
+   * the page can be produced without a valid cluster entry for this host.
+   */
+  perServer?: boolean;
+}
+
+export async function renderHtmlContent(
+  htmlPath: string,
+  opts: RenderHtmlOptions = {},
+): Promise<string> {
+  const perServer = opts.perServer ?? true;
   const htmlContent = await fs.readFile(htmlPath, "utf-8");
   const assetManifest = await getRuntimeAssetManifest();
   const cdnBase = ServerEnv.cdnBase();
+  // Omitted entirely (not set to a falsy string) when perServer is false: the
+  // template guards each of these with `typeof x !== "undefined" && x`, so an
+  // absent local drops the whole line, indentation and trailing comma
+  // included.
+  const perServerLocals = perServer
+    ? {
+        // The fleet map plus which entry is this server. Replaces the old
+        // numWorkers scalar: the client derives its own-server worker count
+        // from cluster[instanceLetter], and (PR 5) routes foreign game ids by
+        // their leading letter.
+        cluster: JSON.stringify(ServerEnv.cluster()),
+        instanceLetter: JSON.stringify(ServerEnv.instanceLetter()),
+        instanceId: JSON.stringify(ServerEnv.instanceId()),
+        serverHost:
+          ServerEnv.publicHost() === undefined
+            ? undefined
+            : JSON.stringify(ServerEnv.publicHost()),
+        // The load-balancer apex, when this deployment sits behind one. The
+        // client uses it as the unknown-letter redirect target — the apex shell
+        // always carries the freshest cluster map. Absent for standalone
+        // deployments (beta, branch previews, dev), which have no apex to
+        // bounce to and fall through to their normal not-found flow.
+        siteHost:
+          ServerEnv.siteHost() === undefined
+            ? undefined
+            : JSON.stringify(ServerEnv.siteHost()),
+      }
+    : {};
   return ejs.render(htmlContent, {
+    ...perServerLocals,
     gitCommit: JSON.stringify(ServerEnv.gitCommit()),
     assetManifest: JSON.stringify(assetManifest),
     cdnBase: JSON.stringify(cdnBase),
@@ -25,28 +82,8 @@ export async function renderHtmlContent(htmlPath: string): Promise<string> {
     // refs to use this placeholder.
     cdnBaseRaw: cdnBase,
     gameEnv: JSON.stringify(ServerEnv.gameEnvName()),
-    // The fleet map plus which entry is this server. Replaces the old
-    // numWorkers scalar: the client derives its own-server worker count from
-    // cluster[instanceLetter], and (PR 5) routes foreign game ids by their
-    // leading letter.
-    cluster: JSON.stringify(ServerEnv.cluster()),
-    instanceLetter: JSON.stringify(ServerEnv.instanceLetter()),
     turnstileSiteKey: JSON.stringify(ServerEnv.turnstileSiteKey()),
     jwtAudience: JSON.stringify(ServerEnv.jwtAudience()),
-    instanceId: JSON.stringify(ServerEnv.instanceId()),
-    serverHost:
-      ServerEnv.publicHost() === undefined
-        ? undefined
-        : JSON.stringify(ServerEnv.publicHost()),
-    // The load-balancer apex, when this deployment sits behind one. The
-    // client uses it as the unknown-letter redirect target — the apex shell
-    // always carries the freshest cluster map. Absent for standalone
-    // deployments (beta, branch previews, dev), which have no apex to
-    // bounce to and fall through to their normal not-found flow.
-    siteHost:
-      ServerEnv.siteHost() === undefined
-        ? undefined
-        : JSON.stringify(ServerEnv.siteHost()),
     manifestHref: buildAssetUrl("manifest.json", assetManifest, cdnBase),
     faviconHref: buildAssetUrl("images/Favicon.svg", assetManifest, cdnBase),
     gameplayScreenshotUrl: buildAssetUrl(

src/server/RenderStaticIndex.ts

@@ -3,13 +3,38 @@
 // this inside the freshly built image at deploy time and uploads the result to
 // the CDN as index-<short-commit>.html, so games archived from this build stay
 // replayable after the deployment itself is gone (#4934).
+//
+// With --environment-only it renders the same template WITHOUT the per-server
+// locals (cluster, instanceLetter, instanceId, serverHost, siteHost). That
+// page describes a build and an environment, not a server, so one copy can be
+// uploaded per version as sites/<site>/v/<short>/index.html and served to
+// every player on that version by the static Worker; the client asks the API
+// for the server list instead (docs/MultiServer.md, "Server list v2").
+//
+// The default is deliberately unchanged: the legacy index-<short>.html replay
+// shell still needs the server values, because today's client throws without a
+// worker-count source (fixed by OPE-431, not before).
 import path from "path";
 import { fileURLToPath } from "url";
 import { renderHtmlContent } from "./RenderHtml";
 
 const __dirname = path.dirname(fileURLToPath(import.meta.url));
 
-renderHtmlContent(path.join(__dirname, "../../static/index.html")).then(
+const ENVIRONMENT_ONLY = "--environment-only";
+
+const args = process.argv.slice(2);
+const unknown = args.filter((a) => a !== ENVIRONMENT_ONLY);
+if (unknown.length > 0) {
+  console.error(
+    `Unknown argument(s): ${unknown.join(" ")}. Usage: RenderStaticIndex.ts [${ENVIRONMENT_ONLY}]`,
+  );
+  process.exit(2);
+}
+const perServer = !args.includes(ENVIRONMENT_ONLY);
+
+renderHtmlContent(path.join(__dirname, "../../static/index.html"), {
+  perServer,
+}).then(
   (html) => process.stdout.write(html),
   (error: unknown) => {
     console.error("Failed to render static index:", error);