feat(simulator): dev-only Discord chat simulator on the real agent

[rayhanadev]

Why

There was no way to see what the bot sends to Discord — streamed edits, tool-activity lines, subagent previews, footers, message splitting, approval cards — without deploying and poking a real server: slow, noisy, not reproducible. This adds a gated, dev-only /simulator that runs the real orchestrator locally and renders that UX faithfully, so message construction and the model's context can be iterated in a tight loop. It's also a harness for running agents locally.

Before:

Iterate on message UX → deploy → @mention the bot in a real Discord server →
read it back by eye → repeat. No view into the context/trace the model saw.

After:

SIMULATOR_ENABLED=1 SKIP_ENV_VALIDATION=1 next dev → open /simulator →
@mention the bot → watch the real thread/stream/footer render, with a live
context snapshot + execution trace in the side inspector.

What changed

• /simulator page (src/app/simulator/**): a two-pane React UI — a high-fidelity Discord chat column (chat → thread → follow-up) and a tabbed inspector (session Controls, live Context snapshot, execution Trace). Hand-rolled Discord-markdown renderer, embeds, and clickable approval cards.
• Drives the bot's real plumbing (src/lib/simulator/**): builds a GATEWAY_MESSAGE_CREATE packet + HandlerContext, runs the actual mention detection and prepareFreshTurn (thread creation, lead-in, placeholder, AgentContext), then streamTurn. Two virtual Discord transports (fake @discordjs/core API + smart @discordjs/rest proxy) capture exactly what the bot emits; events fan out over SSE.
• Context tab reuses the real buildContextSnapshot + /inspect-context breakdown; Trace tab is fed by teeing the orchestrator's tool-call stream (delegations render as delegate › <domain>).
• Production seams — all dev-gated, no-op in production:
  • __setDiscordRestForSimulation / __setRedisForSimulation overrides
  • extracted prepareFreshTurn + replyEmptyMention from the mention handler
  • extracted conversation-turn.ts (history capping) shared with chat.ts
  • gated app.route("/simulator") mount behind isSimEnabled()

Test plan

• bun run typecheck — clean
• bun lint — 0 errors
• bun format --check — clean
• bun test:coverage — 1066/1066 pass; thresholds met (stmts 92.4%, branches 88.7%, funcs 92.3%, lines 92.9%)
• Manual (in-browser, Chrome): sent a mention → real thread created, real orchestrator engaged, reply/footer/error all rendered via real bot code; Context + Trace tabs populate. A full streamed reply needs gateway creds (AI_GATEWAY_API_KEY or a fresh VERCEL_OIDC_TOKEN) — env/ops only, not code.

Dev-only: never set SIMULATOR_ENABLED on a deployment. The transport/Redis swaps no-op in production and the route only mounts behind the gate.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
wack-hacker	Ready	Preview, Comment	Jun 16, 2026 2:13am

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.44%. Comparing base (a6ea950) to head (995fc53).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #146      +/-   ##
==========================================
+ Coverage   98.42%   98.44%   +0.01%     
==========================================
  Files          73       74       +1     
  Lines        1910     1934      +24     
  Branches      488      496       +8     
==========================================
+ Hits         1880     1904      +24     
  Misses         14       14              
  Partials       16       16              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pullfrog]

ℹ️ No critical issues — a few minor seam/lifecycle observations inline.

Reviewed changes — a gated, dev-only /simulator that runs the real orchestrator locally and renders the bot's Discord UX, plus the production-side extractions that let it reuse the real plumbing instead of reimplementing it.

• Simulator runtime (src/lib/simulator/**) — SimConversation turn loop, SimEventBus SSE promise-queue fan-out, run-registry single-active-session, bootstrap installing the process-global fakes, virtual guild + fake core/REST transports, and a trace-orchestrator that tees the real orchestrator stream into the Trace tab.
• Simulator UI (src/app/simulator/**) — two-pane React chat + inspector, hand-rolled Discord-markdown renderer, message-store, and the SSE hook.
• Production seams (dev-gated) — __setDiscordRestForSimulation turns the discord REST export into a swappable Proxy; __setRedisForSimulation overrides the memoized Redis client; app.route("/simulator") mounts only behind isSimEnabled().
• Shared extractions — prepareFreshTurn + replyEmptyMention lifted out of the mention handler, and conversation-turn.ts (truncateForHistory / summarizeDroppedHistory / capHistory / stampCurrentTime) shared with workflows/chat.ts; LEADIN_TURN_LIMIT moved to src/lib/ai/constants.ts.

I verified the production-path refactor carries no semantic drift: the wide-event metric counts (payload.context.recentMessages/referencedContext) are carried unchanged by AgentContext.toJSON(), capHistory's injected summarizer keeps the workflow's "use step" durability while the simulator uses the byte-identical plain default, and the moved constants are unchanged. The findings below are all dev-only and non-blocking.

ℹ️ Concurrent turns on one session are unguarded

SimConversation.runTurn mutates this.messages, this.turnCount, this.totalUsage, this.stableContext, and this.lastBotMessageId, and getOrCreateSession hands back the same instance for a session id. Two overlapping /chat POSTs (two tabs, a double-submit, or a quick retry) interleave those mutations — history ends up with two consecutive user turns, turnCount/totalUsage lose an update via read-modify-write, and both SSE streams see each other's run.finish. It's dev-only and self-inflicted, but a one-line in-flight guard would make the failure obvious instead of silently corrupting the transcript.

Technical details

# Concurrent turns on one session are unguarded

## Affected sites
- `src/lib/simulator/conversation.ts:169` — `runTurn` is re-entrant; no lock/flag.
- `src/lib/simulator/run-registry.ts:11` — `getOrCreateSession` returns the shared instance.

## Required outcome
- Overlapping turns on one session must not interleave `messages`/`turnCount`/`totalUsage`, and one turn's `run.finish` must not terminate another turn's SSE pump early.

## Suggested approach (optional)
- Cheapest: reject re-entry — `if (this.running) throw new Error("turn in progress");` set/cleared around the body. For a dev tool an explicit error is friendlier than silent serialization.
- Alternatively chain: `this.running = (this.running ?? Promise.resolve()).then(() => this.doTurn(req)); return this.running;`

## Open questions for the human
- Is the simulator ever expected to serve more than one local tab/process? If strictly single-tab localhost this is cosmetic.

ℹ️ import simulator is unconditional — module graph enters the production bundle

The if (isSimEnabled()) gate only guards the app.route() call; the import simulator from "./routes/simulator" runs at module load in every process, pulling the whole simulator graph (bootstrap → the override setters, fakes, SimConversation) into the production bundle. No override fires at import time today, so it's latent rather than active — but it removes the isolation guarantee the gate is meant to provide, and any future top-level init in that graph would run in prod.

Technical details

# Unconditional simulator import in production server

## Affected sites
- `src/server/index.ts:10` — `import simulator from "./routes/simulator";` always evaluated.
- `src/server/index.ts:24` — `if (isSimEnabled()) app.route("/simulator", simulator);` only gates the mount.

## Required outcome
- The simulator module graph (and any future top-level side effects in it) must not load in a production process.

## Suggested approach (optional)
- Move to a dynamic import inside the gate:
  `if (isSimEnabled()) app.route("/simulator", (await import("./routes/simulator")).default);`
  (or a top-level `await` / lazy mount, matching how the server already initializes).

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes — the single new commit since my prior review (62986ad → 38da911) is pure cleanup with no behavioral impact.

• Drop dead constant — removes the unused SIM_DEFAULT_TIMEZONE from src/lib/simulator/constants.ts.
• Widen knip coverage — knip.json now scans src/**/*.{ts,tsx} and adds page.tsx / layout.tsx as entry points so the new simulator frontend is dead-code-linted.

Neither change touches the seams or runtime logic from my prior review, and the knip.json edit is a clean single-key update (no duplicate keys). The five informational findings from the initial review (override-setter gating, bus.close() on session swap, concurrent-turn guard, unconditional simulator import, .gitignore .env* glob) still stand and remain non-blocking.

  ^{｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[rayhanadev]

Triaged the review (thanks @pullfrog) in 2503485 — all three were valid:

• client.ts / redis/client.ts no-op gate — good catch; the guard only checked NODE_ENV, weaker than the route mount. Both setters now also require SIMULATOR_ENABLED === "1", matching isSimEnabled(), so the transport/Redis swaps can't fire on a non-prod preview deploy. fake-rest.test sets the flag to keep exercising the swap.
• run-registry session swap — the outgoing SimEventBus is now close()d on swap/reset, releasing any lingering SSE subscriber.
• .gitignore .env* — that blanket pattern was an accidental addition (I only meant .sim/); removed it. .env.local / .env*.local already cover the secrets, and .env.example stays tracked.

[pullfrog]

Verified all three triaged findings are correctly addressed in 2503485 and resolved their threads:

• Override-setter gating — __setDiscordRestForSimulation (discord/client.ts) and __setRedisForSimulation (redis/client.ts) now both require SIMULATOR_ENABLED === "1" in addition to the NODE_ENV check, matching isSimEnabled(). fake-rest.test.ts sets the flag and passes (2/2, verified locally).
• Session swap leak — getOrCreateSession/resetSessions now close() the outgoing SimEventBus before discarding the old SimConversation.
• .gitignore — the stray blanket .env* is removed; .env*.local covers secrets and .env.example stays tracked.

No code changes needed from me — the fixes were already live. Replied to and resolved all three threads. The two body-only informational findings (concurrent-turn guard, unconditional simulator import) had no inline threads and were not part of this triage, so they remain open for your discretion.

  ^{｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes — the new commit since my last review (38da911 → 2503485) tightens the dev-only simulator gates and drops the stray .env* ignore, directly addressing three of my prior informational findings.

• Gated the override setters on SIMULATOR_ENABLED — __setDiscordRestForSimulation (src/lib/ai/tools/discord/client.ts) and __setRedisForSimulation (src/lib/redis/client.ts) now bail unless NODE_ENV !== "production" and SIMULATOR_ENABLED === "1", matching the isSimEnabled() route mount so the public setters can't swap global agent transports on a non-prod preview/staging deploy.
• Closed the bus on session swap — getOrCreateSession and resetSessions (src/lib/simulator/run-registry.ts) now call current?.bus.close() before discarding the old SimConversation, releasing any lingering SSE subscriber instead of leaking a never-resolving next() promise.
• Dropped the .env* ignore — .gitignore no longer shadows the committed .env.example.
• Kept tests green under the new gate — fake-rest.test.ts and conversation.test.ts set SIMULATOR_ENABLED = "1" so the now-gated setters still fire under test.

I verified each fix against the current tree and confirmed the gating logic is the exact De Morgan inverse of isSimEnabled(), so behavior matches the route gate. The affected fake-rest / conversation / redis suites pass under vitest. The two remaining prior findings (concurrent-turn guard, unconditional import simulator) are untouched by this commit and stay non-blocking — not re-raised here.

  ^{｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes — the single new commit since my last review (c924530 → acdd2cc) gates the dev-only /simulator UI segment so it matches its already-gated API.

• Gated the /simulator page server-side — new src/app/simulator/layout.tsx, a Server Component (no "use client") that calls notFound() unless isSimEnabled(). This closes a real gap: previously only the Hono app.route("/simulator") mount was gated, so the Next.js page itself would render in any env that served it. Now the whole segment 404s under NODE_ENV=production, mirroring the API.

I confirmed the gate runs server-side (env-readable), nests correctly under the root <html>/<body> layout, and that a production build simply prerenders the 404 — the desired outcome. The two remaining prior informational findings (concurrent-turn guard in SimConversation.runTurn, unconditional import simulator into the production bundle in src/server/index.ts) are untouched by this commit and stay non-blocking — not re-raised here.

  ^{｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}