Skip to content

Latest commit

 

History

History
78 lines (47 loc) · 21.9 KB

README.md

File metadata and controls

78 lines (47 loc) · 21.9 KB

apps/server — the local REST/SSE server

What this module is responsible for

The HTTP server procni dev starts: the REST API under /api/v1/* (health, provider status, triggering an analysis, listing opportunities, job status and live progress over SSE), static-hosting the built web dashboard, and running the background job worker. It also owns the mechanics of being a well-behaved local server — binding to a free port, refusing to double-start against the same project.

Why it exists / why this approach

Why Fastify, not Express. Fastify's schema-validation hooks integrate cleanly with the Zod schemas already used throughout this codebase (@procni/core's config, and every future pipeline stage's I/O), and its plugin model (app.register(...)) is how buildServer composes the health route and static file serving without either one knowing about the other. Express would work too, but offers nothing this project needs that Fastify doesn't already give for free.

Why the server binds only to 127.0.0.1, never 0.0.0.0. This is a security decision, not a networking default: Procni's API has no authentication, because it's designed to be reached only by the browser running on the same machine as the CLI. Binding to all interfaces would expose an unauthenticated API — including, eventually, endpoints that trigger AI calls billed to the user's own key — to anyone else on the same network. If remote access is ever wanted, that's an explicit, separately-designed feature (see the project plan's Phase 7 sync/self-hosting), not a flag that widens this bind address.

Why port auto-increment instead of always failing on a taken port. A developer working on two Procni-managed projects at once shouldn't have to remember and pass a different --port for the second one — listen.ts's listenOnAvailablePort tries 4173, then 4174, and so on, and always reports back the port it actually bound (reading it from the OS via app.server.address(), not just trusting the number it tried) so the caller is never wrong about where the server actually ended up.

Why a PID lock file, and why it treats an unreachable PID as stale rather than erroring. pidLock.ts is what stops a second procni dev from opening the same project's SQLite file concurrently, which per @procni/core/db's README is exactly the scenario WAL mode doesn't fully protect against on its own. The lock records {pid, port, startedAt}; assertNotAlreadyRunning only objects if that pid is a currently-running process, checked via @procni/core's isProcessAlive (process.kill(pid, 0), which sends no actual signal — see the comment where it's defined). If a previous procni dev was killed uncleanly (crash, kill -9, a machine sleep/wake edge case) its lock file survives but its process doesn't, and the next procni dev needs to recover on its own rather than making a developer hunt down and delete a stale lock file by hand. The job queue's stale-job recovery (packages/core/src/jobs/worker.ts) needs this exact same check for the same reason, which is why it lives in @procni/core rather than being defined locally here a second time.

Why startDevServer checks readProjectConfig before doing anything else. This was found by the test suite, not designed up front: an early version opened the database file immediately, which crashes with a raw native "directory does not exist" error if .procni/ was never created by procni init. startDevServer now fails fast with a clear NOT_INITIALIZED ProcniError instead — see bootstrap.test.ts's "refuses to start against a repo procni init hasn't touched" case, which is the regression test for exactly this.

Why the server still starts successfully when no AI provider is configured. Per the project plan, browsing existing projects and data should never require AI or network access — only new analysis does. resolveConfiguredProvider (in bootstrap.ts) returns null rather than throwing when no API key is found; ServerDeps.hasProvider carries that fact into the routes, so /api/v1/providers/health can report it plainly and POST /api/v1/analyze can refuse with a clear NO_PROVIDER error before ever enqueueing a job that could only fail.

Why /api/v1/analyze enqueues a job and returns immediately (202), rather than running the pipeline inline and returning the result. An analysis run means at least two AI calls in sequence — seconds, not milliseconds. Blocking the HTTP request for that long would tie up a connection and give the browser nothing to show in the meantime. Enqueuing via @procni/core's job queue (see that package's jobs README) means the request returns a jobId instantly, and the UI follows progress separately via /api/v1/jobs/:id/events.

Why SSE (/api/v1/jobs/:id/events) polls the database internally instead of the job worker pushing events directly. There's no in-process event bus between the worker and an HTTP route today, and building one would be more machinery than a single-user local server needs — the worker already writes every state change to generation_jobs, so a route that polls that same table on a short interval and forwards only real changes to the client gets the same observable behavior (see routes/jobs.test.ts's real-HTTP test, which proves a client watching the stream sees a job progress from queued through to succeeded) without a second communication channel to keep in sync with the first.

Why ServerDeps.providerId is a provider id string (or null), not the hasProvider: boolean it used to be. Once @procni/ai grew Anthropic/OpenAI/OpenRouter/Ollama adapters alongside Gemini, "is a provider configured" and "which one" became two different facts a boolean can't hold — and /api/v1/providers/health needs to report the real id, not hardcode "gemini" the way it did when Gemini was the only option. bootstrap.ts gets both the id and the resolved AIProvider from @procni/ai's resolveConfiguredProvider in one call now, instead of the inline Gemini-only resolveConfiguredProvider this file used to define itself.

Why the default AI model is no longer a hardcoded constant in bootstrap.ts. With five possible providers, one hardcoded model name stopped making sense — @procni/ai's resolveConfiguredProvider now returns the right default for whichever provider it actually resolved (DEFAULT_MODEL_BY_PROVIDER, still hardcoded until provider_configs settings storage exists — see packages/ai/README.md's note on that same deferral, just moved to live in one place instead of duplicated per caller).

Why PATCH /api/v1/settings writes straight to .procni/config.toml instead of a database table. Project-level settings (a voice/tone styleGuide and per-platform wordLimits) are exactly the kind of human-editable, git-shareable state the project's domain model deliberately keeps out of SQLite — see packages/core/src/db/README.md's note on Settings being excluded from the 8-table schema for this reason. routes/settings.ts is a thin read-modify-write over @procni/core's existing readProjectConfig/writeProjectConfig — no new persistence layer. Both fields are independently optional in the PATCH body, and a wordLimits update merges into the existing map rather than replacing it ({...config.wordLimits, ...body.wordLimits}) — so setting just twitter's limit from the web UI never requires resending the other three.

Why POST /api/v1/opportunities/custom is standalone (no :id), with its own fresh analysis_runs row. This used to be POST /api/v1/opportunities/:id/custom, scoped to an existing opportunity's evidence — a developer could only pick from files a prior automated analysis had already surfaced. The file-search compose flow replaced that: a developer searches the whole repo by filename (via routes/files.ts, below), not just one opportunity's diff, so there's no sensible existing opportunity to scope the request to. The route re-validates every selected path against listRepoFiles itself (the same allowlist readRepoFile uses) rather than trusting an arbitrary client-supplied path, records the current HEAD sha as both fromSha/toSha via getHeadSha (there's no diff being captured, just point-in-time snippets), and gives the new opportunity its own analysis_runs row (triggeredBy: "manual") since it isn't derived from any prior run. See packages/core/src/pipeline/README.md's note on EvidenceSchema.snippets for how the highlighted text itself (not a whole file's diff) gets captured.

Why DELETE /api/v1/opportunities/:id is a separate action from dismiss, not the same status flip taken further. Dismiss/restore is explicitly non-destructive — a change of mind should always be free. Delete is the opposite: a developer explicitly asking to permanently remove an opportunity and every draft it produced. deleteContentOpportunity runs a real cascade (publications → content_drafts → content_opportunities, in FK-dependency order, inside one transaction) because SQLite foreign keys aren't enforced in this project — see packages/core/src/db/README.md. The web UI gates this behind a confirmation dialog naming exactly what's being deleted, since a 204 from this route is genuinely irreversible.

Why GET /api/v1/files and GET /api/v1/files/content exist, and why they need no AI provider. They back the file-search compose flow's left pane: search the repo by filename, then read one file's current content to highlight lines from. Both are thin wrappers over @procni/core's listRepoFiles/readRepoFile (see that package's git README) — listRepoFiles is the allowlist that keeps a client-supplied path query parameter from ever reaching an arbitrary filesystem read, the same defense-in-depth pattern the rest of this project uses for anything client-facing that touches the filesystem.

Why GET /api/v1/publications/activity and GET /api/v1/publications/recent exist, and why the activity route ships raw per-publication rows instead of pre-bucketed daily counts. They back the home page's GitHub-style activity graph and recent-posts feed. Bucketing by calendar day is inherently a local-timezone concern — the same instant can fall on different calendar days depending on where the developer is — so doing it in the browser (web/src/ActivityGraph.tsx) against the browser's own timezone is simpler and more correct than the server guessing or requiring a timezone parameter. Both routes are thin reads over @procni/core's listPublicationActivity/listRecentPublications — see that package's db README — and need no AI provider, since "published" here means a real publications row (a copy/export action), not anything AI-generated.

Why draft generation accepts an overrides object in its body, on top of the project's saved settings. ProjectConfig's styleGuide/wordLimits/toneToggles/emojiUse are what "every future draft on this platform" defaults to. Sometimes a single post should deviate from that without changing the project-wide default — a per-post overrides field (validated by the same TONE_KEYS-derived schema as settings) is forwarded straight into the job payload and merged on top of the config inside generateDraft.ts, never persisted anywhere. See packages/core/src/pipeline/README.md's note on GenerateDraftOverrides.

Why POST /api/v1/drafts/:id/restore inserts a new row instead of flipping is_current on the old one directly. It reuses insertContentDraft's existing revisionOf transaction (the same one a regeneration uses) so the full chain of what happened to a draft — generated, edited, regenerated, restored — stays visible in content_drafts, rather than a restore silently rewriting history. Restoring the draft that's already current is a no-op that returns it unchanged (200, not 201) rather than creating a pointless duplicate row.

Why POST /api/v1/opportunities/:id/draft now requires a platform in its body, and GET /api/v1/opportunities/:id/draft (singular, no platform) was replaced by GET /api/v1/opportunities/:id/drafts/current. The review UI grew independent platform tabs (X/Twitter, LinkedIn, Blog, Changelog), each generating and holding its own draft — "generate a draft for this opportunity" stopped being well-defined without saying which platform, and "the current draft" became "every platform's current draft" (0 to one-per-platform), which is a list, not a single nullable object. packages/core/src/db/README.md has the repository-level version of this same change (getCurrentDraftForOpportunity gaining a required platform argument).

Why GET /api/v1/opportunities/:id/evidence exists and needs no AI provider. It returns exactly the diffs generateDraft would send to a model (via @procni/core's getEvidenceFileDiffs — see packages/core/src/pipeline/README.md), computed straight from git with no model call involved. The web UI's source-diff panel uses it so a developer can see what evidence a draft is grounded in — or would be, before generating one — without spending an AI call just to look.

Why POST /api/v1/drafts/:id/publish only records an action instead of performing the copy or export itself. Copying to a clipboard and downloading a file are both things only the developer's own browser can actually do (the server has no clipboard, and "downloading a file" server-side would mean writing it somewhere and inventing a second endpoint to fetch it back). routes/drafts.ts's publish route exists purely so those real, human-triggered, client-side actions have a place to be recorded for history and status — never so Procni can publish something on a developer's behalf, which the project's security design explicitly rules out for V1.

Walkthrough of the key code

  • types.ts — ServerDeps, the bundle every route function receives: the open database, the current project's repoRoot/projectName, whether a provider is configured, and the default model. Passed explicitly rather than attached to the Fastify instance via a decorator, so it's always obvious where a route's data comes from.
  • routes/health.ts — the health check. Every route mounts the same way: a function taking a FastifyInstance and ServerDeps, registering paths on the instance.
  • routes/providers.ts — GET /api/v1/providers/health, described above.
  • routes/analyze.ts — POST /api/v1/analyze, described above.
  • routes/opportunities.ts — GET /api/v1/opportunities, a thin pass-through to @procni/core's listContentOpportunities, POST /api/v1/opportunities/:id/{dismiss,restore} — a status flip, not a delete, so a dismissed opportunity stays visible to the UI with status: "dismissed" rather than disappearing — GET /api/v1/opportunities/:id/evidence, POST /api/v1/opportunities/custom (create an opportunity from hand-picked file/line selections, its own fresh analysis run), and DELETE /api/v1/opportunities/:id (a real, irreversible delete), all described above.
  • routes/files.ts — GET /api/v1/files (substring search over listRepoFiles, capped at 50 results) and GET /api/v1/files/content (one file's current content via readRepoFile, 404 if it's not something Procni can read), described above.
  • routes/publications.ts — GET /api/v1/publications/activity (raw {exportedAt, platform} rows over a trailing ?days= window, default 371) and GET /api/v1/publications/recent (the most recent publication events joined with draft/opportunity info, ?limit= default 10), described above.
  • routes/drafts.ts — POST /api/v1/opportunities/:id/draft (enqueues, requires platform in its body and accepts an optional overrides object, both described above), GET /api/v1/opportunities/:id/drafts/current (every platform's current draft at once) / GET /api/v1/drafts/:id (fetch one by id, 404 as "no such draft" rather than an error), POST /api/v1/drafts/:id/publish (record-only, described above), PATCH /api/v1/drafts/:id (edit the body — re-redacted the same way generated text is, see the route's own comment), POST /api/v1/drafts/:id/approve, and the revision-history routes: GET /api/v1/opportunities/:id/drafts (a platform's revision chain, newest first, via an optional ?platform= filter) and POST /api/v1/drafts/:id/restore (make an old revision current again), described above.
  • routes/settings.ts — GET/PATCH /api/v1/settings, described above: read/write config.toml's styleGuide, wordLimits, toneToggles, and emojiUse fields directly, no repository involved; PATCH merges each per-platform map independently rather than replacing it wholesale.
  • routes/jobs.ts — GET /api/v1/jobs/:id (a single status snapshot) and GET /api/v1/jobs/:id/events (the SSE stream, described above). Read streamJobUntilTerminal closely: the handler's returned promise is deliberately held open until the stream closes — resolving early while reply.raw is still open would make Fastify try to send its own response on top of the raw stream it doesn't know about.
  • app.ts — buildServer(webDistDir, deps), purely wiring: register every route above, and register static file serving only if webDistDir exists on disk (a fresh clone that hasn't run pnpm --filter web build still gets a working API, just no dashboard — see apps/cli/src/commands/dev.ts for what a developer sees in that case).
  • listen.ts — listenOnAvailablePort, described above.
  • pidLock.ts — assertNotAlreadyRunning / writeLock / releaseLock, described above.
  • bootstrap.ts — startDevServer, the one function procni dev calls: confirm init has run → confirm no other instance is running → open (and thereby migrate) the database → resolve a configured AI provider if one exists → sweep stale jobs and start the worker → build and bind the server → write the lock → hand back a {url, port, stop} handle. stop() unwinds in the right order too: stop the worker before closing the database it writes to.

How to test it

pnpm --filter @procni/server test

app.test.ts uses Fastify's .inject() (in-process request, no real socket) against a real temporary SQLite database for the cheap route-level checks — health, provider status, the NO_PROVIDER refusal, an empty opportunities list, a 404 for a missing job. listen.test.ts and pidLock.test.ts exercise real sockets and real temp-directory lock files respectively. bootstrap.test.ts runs startDevServer for real against a temp project directory and asserts the NOT_INITIALIZED/ALREADY_RUNNING failure paths. routes/analyze.test.ts confirms a real generation_jobs row is created with the right payload; routes/drafts.test.ts does the same for draft generation (now asserting the enqueued job's payload carries the requested platform), plus the publish/edit/approve/history/restore routes' status-flip, redaction, and 404 cases (including that restoring the already-current draft is a no-op, that restoring an old one leaves its row intact rather than mutating it, and that restoring on one platform never touches another platform's current draft); routes/opportunities.test.ts covers dismiss/restore (including that a dismissed opportunity stays in the list rather than vanishing from it), the evidence route against a real temporary git repo (asserting the returned diff actually contains the real added line), the standalone custom-opportunity route (creating one from hand-picked selections in its own fresh analysis run, joining multiple selections from the same file into one snippet, and rejecting a path that isn't a tracked file), and delete (the opportunity disappears from the list, and deleting a nonexistent one 404s). routes/files.test.ts covers search-by-substring, listing everything when no query is given, reading a file's current content, and the 404/400 paths for an untracked path or a missing path parameter. routes/publications.test.ts covers the activity route returning each publication's platform/timestamp and excluding anything outside the requested ?days= window, and the recent-posts route returning joined draft/opportunity info newest-first and respecting ?limit=. routes/settings.test.ts round-trips a style guide, per-platform word limits, tone toggles, and emoji use through PATCH/GET against a real temp config.toml, including that updating one platform's settings leaves every other platform's (and the style guide's) untouched. routes/jobs.test.ts is the deepest integration test in this package: a real temp git repo, a real database, a real bound HTTP server, a real @procni/core job worker wired to a FakeProvider-backed analyze handler, and a real fetch() reading the SSE stream chunk-by-chunk until it closes — proving a client watching /api/v1/jobs/:id/events actually observes a job move from queued through to succeeded, not just that the final state is queryable. Every test that binds a real port passes port: 0 so Vitest can run test files in parallel without them fighting over a fixed port number.

To try it by hand, see apps/cli's manual-testing section — procni dev (and now procni analyze, once a real Gemini key is configured) are the real entry points into this module.

What it depends on / what depends on it

Depends on @procni/core (db, the job worker/repositories, openDb/readProjectConfig/resolveProjectDir), @procni/ai (resolving the configured provider), and @procni/shared (ProcniError). Depended on by apps/cli's dev command. Depended on by web once its opportunity list UI exists (not yet built) — that's the next piece to consume these routes.