Skip to content

Latest commit

 

History

History
625 lines (470 loc) · 59.5 KB

project-structure.md

File metadata and controls

625 lines (470 loc) · 59.5 KB

Project Structure

Quick reference for organizing features using vertical slice architecture.

lib/ vs features/ vs packages/

Use lib/ for:

  • Internal utilities used across multiple features
  • Framework-specific adapters (Next.js, React)
  • Cross-cutting concerns that aren't ready for packages

Use features/ for:

  • Endatix-specific business logic
  • App configuration (features/config/)
  • Complete vertical slices with use-cases, UI, and infrastructure

Use packages/ for:

  • Future npm packages with their own package.json
  • Reusable libraries that can be published independently
  • Workspace packages that other projects can consume

Use src/ for:

  • Standalone browser bundles (not Next). Today: src/embed → public/embed/v1/embed.js. See AGENTS.md "Embed SDK".

Use components/ for:

  • Cross-feature UI with no single owning domain
  • Design-system vocabulary governed by DESIGN.md
  • Vendor primitives (components/ui/, shadcn)

Where UI for a shared concept lives

A reusable concept that grows a UI (file kinds, list tables, statuses) does not automatically grow a ui/ folder next to its logic. Two shapes already exist in this repo and they are not interchangeable:

Shape When Example
lib/<domain>/<slice>/ui/ The domain is itself a vertical slice — client-only, owns its use-cases, and its UI has exactly one consumer: that domain lib/survey-features/form-diagnostics/ui/, lib/survey-extensions/ui/
lib/<domain>/ + components/… The logic is a catalog or helper many unrelated features read, and the UI is presentation vocabulary over it lib/list-page/table-return-to + components/table/back-to-table-button.tsx; lib/file-kinds/ + components/common/file-kind-icon.tsx

Decision rule — apply in order; the first test that answers, wins:

  1. Does server code import the domain? If a route handler, server action or server page imports it, UI must not live inside it. One "use client" file with a lucide-react import turns the domain's barrel into a client dependency, and the next import { FILE_KINDS } from "@/lib/file-kinds" in a route handler drags React into a request path. This test is mechanical, not taste, and it is why lib/file-kinds/ has no ui/.
  2. Who consumes the UI? One domain, and only through that domain → keep it domain-local in lib/<domain>/<slice>/ui/. Unrelated features import it directly → components/.
  3. Does DESIGN.md govern how it looks? A status pill, a file mark, table chrome — anything with a rule in the design system — lives in components/, where the §5 component index points. A component the design system describes but that is filed inside one domain will be re-invented by the next feature that cannot find it.

Placement inside components/:

  • components/ui/ — shadcn/vendor primitives. Do not add hand-written Hub components here; a local component in this folder reads as vendor code and gets clobbered by the next generator run.
  • components/common/ — a single design-vocabulary component per file, no barrel, imported by full path (@/components/common/status-badge).
  • components/<domain>/ — a set of related parts with an index.ts barrel. components/table/ is the reference. Graduate a common/ component here once it is roughly three files, or once the parts share internal conventions (naming, class-name helpers, meta types) that outsiders should not reach past the barrel for. Moving one file is cheap; splitting a folder nobody can navigate is not.

Worked example — file kinds (H933):

lib/file-kinds/file-kinds.ts        # catalog: extension, MIME, label, group — server-safe
lib/file-kinds/file-kind-icons.ts   # kind → Lucide icon; kept out of index.ts so the
lib/file-kinds/index.ts             #   barrel stays free of lucide-react (test 1)
components/common/file-kind-icon.tsx  # FileKindIcon / FileKindLabel — the rendered mark (tests 2, 3)
features/export/utils.ts            # export wire keys + delivery enum → FileKindKey

The catalog is data and stays in lib/. The mark is design-system vocabulary and stays in components/. Feature code maps its own vocabulary onto a FileKindKey and never returns a LucideIcon — see DESIGN.md §5 "File Type Marks". When file UI grows past this one component (previews, upload chips, size labels), it graduates to components/files/ with a barrel; that is a move, not a redesign.

Anti-patterns:

  • Creating lib/<domain>/ui/ for a single component two features import. That is a components/common/ component with extra path.
  • Creating components/<domain>/ for one file. Start in common/, graduate.
  • Duplicating a design-system component inside a domain because the shared one was hard to find. Fix the index in DESIGN.md §5 instead.

Feature Structure

Vertical Slice Architecture: organize by business feature, then by use-case slice.

features/
├── {feature-name}/
│   ├── server.ts                             # Optional: curated server-safe exports
│   ├── index.ts                              # Optional: curated shared/client-safe exports
│   ├── types.ts                              # Feature-shared types across slices
│   ├── utils.ts                              # Feature-shared pure helpers
│   ├── ui/                                   # Cross-slice reusable UI only
│   ├── {verb-noun}/                          # Use-case slice (preferred)
│   │   ├── {verb-noun}.action.ts             # Server action for this slice
│   │   ├── {verb-noun}.server.ts             # Optional server-only loader/helper
│   │   ├── use-{noun}.hook.ts                # Optional slice-local hook
│   │   ├── ui/                               # Slice-local UI
│   │   ├── __tests__/                        # Slice-local tests
│   │   └── index.ts                          # Slice barrel exports
│   └── __tests__/                            # Optional cross-slice integration tests

Reference Example: features/folders

Use features/folders as the canonical implementation of this pattern:

  • slices like create-folder, update-folder, delete-folder, move-form-to-folder, view-forms-header
  • shared root artifacts in types.ts, utils.ts, and root ui/ for cross-slice components
  • server-safe curated exports via features/folders/server.ts
  • slice-local tests in features/folders/create-folder/__tests__/
  • create, update, and delete revalidate revalidatePath("/", "layout") so the sidebar nav slot reloads. Page paths alone leave the layout cache stale. Reference: create-folder.action.ts.

Diagnostic views project, they do not re-declare

A page that reports on a domain (admin Environment, health views) imports that domain's type instead of writing a matching one. Re-spelling a union or a field list in a view-*/types.ts creates two definitions that drift silently, and the page ends up describing settings it does not own.

features/platform-admin/view-environment-settings/types.ts is the worked example: featureFlags is FlagSettings from lib/feature-flags/flag-settings.ts, analytics is a Pick<ClientEndatixConfig, …>, and the server loader calls the domain's own readFlagSettings() rather than assembling fields. Keep the shared vocabulary in a dependency-free module so client components can type-import it.

Public tenant auth (features/tenants/public-tenant)

Unauthenticated tenant sign-in / self-registration lives here, not under platform-admin: platform-admin is client-imported and must stay free of NextAuth. The slice owns the public-tenant action, the provider allow-list filter, and the (auth) failure states. Hub types use shortUrl; POST /auth/register still sends tenantSlug. See Hub AGENTS.md “Endatix IDs” and “Auth pages”.

Signup request slices (features/platform-admin)

  • list-signup-requests/ owns the queue: URL parsing, the loader, the toolbar and the grid.
  • review-signup-request/ owns the decision: the approve, reject and retry actions, the review panel, the reviewer-name loader (listSignupReviewers), and signup-request-state.ts, the one mapping from decision and provisioning to tones, row badge and next step. The grid imports the state mapping and mounts the panel. It never decides itself. SignupRequestView (review-signup-request/types.ts) is what both render. toSignupRequestView (signup-request-view.server.ts) builds it for the list loader and every decision action: it drops the opaque metadata and keeps only the PostHog visitor reference. load-signup-visitor.server.ts turns that reference into the panel's Visitor section on demand, through features/analytics/posthog/server/posthog-read-api.server.ts. The panel calls getSignupVisitorAction after paint; a server action during render updates the router while the section is rendering. Keep Hub-only fields off lib/endatix-api types.
  • UX pattern: DESIGN.md §6 "Recipe: list page" (review queue) and "Recipe: review and decide", and "Evidence from third-party tools" for the Visitor section.

Test inspector (features/public-form/inspect-test-submission)

The Shared Form's corner on a test page. testCornerFor (features/public-form/test-submission-mode.ts) is the one gate: inspector, notice, or none. The server checks forms.test only on a test page, so a real respondent's load never reaches auth(), and builds a TestInspectorView with toTestInspectorView, the only reader of the wire Submission for it. The slice exports the lazy TestInspector (its own chunk, requested only when a view arrives), toTestInspectorView, and StartOver; the trigger, menu and panels stay internal. SurveyComponent takes renderCorner(model), so the inspector gets the model it sits beside. Embeds keep the plain notice plus Start over.

Reporting export slices (features/export)

Reporting export is a dedicated feature (not nested under forms/ or submissions/). Slices map to API capabilities and UI surfaces:

Slice Scope Responsibility
prepare-reporting-export Action-only Server action: compile FormSchema + backfill flattened submissions (fullRecompile → replace + force). Called by the Export dialog and by the submissions grid readiness banner.
export-submissions Submissions list Tenant-configured export download via exportFormatId (+ legacy CustomExports when flag off). Owns the Export dialog: readiness check, auto prepare when schema missing, optional Rebuild reporting data… (manual prepare + full recompile), format + capability-aware filters.
manage-export-formats Tenant settings (E10b) CRUD UI for tenant export formats + tenant default mapping picker (alias, key separator, include-test)

Shared feature root: types.ts, utils.ts (wire key / delivery enum → FileKindKey), export-url.ts, export-error-message.ts. backfill-reporting-submissions.server.ts is the one backfill paging loop: prepare-reporting-export fails when it stops at the batch cap, while export-submissions/refresh-incomplete-submissions.action.ts (the backfill-only refresh before an Incomplete/All export) reports a partial run and lets the export continue. Export permissions follow the API and are loaded once per submissions page: loadSubmissionExportAccess() (export-submissions/export-access.server.ts, from @/features/export/server) makes one evaluatePermissions call and returns SubmissionExportAccess — canExport (submissions.export: Export Submissions shows), canPrepare (submissions.export or forms.edit, the API's rule for compile and a backfill without force: the banner's Prepare and Include in-progress submissions show), canRebuild (forms.edit: Rebuild reporting data… shows in the dialog, since a full recompile clears flattened data). A failed check allows nothing. The server actions still check: list formats (submissions.export or tenant.settings.manage), prepare (forms.edit for fullRecompile only), and the refresh (submissions.export or forms.edit; skipped, not failed, without either). Reporting wireKey catalog: lib/endatix-api/reporting/reporting-export-wire.ts. File kinds and the rendered mark: lib/file-kinds/ + components/common/file-kind-icon.tsx (see “Where UI for a shared concept lives”). Export route stays thin and delegates parsing to export-submissions/parse-export-query.ts. Create-format delivery options come from the API capabilities catalog (wireKey e.g. xlsx); tenant format rows drive the submissions export dialog.

Import from @/features/export (client UI) or @/features/export/server (server actions). Slice barrels: @/features/export/export-submissions, @/features/export/prepare-reporting-export, @/features/export/manage-export-formats.

Note: Codebook/Shoji downloads use the same tenant formats + submissions export dropdown (no separate export-codebook / manage-export-mappings slices). Locale stays request-time on the export API and is not edited on format definitions. Prepare is entered from the export dialog (auto when not ready, or via Rebuild reporting data…) and from the submissions grid readiness banner (Prepare, and Include in-progress submissions via refreshIncompleteSubmissionsAction, both with canPrepare). Both banner actions router.refresh() the grid when they finish. Import them from @/features/export/server.

Organization Management Slices

Organization identity surfaces follow the same vertical-slice rule:

  • features/organization/user-management/use-cases/create-tenant-user owns tenant invites. Invite actions must not collect a password from the inviter.
  • features/organization/user-management/use-cases/delete-user owns tenant access removal. It keeps typed confirmation in UI and must not delete the global user identity.
  • features/organization/user-management/use-cases/resend-verification owns pending-user activation email resend.
  • features/organization/user-management/use-cases/cancel-invite owns pending invite cancellation and token invalidation.
  • features/organization/user-management/use-cases/manage-user-roles owns assigning and removing user roles.
  • features/organization/role-management owns role CRUD and permission catalog UI.

New organization server actions should return ServerActionState and convert Zod failures through ServerActionState.fromZodError.

Entry Point Pattern (Idiomatic Next.js)

To prevent server-side dependencies (like database clients or SDKs) from leaking into the client bundle, use explicit entry points:

  1. Slice index.ts: Primary import point for each vertical slice.
  2. server.ts: Optional feature-level curated exports for Server Components and Server Actions.
  3. index.ts: Optional feature-level curated shared/client-safe exports.

Why Vertical Slice Architecture?

Cohesion: Related code lives together - business logic, actions, UI, and tests are co-located.

Discoverability: Find everything related to a feature without navigating multiple folders.

Collaboration: Teams work on different features simultaneously with minimal conflicts.

Flexibility: Features vary in complexity without artificial constraints.

Testing Convention

Use __tests__/ folders for test organization:

  • Slice level (default): features/{name}/{verb-noun}/__tests__/ - Test slice behavior close to implementation
  • Feature level (optional): features/{name}/__tests__/ - Cross-slice integration and shared feature helpers
  • Component level: .test.tsx files go in a __tests__/ folder next to the component
  • Routes and repo-wide checks: the root __tests__/ (__tests__/app/, __tests__/pages/, …)
  • Mocks and assertions: hub/AGENTS.md Tests (mocks reset automatically; mock at the boundary)
  • Focus on slices: Test actions, loaders, hooks, and UI inside the slice where they live
  • Environment: .test.ts runs in node, .test.tsx in jsdom; a .ts test opts into the DOM with // @vitest-environment jsdom. Full rule: hub/AGENTS.md Tests.
  • AAA regions: // Arrange / // Act / // Assert (or // Act & Assert when there is no setup). Skip only when the it is a one-liner with no distinct phases. Full rule: hub/AGENTS.md Tests and .cursor/rules/endatix-hub-rules.mdc.

Layers

Use-Case Layer (optional .use-case.ts)

// Pure business logic, no Next.js dependencies (only when needed)
export const createFormUseCase = async (
  request: CreateFormRequest,
): Promise<Result<string>> => {
  // Business logic here
};

Action Layer (.action.ts)

"use server";

import { revalidatePath } from "next/cache";
import { EndatixApi } from "@/lib/endatix-api";
import { Result } from "@/lib/result";
import { toResult } from "@/lib/result/map-api-result-to-result";

export type DeleteItemResult = Result<string>;

export async function deleteItemAction(
  itemId: string,
): Promise<DeleteItemResult> {
  const session = await requireAccess(); // auth + authorization at the boundary
  const api = new EndatixApi(session.accessToken);
  const apiResult = await api.items.delete(itemId);
  const result = toResult(apiResult, {
    fallbackMessage: "Failed to delete item.",
    logMessage: "Failed to delete item.",
    loggerName: "feature.items",
    mapData: (data) => data.message,
  });

  if (Result.isSuccess(result)) {
    revalidatePath("/(main)/items");
  }

  return result;
}

Action rules:

  • Return Result<T> by default — never swallow failures silently. Use ServerActionState only for FormData / useActionState flows that need Zod field-level errors (not for arbitrary Zod multi-field forms)
  • Authenticate and authorize inside the action, or accept a session already validated by the caller
  • Always map Endatix ApiResult with toResult(...) — do not hand-write Result.error(apiResult.error.message). Pass fallbackMessage, logMessage, and loggerName so expected 403/404/validation stay user-facing and unexpected failures log through TelemetryLogger
  • When the UI result type differs from the API body (e.g. Result<void>), use mapData (e.g. mapData: () => undefined)
  • Revalidate paths only after Result.isSuccess(result)
  • Surface success and failure in the UI with toast or inline feedback; pair actions with a client handler using useTransition when not using useActionState

List pages and tables

  • Drive search, filters, and pagination from URL searchParams (parse in a server page or shared util, e.g. parsePlatformAdminListParams, parseFormsListParams). Coerce Next's repeated keys with firstSearchParam (lib/utils/next-utils); type optional searchParams members as SearchParamValue, not SearchParam — ? plus | undefined is redundant.
  • Use one paged API endpoint with scope/tenant filters rather than two parallel lists sharing the same page param.
  • Client tables update the URL via useListUrlState (single debounced field) or useTableFiltersUrlState (2+ debounced fields, one coalesced commit — see below), plus Select filters calling updateUrl directly, PagedTableFooter / PagedListFooter, and receive unresolved promises from the server page.
  • Paged list wiring (one pattern, issue #1011). The server page parses the request, starts the promise, and passes listKey={listQueryKey(listRequest)} (lib/list-page/list-query-key). Never hand-write a per-list key: listQueryKey covers every field the parser emits, so new filters and sort fields can't be missed.
    • A client shell owns useListUrlState once and renders the toolbar outside PagedListFrame (components/table) and the grid inside it. The grid calls use(promise) and gets updateUrl / isPending as props — it never calls a URL hook itself. Reference: features/platform-admin/list-tenants/ui/tenants-list.tsx. Card-style lists (UsersList, RolesList, PlatformAdminsList) use PagedCardList, which puts the toolbar in the card header and the frame around the rows.
    • When the toolbar and grid are server-composed (other server content between them, e.g. folder cards on /forms), wrap them in PagedListUrlProvider and read usePagedListUrl() in both. Same rule: toolbar outside the frame.
    • Why the key: without it, use(promise) keeps the first page after router.replace until a full reload. Why the toolbar stays outside: a key change remounts everything inside the frame and shows the fallback, so a search input inside it loses focus and typed characters on every debounced commit. A page change shows the skeleton; that is intended.
    • Row identity. Every TanStack grid sets getRowId to the entity id. Index-keyed rows let stateful cells (the submissions status dropdown, answers) keep the previous page's values — that was the submissions half of #1011.
    • Submissions key a server section: PagedListFrame wraps the async SubmissionListSection (it keeps the post-fetch redirect() on the server), and SubmissionsWithFilters hands isPending to its DataTable for skeleton rows while the next page is in flight.
    • Data-list items keep Suspense inside the client details section.
    • Height while loading. Whoever renders the pending skeleton holds the list at its last rendered height with useHeldHeight(isPending): PagedListFrame for card lists and forms, DataTableGrid and the submissions DataTable for their row area (the footer sits below it). The skeleton can't shrink the page or move the pager while the next page loads. When rows render the list takes its natural height, so a filter with one result shows one row, every time, and a shorter last page is shorter. One loading look: while a page, filter, or sort change is in flight (isPending from the shell's useListUrlState), the rows become skeleton rows and everything else stays mounted — no toolbar "Updating…" text, no dimming. Table lists pass isPending to DataTableGrid (or the submissions DataTable), which keeps the column header (and its filters) and renders DataTableSkeletonRows. Card lists pass it to PagedCardList, and PagedListFrame swaps in its fallback; under PagedListUrlProvider (forms) the frame picks it up on its own. An empty list that is reloading shows skeleton rows too (pendingRowCount), not its empty state. Every frame fallback is a *Skeleton component. Spinners are for action buttons (grant, save, delete), never for list rows. The conformance test enforces the fallback names, isPending on every DataTableGrid, and no "Updating…" text.
    • Tests, in three layers. (1) __tests__/app/paged-list-conformance.test.ts fails the build when a TanStack grid lacks getRowId, a paged page skips listQueryKey without an allow-listed reason, or a key is hand-rolled. (2) __tests__/pages/paged-list-shells.test.tsx renders each shell, changes listKey, and checks new rows appear while the search input keeps its value and focus — add a new shell to its table. (3) e2e/tests/lists/list-paging.spec.ts pages every list in a real browser and compares the rows with a fresh render of the same URL — add a new list to its lists array.
  • Files in and out. CSV text is read and written by lib/utils/csv.ts (readCsvRecords, readCsvRows with a stray-quote mark per row, writeCsv), server-safe and with no React. Saving a file is lib/utils/files-download.ts: downloadFromUrl for a file a route builds, downloadText for one the browser builds. The rendered control is components/common/download-menu.tsx. A slice keeps only its own columns and copy (reference: features/audience/generate-links/personal-links.ts, features/audience/import-csv/csv-preview.ts). Rules: DESIGN.md §6 "Hand over a file".
  • Shared helpers live in lib/list-page/ (parse-paged-search-params, list-query-key, table-return-to). URL writers and the frame (useListUrlState, useTableFiltersUrlState, PagedListUrlProvider, PagedListFrame, PagedCardList) live in components/table/.
  • List-table chrome (surface, empty state, header/row/cell class helpers, search input, toolbar, paged footer, BackToTableButton, useListUrlState, useTableFiltersUrlState) lives in components/table/. DataTableToolbar is one row: filters scroll, actions stay pinned. Keep the ShadCN primitive (Table, TableRow, …) in components/ui/table.tsx.
  • Do not hand-roll the parts every list table shares — reach for components/table first:
    • DataTableGrid renders the TanStack header/body; DataTableSkeleton is its loading twin (pass columns, override a column's cell only where the placeholder shape differs).
    • useListTableState(urlState, updateUrl) derives sorting, created, modified, and onSortingChange from the URL.
    • auditDateColumns({ created, modified, updateUrl }) is the Created / Modified pair; spread it into the column list rather than redefining the headers and date filters.
    • ColumnMeta.cellClassName / headerClassName are declared once in components/table/data-table-column-meta.ts. Import that module for the class-name fields; a feature types.ts augments ColumnMeta only with its own fields.
  • Two or more debounced filter fields on the same table must share one useTableFiltersUrlState(keys) call (components/table), not one useListUrlState/useUrlSearchParamsUpdater per field — independent debounced writers race (each rebuilds the URL from its own stale searchParams snapshot) and can silently drop each other's change. keys must be a module-level constant array.
  • Detail → list back navigation restores the list's paging/filters via session-scoped storage, not a URL param — see AGENTS.md "Detail → list back navigation" and lib/list-page/table-return-to.
  • Hub list URL key is search; map it to the API query field (query, filter, etc.) inside the slice parser. Do not introduce a parallel q param.
  • The API-client side of a paged list (request type, exported buildList<Entity>Endpoint, list() returning a normalized page, a separate bounded listAll() for pickers, which don't support lazy loading. Prefer list with paging where possible) has exactly one documented shape — see AGENTS.md "Paged list sort and calendar From/To → The one list-client shape" and its reference implementation lib/endatix-api/themes/.
  • Wired this way: admin/tenants, admin/platform-admins, admin/signup-requests, settings/organization/users, settings/organization/roles, forms, forms/folders/[folderSlug], data-lists (DataListsBrowser), and forms/[formId]/submissions (server section).
  • Submissions list (/forms/[formId]/submissions): loader + page-load outcome live in features/submissions/list-submissions/ (not in app/). URL parse helpers stay in list-submission-query/. See AGENTS.md "Page-load outcomes".
  • Submissions grid on reporting data (features/submissions/browse-submissions/): the grid that will replace list-submissions, forked rather than layered: no shared internals with list-submissions or ui/table, only components/table chrome. It calls rememberSubmissionListReturnTo from list-submission-query because the submission page's Back reads that store (it keeps filters and paging, not columns). The page picks it when reportingExportFlag() is on, and SubmissionGridSection renders the legacy grid it is given when GET forms/{formId}/reporting/columns answers 404. It owns its API client (lib/endatix-api/reporting/submission-grid.ts), URL state (submission-grid-url.ts: same keys as the legacy list plus columns, the visible answer keys), columns (submission-grid-columns.ts; answer column ids are answer:<key>, metadata ids match the API's saved-view MetadataKeys) and the readiness banner (grid-readiness.ts). SubmissionGridShell owns one useTableFiltersUrlState writer through SubmissionGridUrlProvider; the toolbar and banner sit outside the keyed PagedListFrame, and the rows are a server section inside it so the out-of-range redirect stays on the server. Rows come from SubmissionGrid.listRows (GET forms/{formId}/reporting/submissions with columns=, at most 500 keys); an API without that endpoint (404) gets rows built from GET forms/{formId}/submissions instead. Answer filters (submission-grid-answer-filters.ts) are URL keys answer.{contains|from|to}.{key} (the key is the whole rest, so . and : in schema keys survive) and go to the API as repeated answer={operator}:{encodeURIComponent(key)}:{value}. Only columns with a filterType get a header filter (text: contains, date/datetime: From/To; searchable: false and submitter columns get none); filters the catalog cannot apply are dropped by the canonical redirect. The contains keys join the metadata text keys in the one debounced writer. Rows with answer filters never fall back to the submissions list; the rows section shows answerFilterCoverage as a banner with Prepare / Include in-progress, and a 409 reporting_not_prepared reads the page again without answer filters and says so. The form's shared view (GET/PUT forms/{formId}/reporting/submission-view, submission-grid-view.ts) fills in columns, page size and sort wherever the URL says nothing, so URL params still win; metadata visibility and order and column widths (resizable through components/table/column-resize-handle.tsx) live in SubmissionGridViewProvider until a form editor saves with saveSubmissionGridViewAction (a 409 shows inline and reloads the view). While no view is saved, the first load moves the legacy grid's ehx_submissions_column_* localStorage keys into the unsaved view once (submission-grid-view-migration.ts) and never reads them again.
  • Audience (/forms/[formId]/audience): behind personalizationFlag (FLAG_PERSONALIZATION / PostHog key personalization, default off). Every server entry goes through requireAudienceAccess(permission) / requireAudienceApi(permission) (features/audience/shared/): requireAllPermissions for Hub access plus forms.view (page) or forms.edit / tenant.settings.manage (writes), then the flag, then an EndatixApi. Writes add revalidation in withHubAudienceApi. The page uses the returned checkPermission for capabilities that must not redirect: Tenant.ManageSettings (match-key editor) and Forms.Edit (canEdit: add, edit, rename and remove controls on People and Properties, and Audience coverage). A miss hides the control, Import CSV and Personalized links included (features/audience/import-csv/, features/audience/generate-links/, gated on forms.edit like the other writes). Personalized links go to the participants the list's search keeps: the panel counts them with GET links/preview?search= on every open, and Generate makes one POST links/batches whose links carry their identifiers, because a token exists only in that one answer (personal-links.ts writes the file). An API without those routes answers a bare 404 (its own 404s are problem+json, shared/missing-route.ts); the panel then keeps the old POST links for everyone, reading every page of people first for the identifiers (issue-links.server.ts, removed with that API's support), and a search on it says it needs a newer API. /share/{formId}/a/{token} is a page that only renders on GET, so a mail scanner or link preview changes nothing; once it has mounted in a browser it redeems by POST (features/public-form/open-personal-link/, a server action, no Hub session), stores the submission token in the form cookie and replaces itself with /share/{formId}. A missing or invalid token shows "This link is not valid.", anything else offers Try again, and the page sends no Referer (endatix-hub#1211). Add property and the import wizard share features/audience/property-draft.ts: choicesError (the API's choice-key rule: at least one, none blank, no |, distinct by ordinal comparison), toCreateInput, isChoiceType and CHOICE_VALUE_SEPARATOR. Import CSV is a staged import (features/audience/import-csv/). The File step states the host's limits from the settings' importLimits: 100,000 rows and 50 MB with background jobs, 5,000 rows and 20 MB inline. It also refuses a file with a 1,000-row chunk over the server action body limit. A column named like no Property is queued to create one; the New properties step (only when one is queued) suggests its name and type from the whole file (type-suggestion.ts, new-properties.ts), warns on names the saved form already writes through checkPropertyNamesAction (the survey-lint collision rule, run on the server so the linter stays out of the browser), and its Continue creates the Properties with createAudiencePropertyAction, then starts the import. Without a queued column, leaving Columns starts the import. The browser sends the chunks one at a time, because Next runs one server action at a time per client, so parallel sends would only queue. The send action checks the chunk's index and size again as a guard, never splitting a chunk: the API takes exactly 1,000 rows per index. While the import checks or writes, the panel reads it every 2 s, paused while the tab is hidden (use-audience-import.hook.ts). This is the one deliberate exception to the no-polling freshness rule (AGENTS.md "Freshness"), and the Hub reads the import, never GET jobs/{id}. On mount the panel reads …/imports/current and opens on an unfinished or stopped import. An upload cut off by a closed tab continues from a re-chosen file, matched by name, row count and header row only: a different file that matches all three would be staged as the same file, and the Check's counts before confirm are where the scripter sees it. A stopped write offers Resume, or Discard, which cancels it before a new import starts. The one-request api.audience.importCsv stays only for e2e seeding. The form itself comes from the shared loadWorkspaceForm. The page only composes; load and paging live in features/audience/get-audience-page/. Coverage (the Used in form and Filled in columns, the People marks) is for form editors only: loadAudienceUsage reads the saved definition alongside the variables read (forms.edit in the API), works out on the server what the form reads (registerCoreSurveyModels first), and sends the browser only the Property names it reads and the counts. People are paged with ?page= and ?pageSize=, filtered with ?test=, and searched by identifier with ?search=; Back from a participant restores all of them (audience-return-to.ts). Test links (Copy test link, Download test links): testLinkAccess (features/audience/shared/test-link-access.server.ts) gives both to a reader with forms.test and forms.edit once a probe finds the API's POST …/audience/test-links route. The probe sends membershipIds: [], which that route's validator refuses with 400 before it reads anything, so it opens nothing; an older API answers 404. On an older API, Download test links is hidden and Copy test link keeps the open-then-mint path (copy-test-link/create-test-link.action.ts), which also needs a submissions grant and deletes the test submission when the mint fails. A 404 from the route itself is shown as is. Download test links ticks Test participants in the People table (features/audience/ui/use-participant-selection.hook.ts; ticks outlive a page or filter change, Remove drops one, Clear selection drops all) and writes test-links-<form>-<date>.csv (download-test-links/). The route's limits are in shared/test-link-limits.ts. Create/edit panel wiring: see PanelForm state below. Auth shape: AGENTS.md "Server Actions".
PanelForm state (create / edit overlay wiring)

A create or edit overlay has three layers. Do not invent a fourth (no React Hook Form, no feature-wide form context).

Multi-step overlays keep the same three layers. The slice hook also owns the step, and the panel hands PanelForm the current step's labels, onBack and cancelable (DESIGN.md §6 Create / edit overlay). Split that hook by concern (the input read, the choices, the write, the step) and compose them in one exported hook. When the write's result is the outcome step (an import's counts and rejected rows), the slice hook keeps the result instead of toasting through the feature's mutation hook. Reference: features/audience/import-csv/use-import-csv.hook.ts, with pure parsing and suggestions in csv-preview.ts and column-map.ts beside it. A write that runs on the server after the panel's request (a staged import) keeps its state in a hook of its own, use-audience-import.hook.ts, and the step follows that state. create-tenant-panel.tsx predates this and keeps its state in the panel.

Layer Owns Lives
Chrome Header, body scroll, footer, failure strip, Cancel (or Back) + pending submit components/common/panel-form.tsx (PanelForm)
Feature write lifecycle pending / error, toast + router.refresh on success, failure stays in the overlay so values are kept One shared hook per feature that writes this way — audience: useAudienceMutation
Slice wiring Draft fields, open/reset/lock-while-pending, submit → action + onSuccess: close Optional use-{noun}.hook.ts in the slice — reference: features/audience/create-person/use-add-person.hook.ts

Shared panel primitives (same feature, several slices) stay next to the mutation hook, not inside one slice: audience uses usePanelOpen, useValueMap, useRowOverlay in use-panel-state.hook.ts.

One call site is fine. The slice hook is not extracted for reuse; it keeps the panel file as copy + PanelForm + field sections, and keeps DESIGN overlay rules (reset on open, refuse dismiss while pending, error in-panel) out of JSX. Mirror it for the next create panel (e.g. add property); do not merge create and edit into one generic hook — edit often differs (row-keyed open, save only changed fields: useEditPerson).

When to colocate instead: a tiny private useConfirm… used only by an AlertDialog in the same file is OK (confirm-delete-dialog.tsx). Prefer a use-*.hook.ts file when the return value spreads into PanelForm and a sibling edit hook already exists.

Forms list scope model (/forms)

Root /forms supports three URL-driven modes (shareable, bookmarkable):

Mode URL API scope Folder shortcuts Info alert
Unassigned browse (default) /forms unassignedOnly Show No
All forms browse /forms?browse=all tenant-wide Show No
Global search/filter ?search=… or status=… or visibility=… tenant-wide Hide Yes (Alert variant="info")
Folder browse /forms/folders/[slug] folderId N/A No

Parsing and helpers live in features/forms/list-forms/utils.ts (parseFormsListParams, resolveRootFormsViewMode, shouldHideFolderShortcuts, isTenantWideFormsList). Breadcrumb dropdown options (Unassigned / All forms / folders) are built in features/folders/view-forms-header/build-forms-breadcrumb-model.server.ts with browse from header searchParams.

Follow-up PRs (not Phase 1):

  • Phase 2 — Sorting: OSS sortBy / sortDir on GET /forms; Hub toolbar Select + URL params.
  • Phase 3 — Layout toggle: layout=cards|table with compact table view for large lists.
  • Phase 4 — Pinning: Backend UserFormPin + pinned strip above list.
  • Phase 5 — Tasks: Assigned/review sections when collaboration exists.
Data lists list + items (/data-lists)

Hub management grids follow the same URL + chrome pattern as forms/users. Parsing stays in the owning slice (not a new query slice).

Surface Slice Hub URL API
List features/data-lists/view-lists/ search, hasLocale (single BCP-47 code), page, pageSize, action=create GET /data-lists (search, hasLocale)
Creator picker features/data-lists/search-data-lists-for-picker/ (property grid; no Hub URL) Same GET /data-lists via getDataListsPageResult
Detail items features/data-lists/view-list-details/ search, page, pageSize, action=replace GET /data-lists/{id}/items (query) + GET /data-lists/{id}?includeItems=false
  • List → detail "Back to Data Lists" restores filters/paging via BackToTableButton (tableKey: "data-lists"), not a URL param — see AGENTS.md "Detail → list back navigation".
  • Replacing items resets the items grid's page/search (stale values from before the replace would otherwise point past the new item set).
  • Items are GET-only in Hub. Authors edit via file replace (replace-items), not per-item CRUD.
  • Stream the paged promise from the App Router page into the client table (use() inside PagedListFrame with listQueryKey). Footer lives inside DataTableSurface. Items keep Suspense inside the details client section.
  • Creator Translations CSV wrap lives in lib/survey-features/data-lists/ (compound keys + SurveyJS CSV merge). Hub actions stay in features/data-lists/translations/ (uploadTranslationsCsvAction, getDataListTranslationCatalogAction).
  • Creator catalog picker vs bound-question items: Hub session GET /data-lists (search-data-lists-for-picker) vs public item search (survey-bindings.ts + searchDataListChoices). Do not load the full catalog at Creator init.

Layout and parallel-route slots (*-slot.tsx)

  • A component mounted by app/ layouts or @slot route pages lives in its feature slice as {feature}-slot.tsx, not inline in the app/ file. app/ stays orchestration: resolve session/params, render the slot. References: features/folders/view-forms-header/forms-folder-header-slot.tsx, features/platform-admin/assume-tenant/ui/support-access-banner-slot.tsx.
  • A slot mounted by a layout must be synchronous and own its Suspense boundaries internally. await in the slot (or in the layout) blocks every route under that layout — decide whether to render from data already in hand, and stream only the parts that need a fetch.
  • Slots call *.server.ts loaders; they do not construct EndatixApi themselves.

Server Helper Layer (*.server.ts)

  • Server-only read-model loaders and helper orchestration
  • Can live inside a slice (preferred) or feature root for shared server-only code
  • Keep plain constants (logger names, revalidation paths) in a sibling *.constants.ts, not in the server-only module. Importing one string otherwise drags @/auth into every consumer — including client components and unit tests. Example: features/platform-admin/tenant-management.constants.ts.
  • A list that can fail while the rest of the payload succeeds is T[] | undefined on the view model (undefined failed, [] loaded empty). Do not add a *Loaded flag or a second mapped copy. Full rule: AGENTS.md “Loaded list vs failed list”. Reference: FormsHeaderData.folders.

Data Fetching and State Management

Data Fetching (Server to Client)

For optimal performance and UX, prefer streaming promises from Server Components to Client Components. Use the React use() hook to unwrap these promises. This prevents blocking the initial render while data is being fetched.

Recommended Pattern:

  1. Server Page/Component: Initiate the fetch and pass the promise (not the awaited result).
  2. Context Provider (Client): Accept the promise as a prop and use use(promise) to resolve it.
// features/my-feature/ui/my-feature.context.tsx (Client Component)
'use client';
import { createContext, use, useMemo } from 'react';

export function MyFeatureClientProvider({ children, dataPromise }) {
  const data = use(dataPromise); // Resolves via React Suspense
  const value = useMemo(() => ({ data }), [data]);
  return <MyContext value={value}>{children}</MyContext>;
}

// features/my-feature/ui/my-feature.provider.tsx (Server Component)
import { MyFeatureClientProvider } from './my-feature.context';

export function MyFeatureProvider({ children }) {
  const dataPromise = getMyData(); // Start fetching locally on server
  return (
    <MyFeatureClientProvider dataPromise={dataPromise}>
      {children}
    </MyFeatureClientProvider>
  );
}

// app/(main)/my-feature/page.tsx (Server Page)
export default async function Page() {
  return (
    <MyFeatureProvider>
      <MyFeatureComponent />
    </MyFeatureProvider>
  );
}

State Management & Interactions

Combine use() with useReducer and useOptimistic for features with complex interactions (like forms or AI assistants).

// Example in FormAssistantProvider
export function FeatureProvider({ children, initialDataPromise }) {
  const initialData = initialDataPromise ? use(initialDataPromise) : defaultState;
  const [state, dispatch] = useReducer(reducer, initialData);
  const [optimisticState, setOptimisticState] = useOptimistic(state, (s, u) => ({ ...s, ...u }));

  const handleAction = async (payload) => {
    setOptimisticState({ isPending: true }); // Immediate UI feedback
    const result = await myServerAction(payload); // Next.js Server Action
    dispatch({ type: 'UPDATE', payload: result });
  };

  const value = useMemo(() => ({ state: optimisticState, handleAction }), [optimisticState]);
  return <MyContext value={value}>{children}</MyContext>;
}

Best Practices

  • Thin Providers: Keep context providers focused on data sharing and state management.
  • Custom Hooks: Expose context via hooks (e.g., useAssetStorage) to provide a clean API.
  • Server-Side Providers: Use Server Component wrappers (like AssetStorageProvider) to orchestrate configuration and data fetching on the server.
  • Zero-Latency Orchestration: By using Server Components for providers, we can pass unresolved promises to the client. This allows the page to start streaming immediately while SAS tokens or other data are generated on the server, avoiding extra HTTP round-trips from the client.
  • Reference Implementations:
    • asset-storage.context.tsx: Clean configuration streaming and client-side state.
    • asset-storage.provider.tsx: Server Component orchestration.
    • form-assistant.context.tsx: Complex state with optimistic updates and reducer.

More on this in the Next.js Data Fetching Documentation.

Naming Conventions

Files & Folders

  • kebab-case: create-form.action.ts, view-forms-header/
  • Functions: camelCase → createFormUseCase
  • Classes/Types: PascalCase → CreateFormRequest

Files

  • Use-cases: {verb-noun}.use-case.ts (optional)
  • Actions: {verb-noun}.action.ts
  • Server-only helpers: {verb-noun}.server.ts
  • Hooks: use-{noun}.hook.ts
  • Components: {feature-name}-{purpose}.tsx

App Folder Structure

app/
├── (main)/                    # Main application routes
│   ├── dashboard/
│   ├── forms/
│   ├── settings/
│   └── ...
├── (public)/                  # Public routes
│   ├── share/
│   └── ...
├── api/                       # API routes
├── globals.css
└── layout.tsx

Keep /app focused on Next.js routing only - no business logic.

Package Structure Guidelines

When to Create a Package

  • Multiple consumers - Used by multiple apps/projects
  • Independent versioning - Can be versioned separately
  • Clear API boundaries - Well-defined public interface
  • No framework coupling - Not tied to Next.js/React

Package.json Structure

{
  "name": "@endatix/api-client",
  "version": "1.0.0",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "exports": {
    ".": "./dist/index.js",
    "./types": "./dist/types.js"
  },
  "peerDependencies": {
    "react": "^19.0.0"
  }
}

Workspace Configuration

// package.json (root)
{
  "workspaces": ["apps/*", "packages/*"]
}

Guidelines

  1. Start with a slice - Create {verb-noun}/ and co-locate action, UI, tests, and exports
  2. Keep actions thin - Wrap API calls and orchestration, move complex logic to helpers/use-case files
  3. Co-locate related code - Keep slice internals inside the slice folder
  4. Use slice exports first - Import from features/{name}/{verb-noun} before feature-level barrels
  5. Move to lib/ when reusable - Extract to lib/ when used across projects
  6. Move to packages/ when publishable - Extract to packages/ when ready for npm
  7. Document Architecture Updates - Any new architectural items, patterns, or cross-cutting features discovered during planning or development must be documented in a dedicated ARCHITECTURE.md file in the root of the project.

Package Evolution Path

Current Structure

hub/
├── lib/                    # Internal utilities
│   ├── endatix-api/       # API client (internal)
│   ├── localization/      # Multi-lingual domain (catalog, submission locale, future i18n)
│   ├── survey-js/         # Shared SurveyJS + Creator types/vocab (vendor first, Hub supersets)
│   └── utils/             # Shared utilities
├── features/              # App-specific features
│   ├── config/            # App configuration
│   ├── auth/              # Auth business logic
│   └── forms/             # Form management
└── packages/              # Future workspace packages

lib/localization domain

Use lib/localization for cross-feature multi-lingual infrastructure, organized as slices:

Slice Responsibility
catalog/ SurveyJS ↔ owned catalog vocabulary (default ↔ "" / runtime default code), culture-code normalize/validate, toCatalogLocales, display names
submission-locale/ Submission metadata language helpers shared by submissions UI and pdf-export
routing/ (future) Locale-prefixed Hub routes / middleware
messages/ (future) Hub UI copy catalogs (e.g. next-intl)

Keep selection policy in the owning feature (e.g. public-form language picker priority: preselected → localStorage → survey model → browser). Do not move feature-specific preference storage or init order into lib/.

Import from @/lib/localization (root barrel) or a slice path such as @/lib/localization/catalog.

lib/personalization domain

Audience values inside a form. It is cross-feature: the public form, the designer, the Audience tab and Diagnostics all read it, so none of them owns it. It has two entries, one per runtime:

Path Runtime Owns Consumers
apply-personalization.ts respondent Snapshot values onto a survey model features/public-form, features/audience/preview-as
snapshot-source.ts any A Snapshot's source label Test inspector, Submission Details
to-survey-presets.ts any Wire presets → ISurveyVariablePresets, shared by both entries authoring/, coverage/
coverage/ (@/lib/personalization/coverage) server-safe What the form reads (audienceUsage, lintedUsage), coverage verdicts (usageVerdicts), the Audience findings (reviewAudience) and their types load-audience-usage.server.ts, the Audience tab, the designer (lazily), Diagnostics (types, through the authoring re-export)
authoring/ (@/lib/personalization/authoring) client The Creator binding (bindAudienceToCreator), Preview as, Find participant, the designer's save body, applying a fix to the canvas features/forms/ui/editor, the design page (types)

coverage/ is survey-core only and imports nothing from authoring/; no-client-imports.test.ts fails on a React, lucide, Creator or "use client" import there, and on any import outside survey-core, survey-lint, the API types, to-survey-presets and coverage/ itself. Server files that need an authoring/ contract type use import type. Import only through the two index.ts files (eslint no-restricted-imports), with one exception: the coverage verdicts (usageVerdicts) have their own browser entry, @/lib/personalization/coverage/audience-usage, and the coverage index exports only their types. The index brings survey-core/linter, and Turbopack keeps it in a client bundle even when only the verdicts are used (+135 KB on the Audience tab).

lib/survey-features/survey-lint domain

The one home for definition analysis: checks that need only the survey JSON. Server-safe and UI-free: survey-core and survey-core/linter only, nothing from lib/personalization, form-diagnostics or features/ (pinned by __tests__/no-client-imports.test.ts). It holds the Hub's only lintSurvey call (runLint). Import from @/lib/survey-features/survey-lint.

What goes where:

  • A check that needs only the definition (unknown references, collisions, undeclared roots, a SurveyJS rule subset) → survey-lint.
  • A check that needs Audience data (Properties, participants, counts) → lib/personalization/coverage, built on survey-lint. To learn which Properties a form reads, call audienceUsage; do not scan again.
  • Rendering findings → lib/survey-features/form-diagnostics/ui/.
  • Applying a lint fix to the Creator canvas is Creator-side, never survey-lint: it stays in personalization/authoring while the binding is its only caller.

lib/survey-js domain

Shared SurveyJS + Creator types/vocab only — no behavior, no choice helpers. Vendor types first; Hub unions close vendor strings.

Path Owns
lib/survey-js/creator/tabs.ts Creator tab ids — ENDATIX_CREATOR_TAB (built-ins + Hub plugin tabs) and canonicalizeCreatorTabId
lib/survey-js/creator/tab-url.ts ?tab= slug ↔ tab id (design → designer; Design omits the param)

Vocab only here. Creator tab/JSON behavior: lib/survey-features/survey-design/ (ui/use-creator-tab-url.ts, ui/use-creator-json.ts, use-cases/). Do not re-declare vendor types (Question, SurveyCreatorModel). Rules: AGENTS.md (SurveyJS domain).

Import from @/lib/survey-js.

Target Monorepo Structure

endatix-saas/
├── apps/
│   └── hub/               # Next.js application
└──packages/
    ├── @endatix/api-client/    # API client package
    ├── @endatix/ui/            # UI components package
    ├── @endatix/config/        # Configuration package
    └── @endatix/utils/         # Utilities package

Migration Steps

  1. lib/endatix-api/ → packages/@endatix/api-client/
    • Includes shared list wire helpers (SortRequest, DateRangeFilter, list-query.ts, query-params.ts, paged-response.ts). Keep new sort/range contract code in lib/endatix-api/shared until that extract; do not place it under features/ or lib/date-utils (formatters). lib/endatix-api/themes/ is the reference module shape for the extract.
  2. lib/utils/ → packages/@endatix/utils/
  3. features/config/ → packages/@endatix/config/
  4. features/auth/ → Keep in apps/hub/features/auth/ (app-specific)