diff --git a/.moon/workspace.yml b/.moon/workspace.yml index 0d1183f1..3f58a15b 100644 --- a/.moon/workspace.yml +++ b/.moon/workspace.yml @@ -27,6 +27,7 @@ projects: logger: "packages/shared/logger" utils: "packages/shared/utils" mocks: "packages/shared/mocks" + local-ai: "packages/shared/local-ai" # Backend Packages backend-chat: "packages/backend/chat" diff --git a/apps/backend/local-stack/README.md b/apps/backend/local-stack/README.md index 156e6379..9a8cf140 100644 --- a/apps/backend/local-stack/README.md +++ b/apps/backend/local-stack/README.md @@ -5,10 +5,15 @@ speech-to-text engines plus an optional web client — with **two commands and no Python, no CUDA toolkit install, no model hunting, and no source build**. ``` -cp .env.example .env +bun run stack init docker compose up -d ``` +`stack init` detects your hardware, picks a backend and model tier, shows +the full download plan, and writes `.env` — it is the only way `.env` is +created (there is no `.env.example` copy step, so nothing ever asks you to +overwrite a hand-edited file). + This is the publishable topology (C-390): one `compose.yaml` whose **profiles select modalities**, and whose **override files select the hardware backend**. All variation lives in `.env`; the runtime command never @@ -22,27 +27,23 @@ changes. ```bash git clone https://github.com/BearlySleeping/aikami.git - cd aikami/apps/backend/local-stack + cd aikami ``` -2. **Pick your hardware.** Copy the example env and set the two variables - that matter (everything else has a working default): +2. **Detect your hardware and generate the `.env` (C-391).** The wizard + probes your GPU, RAM, disk, and container runtime, recommends a backend + and model tier, shows the full download plan, and writes `.env` — no + manual editing, no needing to know what CUDA 12 vs 13 means: ```bash - cp .env.example .env - # .env: - # COMPOSE_PROFILES=text,image,voice,stt - # COMPOSE_FILE=compose.yaml:compose.cpu.yaml + bun run stack init ``` - | Backend | `.env` COMPOSE_FILE | When | - |---|---|---| - | CPU | `compose.yaml:compose.cpu.yaml` | Any machine; slow but works everywhere | - | NVIDIA CUDA | `compose.yaml:compose.cuda.yaml` | Needs the NVIDIA Container Toolkit | - | AMD ROCm | `compose.yaml:compose.rocm.yaml` | linux/amd64 only; image engine uses Vulkan | - | Vulkan (universal GPU) | `compose.yaml:compose.vulkan.yaml` | AMD non-ROCm, Intel Arc, iGPUs | - | Intel / SYCL | `compose.yaml:compose.intel.yaml` | Intel Arc / recent integrated graphics | - | Moore Threads MUSA | `compose.yaml:compose.musa.yaml` | MUSA GPUs | + Non-interactive (CI / power users): + + ```bash + bun run stack init --yes --backend cuda --modalities text,voice + ``` 3. **Start the stack.** @@ -74,6 +75,51 @@ changes. --- +## What `stack init` does + +The wizard (C-391) is a thin CLI over the portable planning core in +`packages/shared/local-ai` (`@aikami/local-ai`): + +- **Detection** — probes `nvidia-smi`, `rocm-smi`, `vulkaninfo`, `/proc/meminfo`, + `sysctl`, `docker info` / `podman info`, and the target volume's free disk. + Every probe is capped at 1 s and non-fatal; with no GPU tooling it reports + `cpu` and still writes a valid `.env`. +- **Recommendation** — maps the profile + your chosen modalities onto + `models.manifest.json` entries using the tier table. Usable VRAM is 70% of + reported for dedicated GPUs and 50% of total memory for unified-memory + systems; a model is only selected when its size fits usable memory, and the + largest tier that fits comfortably wins. +- **Plan first** — backend, per-model sizes with one-line rationale, total + download, free disk, licences, and bound ports are printed before anything + is written. Declining writes nothing; a re-run diffs the existing `.env` and + requires confirmation before overwriting (the old file is backed up). +- **Flags** — `--yes`, `--backend ` + (default `auto` = detect from the hardware profile, exactly like omitting the + flag; only an explicit non-auto value overrides planning), + `--modalities `, `--tier `, `--json` (full profile + + plan as a schema-valid document), `--fetch` (chain C-390's fetcher), and + `--env-path` / `--manifest-path` for scripts. + +## Backend reference + +| Backend | `.env` COMPOSE_FILE | When | +|---|---|---| +| CPU | `compose.yaml:compose.cpu.yaml` | Any machine; slow but works everywhere | +| NVIDIA CUDA | `compose.yaml:compose.cuda.yaml` | Needs the NVIDIA Container Toolkit | +| AMD ROCm | `compose.yaml:compose.rocm.yaml` | linux/amd64 only; image engine uses Vulkan | +| Vulkan (universal GPU) | `compose.yaml:compose.vulkan.yaml` | AMD non-ROCm, Intel Arc, iGPUs | +| Intel / SYCL | `compose.yaml:compose.intel.yaml` | Intel Arc / recent integrated graphics | +| Moore Threads MUSA | `compose.yaml:compose.musa.yaml` | MUSA GPUs | +| Metal (macOS) | `compose.yaml` (native engines) | Apple Silicon — no GPU passthrough | + +If you prefer to hand-edit, the variables that matter are +`COMPOSE_PROFILES` and `COMPOSE_FILE` (everything else has a working +default). `stack init` writes exactly these two plus the model paths it +selected. To change hardware after `init`, re-run it (it diffs and asks) or +edit `.env` directly. + +--- + ## Modalities (profiles) `COMPOSE_PROFILES` is a comma-separated list of: diff --git a/apps/backend/local-stack/moon.yml b/apps/backend/local-stack/moon.yml index ed31d08a..556ef95f 100644 --- a/apps/backend/local-stack/moon.yml +++ b/apps/backend/local-stack/moon.yml @@ -17,6 +17,9 @@ project: dependsOn: - 'constants' + - 'local-ai' + - 'schemas' + - 'types' fileGroups: configs: @@ -81,3 +84,12 @@ tasks: options: cache: false runInCI: false + + init: + command: 'bun stack/init.ts' + options: + cache: false + runInCI: false + # CLI-only interactive/flag-driven command; never part of `moon ci`. + # NB: moon 2.4.6 refuses `moon run` for runInCI:false tasks, so the + # root `stack` script drives the package.json `init` script directly. diff --git a/apps/backend/local-stack/package.json b/apps/backend/local-stack/package.json index 0ca9bb46..20ede994 100644 --- a/apps/backend/local-stack/package.json +++ b/apps/backend/local-stack/package.json @@ -13,6 +13,7 @@ "up:cpu": "docker compose -f compose.yaml -f compose.cpu.yaml up -d", "up:cuda": "docker compose -f compose.yaml -f compose.cuda.yaml up -d", "fetch-models": "bun stack/fetch_models.ts", + "init": "bun stack/init.ts", "down": "docker compose down", "logs": "docker compose logs -f", "build": "bun run build:client && docker compose build", @@ -25,7 +26,10 @@ "run:native-llm": "bash bin/run-native-llm.sh" }, "dependencies": { - "@aikami/constants": "workspace:*" + "@aikami/constants": "workspace:*", + "@aikami/local-ai": "workspace:*", + "@aikami/schemas": "workspace:*", + "@aikami/types": "workspace:*" }, "devDependencies": {} } diff --git a/apps/backend/local-stack/scripts/check.sh b/apps/backend/local-stack/scripts/check.sh index 01c8d242..eac11529 100755 --- a/apps/backend/local-stack/scripts/check.sh +++ b/apps/backend/local-stack/scripts/check.sh @@ -244,6 +244,45 @@ if [ "${LOCAL_STACK_LIVE:-0}" = "1" ]; then | sort -u) fi +# ── C-391 `stack init` (AC-8, AC-10) ────────────────────────────────── +# AC-8: `init --yes` completes without prompting in a non-TTY invocation +# and writes a valid .env. AC-10: the generated .env renders with +# `docker compose config` (the full boot happens in CI / LOCAL_STACK_LIVE). +echo "== stack init (C-391) ==" +# Private temp dir: keeps the generated .env and the init log out of the +# predictable /tmp path, and the EXIT trap removes them on every exit path +# (success, failure, or set -e abort) without leaving a world-readable log. +INIT_TMP="$(mktemp -d)" +trap 'rm -rf "${INIT_TMP:-}"' EXIT +INIT_ENV="$INIT_TMP/.env" +INIT_LOG="$INIT_TMP/init.out" +if timeout 60 bun stack/init.ts --yes --no-color --env-path "$INIT_ENV" >"$INIT_LOG" 2>&1; then + ok "AC-8: stack init --yes runs non-interactively (exit 0)" +else + bad "AC-8: stack init --yes failed — see $INIT_LOG" + tail -30 "$INIT_LOG" >&2 +fi +if [ -f "$INIT_ENV" ] && grep -q '^COMPOSE_PROFILES=' "$INIT_ENV" \ + && grep -q '^COMPOSE_FILE=' "$INIT_ENV"; then + ok "AC-8: generated .env carries COMPOSE_PROFILES and COMPOSE_FILE" +else + bad "AC-8: generated .env missing required keys" +fi +if command -v docker >/dev/null 2>&1; then + COMPOSE_LINE="$(grep '^COMPOSE_FILE=' "$INIT_ENV" | cut -d= -f2)" + PROFILES_LINE="$(grep '^COMPOSE_PROFILES=' "$INIT_ENV" | cut -d= -f2)" + # Render the generated configuration from the local-stack project dir, + # loading the generated env file so compose interpolates the same + # variables (model paths, ports) the wizard wrote. + if COMPOSE_FILE="$COMPOSE_LINE" COMPOSE_PROFILES="$PROFILES_LINE" docker compose --env-file "$INIT_ENV" config --quiet 2>/dev/null; then + ok "AC-10: generated .env renders with docker compose config" + else + bad "AC-10: generated .env does not render (docker available)" + fi +else + ok "AC-10: docker unavailable — boot render deferred to CI" +fi + # ── AC-9 contract support: the published client image must serve runtime # config mounts (the two-mount container test lives in the publish # workflow — publish-local-stack.yml — which actually boots the image; diff --git a/apps/backend/local-stack/stack/detect.test.ts b/apps/backend/local-stack/stack/detect.test.ts new file mode 100644 index 00000000..996925c2 --- /dev/null +++ b/apps/backend/local-stack/stack/detect.test.ts @@ -0,0 +1,113 @@ +/** + * apps/backend/local-stack/stack/detect.test.ts + * + * C-391 detection ACs exercised at the local-stack level (evidence files + * named in the contract matrix): AC-1 empty PATH, AC-2 stubbed nvidia-smi, + * AC-12 stubbed docker info. Detection itself lives in @aikami/local-ai; + * these tests drive it with fixture-replay executors through the CLI + * adapter path. + */ + +import { describe, expect, test } from 'bun:test'; +import type { ProbeResult } from '@aikami/local-ai'; +import { + createFixtureExecutor, + detectHardware, + runProbeExecutorContractSuite, +} from '@aikami/local-ai'; +import { probeExecutor } from './probe_executor.ts'; + +const ok = (stdout: string): ProbeResult => ({ ok: true, stdout, stderr: '', exitCode: 0 }); + +describe('AC-1 — detection degrades to CPU without error (empty PATH)', () => { + test('no GPU tooling → gpu.vendor none, containerRuntime none', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 33554432 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.vendor).toBe('none'); + expect(profile.gpuPassthroughReady).toBe(false); + expect(profile.containerRuntime).toBe('none'); + }); +}); + +describe('AC-2 — stubbed nvidia-smi', () => { + test('CUDA 12 driver → nvidia, cudaMajor 12', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok('NVIDIA GeForce RTX 4070, 12282 MiB, 535.104.05\n'), + }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.vendor).toBe('nvidia'); + expect(profile.gpu.vramMb).toBe(12282); + expect(profile.gpu.cudaMajor).toBe(12); + }); + + test('CUDA 13 driver → cudaMajor 13', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok('NVIDIA GeForce RTX 5070, 12282 MiB, 580.00\n'), + }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.cudaMajor).toBe(13); + }); +}); + +describe('AC-12 — stubbed docker info (toolkit absent)', () => { + test('docker info without nvidia runtime → gpuPassthroughReady false', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok('NVIDIA GeForce RTX 4070, 12282 MiB, 535.104.05\n'), + }, + { command: 'docker', args: ['info'], result: ok('Runtimes: runc\n') }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.vendor).toBe('nvidia'); + expect(profile.gpuPassthroughReady).toBe(false); + }); +}); + +describe('AC-0c — shared contract suite against the Bun/CLI adapter', () => { + runProbeExecutorContractSuite({ + label: 'bun/cli', + factory: () => probeExecutor, + // /proc/1/mem is the only universally-denied read on Linux; on other + // platforms the adapter has no deterministic denial and the test is + // skipped (capability-gated in the suite). + permissionDeniedPath: process.platform === 'linux' ? '/proc/1/mem' : undefined, + }); +}); diff --git a/apps/backend/local-stack/stack/env_writer.ts b/apps/backend/local-stack/stack/env_writer.ts new file mode 100644 index 00000000..05ead641 --- /dev/null +++ b/apps/backend/local-stack/stack/env_writer.ts @@ -0,0 +1,223 @@ +/** + * apps/backend/local-stack/stack/env_writer.ts + * + * .env generation for the local stack (C-391). Turns a StackPlan into the + * C-390 .env contract: COMPOSE_PROFILES, COMPOSE_FILE (platform-correct + * separator), TEXT_MODEL / IMAGE_MODEL pointing at the selected manifest + * entries, licence acceptance, and the port table. + * + * Guarantees (Quality Requirements): + * - Atomic: write to a temp file then rename — an interrupted `init` + * leaves the prior .env intact. + * - Re-run safe: diff against the existing file and require confirmation + * before overwrite; a declined re-run leaves the file byte-identical. + * - Backup: the previous file is copied to .env.bak before overwrite. + */ + +import { constants } from 'node:fs'; +import { access, copyFile, readFile, rename, writeFile } from 'node:fs/promises'; +import { basename, dirname, join } from 'node:path'; +import { EMULATOR_PORTS } from '@aikami/constants'; +import type { HardwareProfile, ModelManifest, StackBackend, StackPlan } from '@aikami/local-ai'; + +/** Backend → compose override file (C-390). `metal` has no override. */ +export const BACKEND_COMPOSE_FILE: Readonly> = { + cpu: 'compose.yaml:compose.cpu.yaml', + cuda: 'compose.yaml:compose.cuda.yaml', + rocm: 'compose.yaml:compose.rocm.yaml', + vulkan: 'compose.yaml:compose.vulkan.yaml', + intel: 'compose.yaml:compose.intel.yaml', + musa: 'compose.yaml:compose.musa.yaml', + metal: 'compose.yaml', +} as const; + +/** Separator for COMPOSE_FILE: ':' on POSIX, ';' on Windows (AC-11). */ +export const composeSeparator = (platform: HardwareProfile['platform']): ':' | ';' => + platform === 'win32' ? ';' : ':'; + +/** + * Renders the .env content for a plan + profile. + * + * @returns The full .env text. + */ +export const renderEnv = (options: { + readonly profile: HardwareProfile; + readonly plan: StackPlan; + readonly manifest: ModelManifest; + /** Extra env lines (e.g. TEXT_SERVER_IMAGE for CUDA 13 hosts). */ + readonly extras?: Readonly>; +}): string => { + const { profile, plan, manifest, extras } = options; + const separator = composeSeparator(profile.platform); + const composeFile = BACKEND_COMPOSE_FILE[plan.backend].split(':').join(separator); + const lines: string[] = []; + + lines.push('# Generated by `bun run stack init` (C-391). Edit freely; a re-run shows a diff.'); + lines.push('# Hardware profile: ' + describeProfile(profile)); + lines.push(''); + lines.push('# ── Modality selection ────────────────────────────────────'); + lines.push(`COMPOSE_PROFILES=${plan.modalities.join(',')}`); + lines.push(''); + lines.push('# ── Hardware backend ──────────────────────────────────────'); + lines.push(`COMPOSE_FILE=${composeFile}`); + if (plan.nativeEngines) { + lines.push('# Engines run natively (bin/run-native-*.sh) — only web is containerised.'); + } + lines.push(''); + + // ── Model selection (paths inside the models volume) ────────────────── + // The compose services read TEXT_MODEL / IMAGE_MODEL as file names inside + // the models volume. Resolve from the manifest entries via the plan's + // manifestId → the targetPath the fetcher wrote. + const targetOf = (manifestId: string): string | undefined => + manifest.entries.find((entry) => entry.id === manifestId)?.targetPath; + const textEntry = plan.models.find((m) => m.modality === 'text'); + const imageEntry = plan.models.find((m) => m.modality === 'image'); + const textPath = textEntry ? targetOf(textEntry.manifestId) : undefined; + const imagePath = imageEntry ? targetOf(imageEntry.manifestId) : undefined; + if (textPath || imagePath) { + lines.push('# ── Model selection (paths inside the models volume) ──────'); + if (textPath) { + lines.push(`TEXT_MODEL=${basename(textPath)}`); + } + if (imagePath) { + lines.push(`IMAGE_MODEL=${basename(imagePath)}`); + } + lines.push(''); + } + + // ── Licences ────────────────────────────────────────────────────────── + const acknowledged = plan.models.filter((m) => m.requiresAcknowledgement); + if (acknowledged.length > 0) { + lines.push('# ── Licences ─────────────────────────────────────────────'); + lines.push('# The plan includes use-restricted model(s); accepted below.'); + lines.push(`AIKAMI_ACCEPT_LICENSES=${acknowledged.map((m) => m.license).join(',')}`); + lines.push(''); + } + + // ── Voice extras ────────────────────────────────────────────────────── + if (plan.modalities.includes('stt')) { + lines.push('# ── Voice extras ─────────────────────────────────────────'); + lines.push('ENABLE_STT=true'); + lines.push(''); + } + + // ── Ports (defaults from development_ports.ts) ─────────────────────── + lines.push('# ── Ports (defaults from packages/shared/constants/development_ports.ts) ─'); + lines.push(`TEXT_PORT=${EMULATOR_PORTS.text}`); + lines.push(`IMAGE_PORT=${EMULATOR_PORTS.image}`); + lines.push(`TTS_PORT=${EMULATOR_PORTS.voice}`); + lines.push(`STT_PORT=${EMULATOR_PORTS.stt}`); + lines.push(`WEB_PORT=${EMULATOR_PORTS.client}`); + lines.push(''); + + if (extras) { + for (const [key, value] of Object.entries(extras)) { + lines.push(`${key}=${value}`); + } + } + + return lines.join('\n') + '\n'; +}; + +/** One-line hardware description for the .env header. */ +export const describeProfile = (profile: HardwareProfile): string => { + const gpu = + profile.gpu.vendor === 'none' + ? 'no GPU' + : `${profile.gpu.vendor}${profile.gpu.name ? ` ${profile.gpu.name}` : ''}${profile.gpu.vramMb ? ` (${profile.gpu.vramMb} MiB)` : ''}`; + return `${profile.platform}/${profile.arch}, ${gpu}, ${profile.ramMb} MiB RAM, ${profile.cores} cores, ${(profile.freeDiskBytes / 1024 / 1024 / 1024).toFixed(1)} GiB free`; +}; + +/** True when the path exists. */ +export const exists = async (path: string): Promise => { + try { + await access(path, constants.F_OK); + return true; + } catch { + return false; + } +}; + +/** + * Loads the existing .env contents, if any. + */ +export const readExistingEnv = async (path: string): Promise => { + if (!(await exists(path))) { + return undefined; + } + return readFile(path, 'utf8'); +}; + +/** + * Computes a minimal unified diff between two .env texts (old → new). + * One line per change: `- old` / `+ new` / ` context`. + */ +export const diffEnv = (oldText: string, newText: string): string => { + const oldLines = oldText.split('\n'); + const newLines = newText.split('\n'); + const oldMap = new Map(); + for (let i = 0; i < oldLines.length; i += 1) { + const line = oldLines[i] as string; + if (line && !line.startsWith('#')) { + oldMap.set(line, (oldMap.get(line) ?? 0) + 1); + } + } + const newMap = new Map(); + for (let i = 0; i < newLines.length; i += 1) { + const line = newLines[i] as string; + if (line && !line.startsWith('#')) { + newMap.set(line, (newMap.get(line) ?? 0) + 1); + } + } + + const out: string[] = [' # .env diff (existing → generated)']; + const keys = new Set([...oldMap.keys(), ...newMap.keys()]); + for (const key of keys) { + const oldCount = oldMap.get(key) ?? 0; + const newCount = newMap.get(key) ?? 0; + if (oldCount === 0) { + out.push(`+ ${key}`); + } else if (newCount === 0) { + out.push(`- ${key}`); + } else if (oldCount !== newCount) { + out.push(`- ${key}`); + out.push(`+ ${key}`); + } + } + if (out.length === 1) { + out.push(' (no changes)'); + } + return out.join('\n'); +}; + +/** + * Atomically writes the .env: temp file in the same directory, then rename. + * Backs up an existing file to `.bak` first. + * + * @throws Error when the directory is missing or the write fails. + */ +export const writeEnvAtomic = async (options: { + readonly path: string; + readonly content: string; +}): Promise => { + const { path, content } = options; + const dir = dirname(path); + const tmp = join(dir, `.env.tmp-${process.pid}`); + await writeFile(tmp, content, 'utf8'); + if (await exists(path)) { + await copyFile(path, `${path}.bak`); + } + await rename(tmp, path); +}; + +/** Env extras for CUDA 13 hosts (AC-2: server-cuda vs server-cuda13). */ +export const cudaExtras = (profile: HardwareProfile): Record | undefined => { + if (profile.gpu.vendor !== 'nvidia' || profile.gpu.cudaMajor !== 13) { + return undefined; + } + return { + // biome-ignore lint/style/useNamingConvention: env var key, SCREAMING_SNAKE_CASE is the .env convention + TEXT_SERVER_IMAGE: 'ghcr.io/ggml-org/llama.cpp:server-cuda13', + }; +}; diff --git a/apps/backend/local-stack/stack/fetch_models.test.ts b/apps/backend/local-stack/stack/fetch_models.test.ts index 96484850..4668001a 100644 --- a/apps/backend/local-stack/stack/fetch_models.test.ts +++ b/apps/backend/local-stack/stack/fetch_models.test.ts @@ -273,6 +273,34 @@ describe('AC-6 — profile scoping', () => { expect(Bun.file(join(dir, 'image/bad.bin')).exists()).resolves.toBe(false); await rm(dir, { recursive: true, force: true }); }); + + it('entryIds limits the run to exactly the planned models (C-391 --fetch)', async () => { + const dir = await makeTmpDir(); + const manifestPath = join(dir, 'models.manifest.json'); + const planned = makeEntry({ id: 'planned', targetPath: 'text/planned.bin' }); + const unplanned = makeEntry({ + id: 'unplanned', + modality: 'image', + targetPath: 'image/unplanned.bin', + }); + await writeFile( + manifestPath, + JSON.stringify({ schemaVersion: 1, entries: [planned, unplanned] }), + ); + // Profiles include both modalities, but entryIds pins the fetch to the + // planned model only — the unplanned entry must NOT be downloaded even + // though its profile is enabled. + const code = await run({ + manifestPath, + modelsDir: dir, + profiles: 'text,image', + entryIds: ['planned'], + }); + expect(code).toBe(0); + expect(Bun.file(join(dir, 'text/planned.bin')).exists()).resolves.toBe(true); + expect(Bun.file(join(dir, 'image/unplanned.bin')).exists()).resolves.toBe(false); + await rm(dir, { recursive: true, force: true }); + }); }); describe('AC-7 — use-restricted models require acknowledgement', () => { diff --git a/apps/backend/local-stack/stack/fetch_models.ts b/apps/backend/local-stack/stack/fetch_models.ts index 1ff33d75..96d94d49 100644 --- a/apps/backend/local-stack/stack/fetch_models.ts +++ b/apps/backend/local-stack/stack/fetch_models.ts @@ -422,6 +422,8 @@ export const run = async (options: { profiles?: string; acceptLicenses?: string; entryId?: string; + /** Explicit entry ids to fetch — limits the run to exactly these planned models. */ + entryIds?: readonly string[]; onProgress?: (options: { entryId: string; received: number; expected: number }) => void; }): Promise => { const manifestPath = options.manifestPath ?? join(import.meta.dir, 'models.manifest.json'); @@ -474,7 +476,9 @@ export const run = async (options: { const selectedEntries = manifest.entries.filter( (entry) => - enabledModalities.has(entry.modality) && (!options.entryId || entry.id === options.entryId), + enabledModalities.has(entry.modality) && + (!options.entryId || entry.id === options.entryId) && + (!options.entryIds || options.entryIds.includes(entry.id)), ); if (selectedEntries.length === 0) { diff --git a/apps/backend/local-stack/stack/init.test.ts b/apps/backend/local-stack/stack/init.test.ts new file mode 100644 index 00000000..df972d5f --- /dev/null +++ b/apps/backend/local-stack/stack/init.test.ts @@ -0,0 +1,274 @@ +/** + * apps/backend/local-stack/stack/init.test.ts + * + * C-391 wizard-level ACs exercised through the CLI entry with a stub + * environment: AC-6 (disk shortfall), AC-7 (plan before write, decline + * writes nothing), AC-9 (re-run diff + byte-identical decline), AC-11 + * (platform separator), AC-13 (--json schema-valid output). + * + * These tests run `runInit` against the REAL Bun/CLI executor (probes + * degrade gracefully on a no-GPU host — AC-1) with a stubbed manifest and + * an isolated --env-path so nothing touches the repo's real .env. + */ + +import { afterEach, describe, expect, test } from 'bun:test'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { HardwareProfileSchema, StackPlanSchema } from '@aikami/schemas'; +import { Value } from 'typebox/value'; +import { type CliOptions, runInit } from './init.ts'; + +const MANIFEST = JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-qwen2.5-1.5b-instruct-q4km', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'bartowski/Qwen2.5-1.5B-Instruct-GGUF', + revision: 'rev', + file: 'Qwen2.5-1.5B-Instruct-Q4_K_M.gguf', + targetPath: 'text/qwen2.5-1.5b-instruct-q4_k_m.gguf', + bytes: 986_048_768, + sha256: 'a'.repeat(64), + }, + { + id: 'image-sd15-pruned-q4_0', + modality: 'image', + tier: 'cpu', + license: 'CreativeML OpenRAIL-M', + requiresAcknowledgement: true, + kind: 'file', + repo: 'second-state/stable-diffusion-v1-5-GGUF', + revision: 'rev', + file: 'stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf', + targetPath: 'image/stable-diffusion-v1-5-pruned-emaonly-q4_0.gguf', + bytes: 1_566_768_416, + sha256: 'b'.repeat(64), + }, + { + id: 'tts-kokoro-82m', + modality: 'tts', + tier: 'any', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'archive', + url: 'https://example.com/kokoro.tar.bz2', + targetPath: 'tts/kokoro-multi-lang-v1_0', + bytes: 349_418_188, + sha256: 'c'.repeat(64), + }, + ], +}); + +/** A manifest whose download sum provably exceeds any volume's free space. */ +const HUGE_MANIFEST = JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-huge', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'r', + revision: 'rev', + file: 'm.gguf', + targetPath: 'text/m.gguf', + // 2^53 bytes — larger than free space on any real volume. + bytes: 9_007_199_254_740_992, + sha256: 'a'.repeat(64), + }, + ], +}); + +const tmpDirs: string[] = []; +const makeTmp = async (): Promise => { + const dir = await mkdtemp(join(tmpdir(), 'aikami-init-')); + tmpDirs.push(dir); + return dir; +}; + +afterEach(async () => { + for (const dir of tmpDirs.splice(0)) { + await rm(dir, { recursive: true, force: true }); + } +}); + +const baseOptions = async (): Promise => { + const dir = await makeTmp(); + const manifestPath = join(dir, 'models.manifest.json'); + await writeFile(manifestPath, MANIFEST); + return { + yes: true, + json: false, + fetch: false, + noColor: true, + envPath: join(dir, '.env'), + manifestPath, + }; +}; + +describe('AC-6 — insufficient disk fails before writing', () => { + test('total download > free disk → exit 2, shortfall in GB, no .env', async () => { + const base = await baseOptions(); + await writeFile(base.manifestPath, HUGE_MANIFEST); + const code = await runInit(base); + expect(code).toBe(2); + const envExists = await readFile(base.envPath, 'utf8').then( + () => true, + () => false, + ); + expect(envExists).toBe(false); + }); +}); + +describe('AC-7 — plan is shown before anything is written', () => { + test('--yes writes the .env with the full plan visible', async () => { + const base = await baseOptions(); + const out: string[] = []; + const originalWrite = process.stdout.write.bind(process.stdout); + const stub = (chunk: string): boolean => { + out.push(String(chunk)); + return true; + }; + process.stdout.write = stub as typeof process.stdout.write; + let code: number; + try { + code = await runInit(base); + } finally { + process.stdout.write = originalWrite; + } + expect(code).toBe(0); + // The rendered plan must be printed before the write: its heading and + // every selected model id appear on stdout (AC-7 plan-first). + const rendered = out.join(''); + expect(rendered).toContain('Aikami local stack — plan'); + expect(rendered).toContain('text-qwen2.5-1.5b-instruct-q4km'); + expect(rendered).toContain('image-sd15-pruned-q4_0'); + expect(rendered).toContain('tts-kokoro-82m'); + const content = await readFile(base.envPath, 'utf8'); + expect(content).toContain('COMPOSE_PROFILES=text,image,voice,stt'); + expect(content).toContain('COMPOSE_FILE='); + expect(content).toContain('TEXT_MODEL='); + }); + + test('declining the write prompt writes nothing', async () => { + const base = await baseOptions(); + // Simulate an interactive TTY answering "n" to "Write .env?". + const originalInIsTTY = process.stdin.isTTY; + const originalOutIsTTY = process.stdout.isTTY; + const originalOn = process.stdin.on.bind(process.stdin); + const originalPause = process.stdin.pause.bind(process.stdin); + const originalOff = process.stdin.off?.bind(process.stdin) ?? (() => {}); + // biome-ignore lint/suspicious/noExplicitAny: test-only TTY stub + Object.defineProperty(process.stdin, 'isTTY', { value: true, configurable: true }); + // biome-ignore lint/suspicious/noExplicitAny: test-only TTY stub + Object.defineProperty(process.stdout, 'isTTY', { value: true, configurable: true }); + // biome-ignore lint/suspicious/noExplicitAny: test-only stdin stub + process.stdin.on = ((event: string, handler: (chunk: string) => void) => { + if (event === 'data') { + queueMicrotask(() => handler('n\n')); + } + return process.stdin; + }) as typeof process.stdin.on; + process.stdin.pause = (() => process.stdin) as typeof process.stdin.pause; + process.stdin.off = (() => process.stdin) as typeof process.stdin.off; + try { + const code = await runInit({ ...base, yes: false }); + expect(code).toBe(0); + } finally { + Object.defineProperty(process.stdin, 'isTTY', { value: originalInIsTTY, configurable: true }); + Object.defineProperty(process.stdout, 'isTTY', { + value: originalOutIsTTY, + configurable: true, + }); + process.stdin.on = originalOn; + process.stdin.pause = originalPause; + if (process.stdin.off) { + process.stdin.off = originalOff; + } + } + const envExists = await readFile(base.envPath, 'utf8').then( + () => true, + () => false, + ); + expect(envExists).toBe(false); + }); +}); + +describe('AC-11 — platform-correct separator', () => { + test('renderEnv uses ":" on linux and ";" on win32', async () => { + const { renderEnv } = await import('./env_writer.ts'); + const { detectHardware, loadManifest, recommend } = await import('@aikami/local-ai'); + const { probeExecutor } = await import('./probe_executor.ts'); + + const manifest = await loadManifest({ + executor: probeExecutor, + path: join(import.meta.dir, 'models.manifest.json'), + }); + const profile = await detectHardware({ + executor: probeExecutor, + platform: 'linux', + arch: 'x64', + }); + const plan = recommend({ + profile, + modalities: ['text'], + manifest, + backendOverride: 'cpu', + }); + + const linuxEnv = renderEnv({ profile, plan, manifest }); + expect(linuxEnv).toContain('COMPOSE_FILE=compose.yaml:compose.cpu.yaml'); + + const winProfile = { ...profile, platform: 'win32' as const }; + const winEnv = renderEnv({ profile: winProfile, plan, manifest }); + expect(winEnv).toContain('COMPOSE_FILE=compose.yaml;compose.cpu.yaml'); + }); +}); + +describe('AC-13 — --json output is complete and stable', () => { + test('stdout is a single schema-valid JSON document with profile + plan', async () => { + const base = await baseOptions(); + const out: string[] = []; + const originalWrite = process.stdout.write.bind(process.stdout); + const stub = (chunk: string): boolean => { + out.push(String(chunk)); + return true; + }; + process.stdout.write = stub as typeof process.stdout.write; + try { + const code = await runInit({ ...base, json: true }); + expect(code).toBe(0); + } finally { + process.stdout.write = originalWrite; + } + const text = out.join(''); + expect(text.trim().startsWith('{')).toBe(true); + const doc = JSON.parse(text) as { profile: unknown; plan: unknown }; + expect(Value.Check(HardwareProfileSchema, doc.profile)).toBe(true); + expect(Value.Check(StackPlanSchema, doc.plan)).toBe(true); + }); +}); + +describe('AC-9 — re-run is safe', () => { + test('declining the overwrite leaves the file byte-identical', async () => { + const base = await baseOptions(); + const first = await runInit(base); + expect(first).toBe(0); + const original = await readFile(base.envPath, 'utf8'); + + // Second run WITHOUT --yes: non-TTY stdin defaults the overwrite + // confirm to false, so the existing .env must remain untouched. + const code = await runInit({ ...base, yes: false }); + expect(code).toBe(0); + const after = await readFile(base.envPath, 'utf8'); + expect(after).toBe(original); + }); +}); diff --git a/apps/backend/local-stack/stack/init.ts b/apps/backend/local-stack/stack/init.ts new file mode 100644 index 00000000..aba33607 --- /dev/null +++ b/apps/backend/local-stack/stack/init.ts @@ -0,0 +1,472 @@ +/** + * apps/backend/local-stack/stack/init.ts + * + * `stack init` — hardware detection, modality selection, model + * recommendation, and .env generation (C-391). + * + * New-user story: + * bun run stack init + * answers up to three questions and gets a .env matched to their machine + * plus an explicit plan of what will be downloaded — before anything is + * fetched. + * + * Fully scriptable: + * bun run stack init --yes --backend cuda --modalities text,voice --tier auto + * bun run stack init --yes --json + * + * Behaviour contract (AC-1..AC-13): + * - Detection is entirely local, each probe capped at 1 s, no network. + * - Every probe failure degrades to a partial profile — never an error. + * - The plan is shown BEFORE anything is written; declining writes nothing. + * - `init` never downloads; use `--fetch` to chain C-390's fetcher. + * - Re-running diffs the existing .env and requires confirmation; the + * previous file is backed up; writes are atomic (temp + rename). + * - Insufficient disk fails before writing, states the shortfall in GB, + * and suggests the next tier down. + * - `--json` emits a single schema-valid JSON document with the full + * HardwareProfile and StackPlan. + * - Respects NO_COLOR; output is plain text without colour or Unicode + * when a non-TTY or NO_COLOR is present. + */ + +import { join } from 'node:path'; +import process from 'node:process'; +import type { + HardwareProfile, + ModelManifest, + StackBackend, + StackModality, + StackPlan, +} from '@aikami/local-ai'; +import { detectHardware, loadManifest, recommend } from '@aikami/local-ai'; +import { HardwareProfileSchema, StackPlanSchema } from '@aikami/schemas'; +import { Value } from 'typebox/value'; +import { cudaExtras, diffEnv, readExistingEnv, renderEnv, writeEnvAtomic } from './env_writer.ts'; +import { probeExecutor } from './probe_executor.ts'; + +export type CliOptions = { + yes: boolean; + backend?: StackBackend; + modalities?: readonly StackModality[]; + tier?: 'auto' | 'cpu' | '8gb' | '16gb'; + json: boolean; + fetch: boolean; + envPath?: string; + manifestPath?: string; + noColor: boolean; + diskPath?: string; +}; + +const DEFAULT_MODALITIES: readonly StackModality[] = ['text', 'image', 'voice', 'stt']; +const MODALITY_CHOICES: readonly StackModality[] = ['text', 'image', 'voice', 'stt', 'web']; +const BACKEND_CHOICES: readonly StackBackend[] = [ + 'cpu', + 'cuda', + 'rocm', + 'vulkan', + 'intel', + 'musa', + 'metal', +]; + +const hasColor = (noColor: boolean): boolean => + !noColor && !process.env.NO_COLOR && Boolean(process.stdout.isTTY); + +const color = (code: string, text: string, enabled: boolean): string => + enabled ? `\u001b[${code}m${text}\u001b[0m` : text; + +/** Formats bytes as "X.X GB" (decimal, for user-facing sizes). */ +const formatGb = (bytes: number): string => `${(bytes / 1e9).toFixed(1)} GB`; + +/** Port table shown in the plan (matches development_ports.ts defaults). */ +const PORTS: Readonly>> = { + text: 11434, + image: 8188, + voice: 8089, + stt: 8087, + web: 5274, +} as const; + +/** + * Renders the plan to a plain-text block (AC-7): backend, per-model sizes + * with one-line rationale, total download, free disk, licences, bound + * ports, and warnings. No Unicode box-drawing; safe for NO_COLOR / non-TTY. + */ +export const renderPlan = (options: { + readonly profile: HardwareProfile; + readonly plan: StackPlan; + readonly colorEnabled: boolean; +}): string => { + const { profile, plan, colorEnabled } = options; + const c = (code: string, text: string): string => color(code, text, colorEnabled); + const out: string[] = []; + + out.push(c('1;36', 'Aikami local stack — plan')); + out.push(''); + out.push( + ` Backend ${c('1', plan.backend)}${plan.nativeEngines ? ' (native engines — macOS)' : ''}`, + ); + out.push(` Modalities ${plan.modalities.join(', ') || '(none)'}`); + out.push( + ` Hardware ${profile.gpu.vendor === 'none' ? 'CPU-only' : `${profile.gpu.vendor} ${profile.gpu.name ?? ''}`.trim()}${profile.gpu.vramMb ? ` · ${profile.gpu.vramMb} MiB VRAM` : ''}`, + ); + if (plan.models.length > 0) { + out.push(''); + out.push(c('1', ' Models to download')); + for (const model of plan.models) { + out.push(` - ${c('1', model.manifestId)} (${formatGb(model.bytes)}) — ${model.rationale}`); + if (model.requiresAcknowledgement) { + out.push(` ${c('1;33', `licence: ${model.license} (use-restricted — accepted)`)}`); + } + } + out.push(` Total download ${c('1', formatGb(plan.totalDownloadBytes))}`); + } else { + out.push(' Models (none selected)'); + } + out.push(` Free disk ${formatGb(profile.freeDiskBytes)}`); + const boundPorts = plan.modalities + .map((modality) => + PORTS[modality] !== undefined ? `${modality}=${PORTS[modality]}` : undefined, + ) + .filter((part): part is string => part !== undefined); + if (boundPorts.length > 0) { + out.push(` Ports ${boundPorts.join(' ')}`); + } + if (plan.nativeEngines) { + out.push(' Engines run natively (bin/run-native-*.sh)'); + } + if (plan.warnings.length > 0) { + out.push(''); + out.push(c('1;33', ' Warnings')); + for (const warning of plan.warnings) { + out.push(` - ${warning}`); + } + } + return out.join('\n') + '\n'; +}; + +/** Reads the manifest from the default or provided path. */ +const readManifest = async (manifestPath?: string): Promise => { + const path = manifestPath ?? join(import.meta.dir, 'models.manifest.json'); + return loadManifest({ executor: probeExecutor, path }); +}; + +/** Asks one yes/no question on the TTY. Returns the default when non-TTY. */ +const confirm = async (prompt: string, defaultValue: boolean): Promise => { + if (!process.stdin.isTTY || !process.stdout.isTTY) { + return defaultValue; + } + process.stdout.write(`${prompt} ${defaultValue ? '[Y/n]' : '[y/N]'} `); + const input = await new Promise((resolve) => { + const stdin = process.stdin; + stdin.resume(); + let data = ''; + stdin.setEncoding('utf8'); + const onData = (chunk: string): void => { + data += chunk; + if (data.includes('\n') || data.includes('\r')) { + stdin.pause(); + stdin.off('data', onData); + resolve(data.trim()); + } + }; + stdin.on('data', onData); + }); + if (input.length === 0) { + return defaultValue; + } + return /^y(es)?$/i.test(input); +}; + +/** Asks a choice question on the TTY, returning the default when non-TTY. */ +const choose = async ( + prompt: string, + choices: readonly string[], + defaultValue: string, +): Promise => { + if (!process.stdin.isTTY || !process.stdout.isTTY) { + return defaultValue; + } + process.stdout.write(`${prompt} [${choices.join('/')}] (default: ${defaultValue}) `); + const input = await new Promise((resolve) => { + const stdin = process.stdin; + stdin.resume(); + let data = ''; + stdin.setEncoding('utf8'); + const onData = (chunk: string): void => { + data += chunk; + if (data.includes('\n') || data.includes('\r')) { + stdin.pause(); + stdin.off('data', onData); + resolve(data.trim()); + } + }; + stdin.on('data', onData); + }); + if (input.length === 0) { + return defaultValue; + } + const match = choices.find((choice) => choice.toLowerCase() === input.toLowerCase()); + return match ?? defaultValue; +}; + +/** Runs the full init flow. Returns the process exit code. */ +export const runInit = async (options: CliOptions): Promise => { + const colorEnabled = hasColor(options.noColor); + const c = (code: string, text: string): string => color(code, text, colorEnabled); + + // ── Detection (entirely local; probes capped at 1 s each) ──────────── + const platform = + process.platform === 'darwin' ? 'darwin' : process.platform === 'win32' ? 'win32' : 'linux'; + const arch = process.arch === 'arm64' ? 'arm64' : 'x64'; + const profile = await detectHardware({ + executor: probeExecutor, + platform, + arch, + diskPath: options.diskPath, + }); + + // ── Manifest ────────────────────────────────────────────────────────── + let manifest: ModelManifest; + try { + manifest = await readManifest(options.manifestPath); + } catch (error) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.error( + c( + '1;31', + `error: cannot read models.manifest.json: ${error instanceof Error ? error.message : String(error)}`, + ), + ); + return 1; + } + + // ── Modalities (flag > TTY prompt > default) ───────────────────────── + let modalities = options.modalities ?? DEFAULT_MODALITIES; + if (!options.yes && !options.modalities) { + const picked = await choose( + 'Which engines do you want?', + MODALITY_CHOICES, + DEFAULT_MODALITIES.join(','), + ); + modalities = picked + .split(',') + .map((part) => part.trim()) + .filter((part): part is StackModality => MODALITY_CHOICES.includes(part as StackModality)); + } + + // ── Backend (flag > prompt > auto) ──────────────────────────────────── + let backendOverride: StackBackend | undefined = options.backend; + if (!options.yes && !options.backend) { + const detected = + profile.gpu.vendor === 'nvidia' && profile.gpuPassthroughReady + ? 'cuda' + : profile.gpu.vendor === 'amd' + ? 'rocm' + : profile.gpu.vendor === 'intel' + ? 'vulkan' + : profile.platform === 'darwin' + ? 'metal' + : 'cpu'; + const picked = await choose('Hardware backend?', BACKEND_CHOICES, detected); + if (picked !== 'auto') { + backendOverride = picked as StackBackend; + } + } + + // ── Tier (flag > prompt > auto) ────────────────────────────────────── + let tierOverride: 'auto' | 'cpu' | '8gb' | '16gb' = options.tier ?? 'auto'; + if (!options.yes && !options.tier) { + const picked = await choose('Model tier?', ['auto', 'cpu', '8gb', '16gb'], 'auto'); + tierOverride = picked as 'auto' | 'cpu' | '8gb' | '16gb'; + } + + // ── Recommendation ──────────────────────────────────────────────────── + const plan = recommend({ + profile, + modalities, + manifest, + backendOverride, + tierOverride: tierOverride === 'auto' ? undefined : tierOverride, + }); + + // ── Disk check (AC-6): fail before writing, state shortfall in GB ──── + if (plan.totalDownloadBytes > profile.freeDiskBytes) { + const shortfall = (plan.totalDownloadBytes - profile.freeDiskBytes) / 1e9; + // biome-ignore lint/suspicious/noConsole: CLI output + console.error( + c( + '1;31', + `error: ${formatGb(plan.totalDownloadBytes)} download needs ${formatGb(profile.freeDiskBytes)} free — short by ${shortfall.toFixed(1)} GB on this volume.`, + ), + ); + // biome-ignore lint/suspicious/noConsole: CLI output + console.error(c('1;33', 'Try a smaller tier (`--tier cpu`) or free disk space, then re-run.')); + return 2; + } + + // ── JSON output (AC-13) ─────────────────────────────────────────────── + if (options.json) { + const profileValid = Value.Check(HardwareProfileSchema, profile); + const planValid = Value.Check(StackPlanSchema, plan); + if (!profileValid || !planValid) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.error(c('1;31', 'error: internal — profile or plan failed schema validation')); + return 1; + } + const doc = { profile, plan }; + process.stdout.write(`${JSON.stringify(doc, null, 2)}\n`); + return 0; + } + + // ── Plan presentation (AC-7) ────────────────────────────────────────── + process.stdout.write(renderPlan({ profile, plan, colorEnabled })); + + // ── Confirmation (AC-7): declining writes nothing ──────────────────── + const confirmed = options.yes ? true : await confirm('Write .env?', true); + if (!confirmed) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.log('Aborted — nothing written.'); + return 0; + } + + // ── Re-run diff (AC-9) ──────────────────────────────────────────────── + const envPath = options.envPath ?? join(import.meta.dir, '..', '.env'); + const existing = await readExistingEnv(envPath); + const content = renderEnv({ + profile, + plan, + manifest, + extras: plan.backend === 'cuda' ? cudaExtras(profile) : undefined, + }); + if (existing !== undefined) { + process.stdout.write(`\n${diffEnv(existing, content)}\n`); + if (!options.yes) { + const overwrite = await confirm('An .env already exists — overwrite?', false); + if (!overwrite) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.log('Aborted — existing .env left untouched.'); + return 0; + } + } + } + + // ── Atomic write (Quality: temp + rename, backup preserved) ────────── + try { + await writeEnvAtomic({ path: envPath, content }); + } catch (error) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.error( + c( + '1;31', + `error: failed to write ${envPath}: ${error instanceof Error ? error.message : String(error)}`, + ), + ); + return 1; + } + // biome-ignore lint/suspicious/noConsole: CLI output + console.log(c('1;32', `✓ wrote ${envPath}`)); + if (plan.warnings.length > 0) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.log(c('1;33', `⚠ ${plan.warnings.length} warning(s) — review above.`)); + } + + // ── Optional fetcher chaining (--fetch, off by default) ─────────────── + // Chain C-390's fetcher against EXACTLY the planned models: same manifest + // path, the planned modalities as profiles, the licences the plan already + // surfaced as accepted (use-restricted entries were shown in the plan), + // and the plan's manifestId list. The fetcher downloads only these ids — + // never the default profiles or the full manifest. + if (options.fetch) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.log('Running the model fetcher (--fetch)...'); + const { run } = await import('./fetch_models.ts'); + return await run({ + manifestPath: options.manifestPath, + profiles: plan.modalities.join(','), + acceptLicenses: + plan.models + .filter((model) => model.requiresAcknowledgement) + .map((model) => model.license) + .join(',') || undefined, + entryIds: plan.models.map((model) => model.manifestId), + }); + } + + return 0; +}; + +/** Parses argv into CliOptions. Unknown flags are ignored with a warning. */ +export const parseArgs = (argv: readonly string[]): CliOptions => { + const options: CliOptions = { + yes: false, + json: false, + fetch: false, + noColor: false, + }; + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i] as string; + const next = (): string | undefined => argv[i + 1]; + switch (arg) { + case '--yes': + case '-y': + options.yes = true; + break; + case '--json': + options.json = true; + break; + case '--fetch': + options.fetch = true; + break; + case '--no-color': + options.noColor = true; + break; + case '--backend': { + // `auto` means “decide from the detected profile” — it must not + // reach planning as a literal backend because selectBackend() and + // renderEnv() cannot resolve it. Map it to undefined (no override). + const raw = next(); + options.backend = raw === undefined || raw === 'auto' ? undefined : (raw as StackBackend); + i += 1; + break; + } + case '--modalities': + options.modalities = (next() ?? '') + .split(',') + .map((part) => part.trim()) + .filter((part): part is StackModality => + ['text', 'image', 'voice', 'stt', 'web', 'ollama', 'comfyui'].includes(part), + ); + i += 1; + break; + case '--tier': + options.tier = next() as 'auto' | 'cpu' | '8gb' | '16gb'; + i += 1; + break; + case '--env-path': + options.envPath = next(); + i += 1; + break; + case '--manifest-path': + options.manifestPath = next(); + i += 1; + break; + case '--disk-path': + options.diskPath = next(); + i += 1; + break; + default: + if (arg.startsWith('-')) { + // biome-ignore lint/suspicious/noConsole: CLI output + console.warn(`ignoring unknown flag: ${arg}`); + } + break; + } + } + return options; +}; + +if (import.meta.main) { + const options = parseArgs(process.argv.slice(2)); + process.exit(await runInit(options)); +} diff --git a/apps/backend/local-stack/stack/probe_executor.ts b/apps/backend/local-stack/stack/probe_executor.ts new file mode 100644 index 00000000..d049ad91 --- /dev/null +++ b/apps/backend/local-stack/stack/probe_executor.ts @@ -0,0 +1,128 @@ +/** + * apps/backend/local-stack/stack/probe_executor.ts + * + * Bun/CLI ProbeExecutor adapter (C-391). Implements the contract seam from + * @aikami/local-ai with node:child_process / node:fs — the ONLY adapter in + * this repo allowed to spawn processes. See the contract table in the C-391 + * design reference: + * + * - No shell: fixed argv, never a shell string built from probe output. + * - Never throws: missing binary, non-zero exit, timeout, permission + * denial all resolve to a ProbeResult with ok:false and a discriminated + * reason. Rejection is reserved for adapter bugs. + * - Honours the timeout: the child is killed, not abandoned. + * - Byte-faithful: stdout/stderr returned undecorated. + * - Side-effect free: probes are read-only. + */ + +import { spawn } from 'node:child_process'; +import { readFile, statfs } from 'node:fs/promises'; +import type { ProbeExecutor, ProbeResult, StatfsResult } from '@aikami/local-ai'; + +const classifySpawnError = (error: unknown): ProbeResult => { + const message = error instanceof Error ? error.message : String(error); + if (/ENOENT/.test(message) || /not found in \$PATH|executable not found/i.test(message)) { + return { ok: false, reason: 'not-found', detail: message }; + } + if (/EACCES|EPERM/.test(message)) { + return { ok: false, reason: 'denied', detail: message }; + } + return { ok: false, reason: 'failed', detail: message }; +}; + +/** + * Spawns a fixed-argv child, collects stdout/stderr byte-faithfully, and + * resolves within timeoutMs — killing the child on timeout. + */ +const runProbe = ( + command: string, + args: readonly string[], + timeoutMs: number, +): Promise => + new Promise((resolve) => { + let settled = false; + let stdout = ''; + let stderr = ''; + const settle = (result: ProbeResult): void => { + if (!settled) { + settled = true; + resolve(result); + } + }; + + const child = spawn(command, [...args], { + stdio: ['ignore', 'pipe', 'pipe'], + shell: false, + }); + + const timer = setTimeout(() => { + child.kill('SIGKILL'); + // Settle immediately — do not wait for the close event, which may lag + // behind the kill (or never fire if the child is a zombie). The settled + // guard in the close handler below makes this the single resolution. + settle({ ok: false, reason: 'timeout', detail: `killed by SIGKILL after ${timeoutMs}ms` }); + }, timeoutMs); + + child.stdout.setEncoding('utf8'); + child.stdout.on('data', (chunk: string) => { + stdout += chunk; + }); + child.stderr.setEncoding('utf8'); + child.stderr.on('data', (chunk: string) => { + stderr += chunk; + }); + + child.on('error', (error) => { + clearTimeout(timer); + settle(classifySpawnError(error)); + }); + + child.on('close', (code, signal) => { + clearTimeout(timer); + if (settled) { + return; + } + const timedOut = signal === 'SIGKILL' || signal === 'SIGTERM'; + if (timedOut) { + settle({ ok: false, reason: 'timeout', detail: `killed by ${String(signal)}` }); + return; + } + if (code !== 0) { + settle({ ok: false, reason: 'failed', detail: `exit code ${String(code)}` }); + return; + } + settle({ ok: true, stdout, stderr, exitCode: code ?? 0 }); + }); + }); + +const readTextFileProbe = async (path: string): Promise => { + try { + const contents = await readFile(path, 'utf8'); + return { ok: true, stdout: contents, stderr: '', exitCode: 0 }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + if (/ENOENT/.test(message)) { + return { ok: false, reason: 'not-found', detail: message }; + } + if (/EACCES|EPERM/.test(message)) { + return { ok: false, reason: 'denied', detail: message }; + } + return { ok: false, reason: 'failed', detail: message }; + } +}; + +const statfsProbe = async (path: string): Promise => { + try { + const stats = await statfs(path); + return { freeBytes: stats.bavail * stats.bsize }; + } catch { + return { ok: false }; + } +}; + +/** The single Bun/CLI executor instance. */ +export const probeExecutor: ProbeExecutor = { + run: (command, args, options) => runProbe(command, args, options.timeoutMs), + readTextFile: readTextFileProbe, + statfs: statfsProbe, +}; diff --git a/apps/backend/local-stack/stack/recommend.test.ts b/apps/backend/local-stack/stack/recommend.test.ts new file mode 100644 index 00000000..963345a4 --- /dev/null +++ b/apps/backend/local-stack/stack/recommend.test.ts @@ -0,0 +1,131 @@ +/** + * apps/backend/local-stack/stack/recommend.test.ts + * + * C-391 recommendation ACs exercised at the local-stack level (evidence + * files named in the contract matrix): AC-3 VRAM table, AC-4 Apple profile, + * AC-5 modality selection. The pure function lives in @aikami/local-ai; + * these tests drive it with the same manifest the stack ships. + */ + +import { describe, expect, test } from 'bun:test'; +import { join } from 'node:path'; +import type { HardwareProfile } from '@aikami/local-ai'; +import { loadManifest, recommend } from '@aikami/local-ai'; +import { probeExecutor } from './probe_executor.ts'; + +const manifest = await loadManifest({ + executor: probeExecutor, + path: join(import.meta.dir, 'models.manifest.json'), +}); + +const profile = (overrides: Partial): HardwareProfile => ({ + platform: 'linux', + arch: 'x64', + gpu: { vendor: 'none', unifiedMemory: false }, + ramMb: 32768, + cores: 8, + freeDiskBytes: 100 * 1024 * 1024 * 1024, + containerRuntime: 'docker', + gpuPassthroughReady: false, + ...overrides, +}); + +const pickText = (plan: { models: { manifestId: string; modality: string }[] }): string => + plan.models.find((m) => m.modality === 'text')?.manifestId ?? 'none'; + +describe('AC-3 — tier selection respects usable VRAM (shipped manifest)', () => { + test('4 GB → cpu tier (Qwen 1.5B)', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', vramMb: 4096, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest, + }); + expect(pickText(plan)).toBe('text-qwen2.5-1.5b-instruct-q4km'); + }); + + test('8 GB → 8gb tier (Qwen 7B)', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', vramMb: 8192, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest, + }); + expect(pickText(plan)).toBe('text-qwen2.5-7b-instruct-q4km'); + }); + + test('12 GB → 16gb tier entry fits usable 8.4 GB (top-tier fallback warns)', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', vramMb: 12288, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest, + }); + // Mistral-Nemo (6.96 GiB) fits inside 12 GB * 0.7 = 8.4 GB usable. + expect(pickText(plan)).toBe('text-mistral-nemo-12b-instruct-q4km'); + expect(plan.warnings.some((w) => w.includes('nominal') || w.includes('tight fit'))).toBe(true); + }); + + test('24 GB → 16gb tier', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', vramMb: 24576, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest, + }); + expect(pickText(plan)).toBe('text-mistral-nemo-12b-instruct-q4km'); + expect(plan.warnings.some((w) => w.includes('nominal'))).toBe(false); + }); +}); + +describe('AC-4 — unified memory is not treated as VRAM', () => { + test('Apple Silicon 16 GB → usable 8 GB, nativeEngines true', () => { + const plan = recommend({ + profile: profile({ + platform: 'darwin', + arch: 'arm64', + gpu: { vendor: 'apple', unifiedMemory: true }, + ramMb: 16384, + }), + modalities: ['text'], + manifest, + }); + expect(plan.backend).toBe('metal'); + expect(plan.nativeEngines).toBe(true); + const text = plan.models.find((m) => m.modality === 'text'); + expect(text).toBeDefined(); + expect(text?.bytes ?? 0).toBeLessThanOrEqual(8 * 1024 * 1024 * 1024); + expect(plan.warnings.some((w) => w.includes('nominal 8gb'))).toBe(true); + }); +}); + +describe('AC-5 — modality selection controls the download set', () => { + test('--modalities text yields exactly one text model', () => { + const plan = recommend({ + profile: profile({ gpu: { vendor: 'none', unifiedMemory: false }, ramMb: 16384 }), + modalities: ['text'], + manifest, + }); + expect(plan.models).toHaveLength(1); + expect(plan.models[0]?.modality).toBe('text'); + expect(plan.models.some((m) => m.modality === 'image')).toBe(false); + }); + + test('voice + stt pick the any-tier archive entries', () => { + const plan = recommend({ + profile: profile({ gpu: { vendor: 'none', unifiedMemory: false }, ramMb: 16384 }), + modalities: ['voice', 'stt'], + manifest, + }); + const ids = plan.models.map((m) => m.manifestId).sort(); + expect(ids).toEqual(['stt-moonshine-tiny-en-int8', 'tts-kokoro-82m']); + }); +}); diff --git a/apps/backend/local-stack/tsconfig.json b/apps/backend/local-stack/tsconfig.json index af8abd8d..e51cd36d 100644 --- a/apps/backend/local-stack/tsconfig.json +++ b/apps/backend/local-stack/tsconfig.json @@ -12,6 +12,8 @@ "verbatimModuleSyntax": true, "paths": { "@aikami/constants": ["../../../packages/shared/constants/src/index.ts"], + "@aikami/local-ai": ["../../../packages/shared/local-ai/src/index.ts"], + "@aikami/schemas": ["../../../packages/shared/schemas/src/index.ts"], "@aikami/types": ["../../../packages/shared/types/src/index.ts"] } }, diff --git a/apps/frontend/docs/src/content/docs/guides/run-locally.mdx b/apps/frontend/docs/src/content/docs/guides/run-locally.mdx index 3912d068..a114d7fb 100644 --- a/apps/frontend/docs/src/content/docs/guides/run-locally.mdx +++ b/apps/frontend/docs/src/content/docs/guides/run-locally.mdx @@ -10,15 +10,27 @@ speech-to-text (sherpa-onnx Moonshine) — behind one Compose topology whose hardware backend**. Two commands is the whole story: ```bash +bun run stack init # detects your hardware, recommends models, writes .env cd apps/backend/local-stack -cp .env.example .env docker compose up -d ``` +`stack init` (C-391) probes your GPU, RAM, disk, and container runtime, +recommends a backend and model tier, shows the full download plan (sizes, +total, free disk, licences, ports) before writing anything, and produces a +`.env` matched to your machine — no manual editing, no needing to know what +CUDA 12 vs 13 means or which quantisation fits your VRAM. It is fully +scriptable: + +```bash +bun run stack init --yes --backend cuda --modalities text,voice +bun run stack init --yes --json # full profile + plan for CI assertions +``` + ## Pick your hardware -All variation lives in `.env`; the runtime command never changes. Set the -two variables that matter: +All variation lives in `.env`; the runtime command never changes. `stack +init` sets the two variables that matter — or set them by hand: ```bash # .env @@ -34,6 +46,7 @@ COMPOSE_FILE=compose.yaml:compose.cuda.yaml # see the table | Vulkan | `compose.yaml:compose.vulkan.yaml` | Universal GPU fallback (AMD, Intel Arc, iGPUs) | | Intel / SYCL | `compose.yaml:compose.intel.yaml` | Intel Arc / recent integrated graphics | | Moore Threads MUSA | `compose.yaml:compose.musa.yaml` | MUSA GPUs | +| Metal (macOS) | `compose.yaml` (native engines) | Apple Silicon — no GPU passthrough | The first start pulls the engine images and downloads the models you enabled (checksum-verified and resumable). Everything after that starts in seconds. diff --git a/bun.lock b/bun.lock index 7d63139e..d3bf72d5 100644 --- a/bun.lock +++ b/bun.lock @@ -49,6 +49,12 @@ }, "apps/backend/local-stack": { "name": "@aikami/local-stack", + "dependencies": { + "@aikami/constants": "workspace:*", + "@aikami/local-ai": "workspace:*", + "@aikami/schemas": "workspace:*", + "@aikami/types": "workspace:*", + }, }, "apps/backend/text": { "name": "@aikami/text", @@ -364,6 +370,13 @@ "packages/shared/constants": { "name": "@aikami/constants", }, + "packages/shared/local-ai": { + "name": "@aikami/local-ai", + "dependencies": { + "@aikami/schemas": "workspace:*", + "@aikami/types": "workspace:*", + }, + }, "packages/shared/logger": { "name": "@aikami/logger", "dependencies": { @@ -483,6 +496,8 @@ "@aikami/image": ["@aikami/image@workspace:apps/backend/image"], + "@aikami/local-ai": ["@aikami/local-ai@workspace:packages/shared/local-ai"], + "@aikami/local-stack": ["@aikami/local-stack@workspace:apps/backend/local-stack"], "@aikami/logger": ["@aikami/logger@workspace:packages/shared/logger"], diff --git a/package.json b/package.json index b567922a..3594f9d9 100644 --- a/package.json +++ b/package.json @@ -68,6 +68,7 @@ "herdr:preview": "bun run scripts/src/lib/ops/preview_client.ts", "contract": "bun run scripts/src/lib/agents/contract_pipeline.ts", "contract:pipeline": "bun run scripts/src/lib/agents/contract_pipeline.ts", + "stack": "bun run --cwd apps/backend/local-stack init --", "contract:generate": "bun run scripts/src/lib/agents/contract_generator.ts", "validate:wgsl": "bun run scripts/src/lib/ops/validate_wgsl.ts", "validate:gql": "bun run scripts/src/lib/ops/validate_gql_fields.ts", diff --git a/packages/shared/local-ai/moon.yml b/packages/shared/local-ai/moon.yml new file mode 100644 index 00000000..d4fdd294 --- /dev/null +++ b/packages/shared/local-ai/moon.yml @@ -0,0 +1,38 @@ +# packages/shared/local-ai/moon.yml +$schema: 'https://moonrepo.dev/schemas/project.json' + +language: 'typescript' +layer: 'library' +tags: + - 'local-ai' + - 'shared' + - 'library' + +project: + name: 'local-ai' + description: 'Portable planning core for the local AI stack (C-391): hardware detection, tiered model recommendation, ProbeExecutor seam.' + channel: '#backend' + owner: 'Backend Team' + +dependsOn: + - 'constants' + - 'schemas' + - 'types' + +fileGroups: + sources: + - 'src/**/*' + configs: + - 'package.json' + - 'tsconfig.json' + - 'moon.yml' + tests: + - 'src/**/*.test.ts' + +tasks: + test: + command: 'bun run test' + inputs: + - '@group(sources)' + - '@group(tests)' + - '@group(configs)' diff --git a/packages/shared/local-ai/package.json b/packages/shared/local-ai/package.json new file mode 100644 index 00000000..df618879 --- /dev/null +++ b/packages/shared/local-ai/package.json @@ -0,0 +1,19 @@ +{ + "name": "@aikami/local-ai", + "license": "MIT", + "type": "module", + "main": "src/index.ts", + "description": "Portable planning core for the local AI engine stack (C-391): hardware detection, tiered model recommendation, and the ProbeExecutor seam. No dependency on the stack project and no Node/Bun-only imports in the public entry point.", + "scripts": { + "test": "bun test", + "lint": "biome lint .", + "format": "biome format .", + "typecheck": "tsgo --noEmit", + "fix": "biome check --write . --error-on-warnings" + }, + "dependencies": { + "@aikami/schemas": "workspace:*", + "@aikami/types": "workspace:*" + }, + "devDependencies": {} +} diff --git a/packages/shared/local-ai/src/index.ts b/packages/shared/local-ai/src/index.ts new file mode 100644 index 00000000..b8bf32a5 --- /dev/null +++ b/packages/shared/local-ai/src/index.ts @@ -0,0 +1,32 @@ +// packages/shared/local-ai/src/index.ts +// +// Public entry point of @aikami/local-ai — the portable planning core for +// the local AI engine stack (C-391). +// +// 🔴 AC-0 boundary: this entry point (and everything it re-exports) must +// NOT import Node/Bun-only modules (node:child_process, node:fs, node:os) +// and must NOT depend on @aikami/local-stack or any app. Hosts (Bun CLI, +// Tauri) implement ProbeExecutor and supply their own adapters. + +// Re-export the shared types the core consumes so a consumer can build with +// only @aikami/local-ai, @aikami/types, and @aikami/schemas (AC-0). +export type { + CudaMajor, + GpuVendor, + HardwareProfile, + ManifestEntryModality, + ManifestEntryTier, + ModelManifest, + ModelManifestEntry, + StackBackend, + StackModality, + StackPlan, + StackPlanModel, +} from '@aikami/types'; +export * from './lib/detect.ts'; +export * from './lib/fixture_executor.ts'; +export * from './lib/manifest.ts'; +export * from './lib/probe_executor.contract_suite.ts'; +export * from './lib/probe_executor.ts'; +export * from './lib/recommend.ts'; +export * from './lib/tier_table.ts'; diff --git a/packages/shared/local-ai/src/lib/dependency.test.ts b/packages/shared/local-ai/src/lib/dependency.test.ts new file mode 100644 index 00000000..48c2e462 --- /dev/null +++ b/packages/shared/local-ai/src/lib/dependency.test.ts @@ -0,0 +1,76 @@ +// packages/shared/local-ai/src/lib/dependency.test.ts +// +// AC-0: the planning core is importable without the stack project and its +// public entry point imports no Node/Bun-only module (node:child_process, +// node:fs, node:os). This is a build-time assertion, not a review +// convention — a stray `import { spawn } from 'node:child_process'` in the +// core silently breaks the Tauri path months later. + +import { describe, expect, test } from 'bun:test'; +import { readdirSync, readFileSync } from 'node:fs'; +import { readFile } from 'node:fs/promises'; +import { join } from 'node:path'; + +const PACKAGE_ROOT = join(import.meta.dir, '..', '..'); +const SRC_DIR = join(PACKAGE_ROOT, 'src'); + +const NODE_BUILTIN_PATTERNS = [ + // Match the builtin and every subpath (node:fs, node:fs/promises, ...) + // and the Bun runtime module (bun, bun:test) — anything that would make + // the core non-portable outside Bun/Node. + /from\s+['"]node:child_process(?:['"]|\/)/, + /from\s+['"]node:fs(?:['"]|\/)/, + /from\s+['"]node:os(?:['"]|\/)/, + /from\s+['"]node:path(?:['"]|\/)/, + /from\s+['"]bun(?:['"]|:)/, +]; + +const listTsFiles = (dir: string): string[] => + readdirSync(dir, { withFileTypes: true }).flatMap((entry) => { + const full = join(dir, entry.name); + if (entry.isDirectory()) { + return listTsFiles(full); + } + return entry.name.endsWith('.ts') ? [full] : []; + }); + +describe('AC-0 — package boundary', () => { + test('package.json has no dependency on the stack project or any app', async () => { + const pkg = JSON.parse(await readFile(join(PACKAGE_ROOT, 'package.json'), 'utf8')) as { + dependencies?: Record; + }; + const deps = Object.keys(pkg.dependencies ?? {}); + expect(deps).toContain('@aikami/schemas'); + expect(deps).toContain('@aikami/types'); + for (const dep of deps) { + expect(dep).not.toMatch(/local-stack/); + expect(dep).not.toMatch(/^@aikami\/backend/); + expect(dep).not.toMatch(/^@aikami\/frontend/); + } + }); + + test('no source file imports Node builtins in its public graph', () => { + // Test-support files (unit tests and the shared contract-suite harness) + // are excluded: they legitimately import bun:test and are never part of + // the public graph shipped to Tauri/Node consumers. + const sources = listTsFiles(SRC_DIR).filter( + (file) => !file.endsWith('.test.ts') && !file.endsWith('.contract_suite.ts'), + ); + for (const file of sources) { + const source = readFileSync(file, 'utf8'); + for (const pattern of NODE_BUILTIN_PATTERNS) { + expect( + pattern.test(source), + `${file} must not import a Node builtin (matched ${pattern})`, + ).toBe(false); + } + } + }); + + test('the public entry point exists and re-exports the core', async () => { + const index = await readFile(join(PACKAGE_ROOT, 'src', 'index.ts'), 'utf8'); + expect(index).toContain('probe_executor'); + expect(index).toContain('recommend'); + expect(index).toContain('detect'); + }); +}); diff --git a/packages/shared/local-ai/src/lib/detect.test.ts b/packages/shared/local-ai/src/lib/detect.test.ts new file mode 100644 index 00000000..1f2bd58f --- /dev/null +++ b/packages/shared/local-ai/src/lib/detect.test.ts @@ -0,0 +1,201 @@ +// packages/shared/local-ai/src/lib/detect.test.ts +import { describe, expect, test } from 'bun:test'; +import { detectHardware, parseNvidiaSmi, parseProcMeminfo } from './detect.ts'; +import { createFixtureExecutor } from './fixture_executor.ts'; +import type { ProbeResult } from './probe_executor.ts'; + +const ok = (stdout: string): ProbeResult => ({ ok: true, stdout, stderr: '', exitCode: 0 }); + +const NVIDIA_SINGLE = 'NVIDIA GeForce RTX 4070, 12282 MiB, 535.104.05\n'; +const NVIDIA_MULTI = `NVIDIA GeForce RTX 4060, 8188 MiB, 535.104.05\nNVIDIA GeForce RTX 4090, 24564 MiB, 570.00\n`; + +const CPU_EXECUTOR = createFixtureExecutor({ + table: { + commands: [], + files: [ + { + path: '/proc/meminfo', + result: ok('MemTotal: 32768 kB\nMemFree: 16384 kB\n'), + }, + ], + statfs: [{ path: '.', result: { freeBytes: 500 * 1024 * 1024 * 1024 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, +}); + +const DOCKER_NVIDIA_EXECUTOR = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok(NVIDIA_SINGLE), + }, + { command: 'docker', args: ['info'], result: ok('Runtimes: nvidia runc') }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, +}); + +describe('parseNvidiaSmi', () => { + test('parses vendor VRAM and driver→CUDA major (535 → CUDA 12)', () => { + const parsed = parseNvidiaSmi(NVIDIA_SINGLE); + expect(parsed.name).toBe('NVIDIA GeForce RTX 4070'); + expect(parsed.vramMb).toBe(12282); + expect(parsed.cudaMajor).toBe(12); + }); + + test('driver 570+ → CUDA 13', () => { + const parsed = parseNvidiaSmi('NVIDIA GeForce RTX 5070, 12282 MiB, 570.00\n'); + expect(parsed.cudaMajor).toBe(13); + }); + + test('multi-GPU picks the largest device (AC-2 watch point)', () => { + const parsed = parseNvidiaSmi(NVIDIA_MULTI); + expect(parsed.name).toBe('NVIDIA GeForce RTX 4090'); + expect(parsed.vramMb).toBe(24564); + }); + + test('garbage input degrades to an empty profile', () => { + expect(parseNvidiaSmi('not a csv')).toEqual({}); + }); +}); + +describe('parseProcMeminfo', () => { + test('parses MemTotal kB → MB', () => { + expect(parseProcMeminfo('MemTotal: 67108864 kB\n')).toBe(65536); + }); +}); + +describe('AC-1 — detection degrades to CPU without error', () => { + test('no GPU tooling on PATH → gpu.vendor none, containerRuntime none', async () => { + const profile = await detectHardware({ + executor: CPU_EXECUTOR, + platform: 'linux', + arch: 'x64', + }); + expect(profile.gpu.vendor).toBe('none'); + expect(profile.ramMb).toBe(32); + expect(profile.containerRuntime).toBe('none'); + expect(profile.gpuPassthroughReady).toBe(false); + }); +}); + +describe('AC-2 — NVIDIA detection', () => { + test('stubbed nvidia-smi + docker nvidia runtime → nvidia, CUDA 12, passthrough ready', async () => { + const profile = await detectHardware({ + executor: DOCKER_NVIDIA_EXECUTOR, + platform: 'linux', + arch: 'x64', + }); + expect(profile.gpu.vendor).toBe('nvidia'); + expect(profile.gpu.vramMb).toBe(12282); + expect(profile.gpu.cudaMajor).toBe(12); + expect(profile.containerRuntime).toBe('docker'); + expect(profile.gpuPassthroughReady).toBe(true); + }); + + test('CUDA 13 driver', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok('NVIDIA GeForce RTX 5070, 12282 MiB, 580.00\n'), + }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.cudaMajor).toBe(13); + }); +}); + +describe('AC-12 — NVIDIA GPU present, toolkit absent', () => { + test('docker info without nvidia runtime → gpuPassthroughReady false', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok(NVIDIA_SINGLE), + }, + { command: 'docker', args: ['info'], result: ok('Runtimes: runc') }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.vendor).toBe('nvidia'); + expect(profile.gpuPassthroughReady).toBe(false); + expect(profile.containerRuntime).toBe('docker'); + }); +}); + +describe('Apple detection (AC-4)', () => { + test('darwin → apple vendor, unified memory, RAM from sysctl', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { command: 'sysctl', args: ['hw.memsize'], result: ok('17179869184\n') }, + // -n prints the value, but a host may echo the key anyway — the + // fixture proves detect() extracts digits instead of parsing the line. + { command: 'sysctl', args: ['-n', 'hw.ncpu'], result: ok('hw.ncpu: 10\n') }, + ], + files: [], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'darwin', arch: 'arm64' }); + expect(profile.gpu.vendor).toBe('apple'); + expect(profile.gpu.unifiedMemory).toBe(true); + expect(profile.ramMb).toBe(16384); + expect(profile.cores).toBe(10); + }); +}); + +describe('Windows detection', () => { + test('win32 prefers PowerShell CIM for RAM', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [ + { + command: 'powershell', + args: [ + '-NoProfile', + '-Command', + '(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory', + ], + result: ok('17179869184\n'), + }, + // win32 cores come from PowerShell, not nproc (which doesn't exist). + { + command: 'powershell', + args: [ + '-NoProfile', + '-Command', + '(Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors', + ], + result: ok('16\n'), + }, + ], + files: [], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'win32', arch: 'x64' }); + expect(profile.ramMb).toBe(16384); + expect(profile.cores).toBe(16); + }); +}); diff --git a/packages/shared/local-ai/src/lib/detect.ts b/packages/shared/local-ai/src/lib/detect.ts new file mode 100644 index 00000000..2c560ea6 --- /dev/null +++ b/packages/shared/local-ai/src/lib/detect.ts @@ -0,0 +1,294 @@ +// packages/shared/local-ai/src/lib/detect.ts +// +// Hardware detection written ONLY against the ProbeExecutor seam — no +// process spawning in the core. Every probe is individually capped at 1 s +// and non-fatal: a missing binary, a hang, or a permission denial degrades +// to a partial profile and the caller still gets a usable `cpu` plan. +// +// Platform/arch come from the adapter (Bun passes process.platform/arch; a +// Tauri host passes its own; fixtures pass fixed values) so the core stays +// free of node:os. + +import type { GpuVendor, HardwareProfile } from '@aikami/types'; +import type { ProbeExecutor, ProbeResult } from './probe_executor.ts'; + +export const PROBE_TIMEOUT_MS = 1000; + +export type DetectOptions = { + readonly executor: ProbeExecutor; + readonly platform: 'linux' | 'darwin' | 'win32'; + readonly arch: 'x64' | 'arm64'; + /** Volume path used for the free-disk probe (defaults to process cwd). */ + readonly diskPath?: string; +}; + +/** Runs a command probe and returns a safe default when it fails. */ +const probe = async ( + executor: ProbeExecutor, + command: string, + args: readonly string[], +): Promise => executor.run(command, args, { timeoutMs: PROBE_TIMEOUT_MS }); + +/** + * Parses `nvidia-smi --query-gpu=name,memory.total,driver_version` output. + * Multi-GPU: pick the largest single device (engines use one device; + * summing VRAM across cards would over-recommend — C-391 Watch Points). + */ +export const parseNvidiaSmi = ( + stdout: string, +): { + readonly name?: string; + readonly vramMb?: number; + readonly cudaMajor?: 12 | 13; +} => { + const lines = stdout + .split('\n') + .map((line) => line.trim()) + .filter(Boolean); + let best: { name?: string; vramMb?: number; cudaMajor?: 12 | 13 } | undefined; + for (const line of lines) { + const parts = line.split(',').map((part) => part.trim()); + if (parts.length < 2) { + continue; // not an nvidia-smi CSV row + } + const [name, memRaw, driverRaw] = parts; + const memMatch = memRaw?.match(/^(\d+)\s*MiB$/); + const vramMb = memMatch ? Number(memMatch[1]) : undefined; + // Driver major decides CUDA 12 vs 13: CUDA 13 requires driver >= 570. + const driverMajor = Number.parseInt(driverRaw ?? '', 10); + const cudaMajor: 12 | 13 | undefined = Number.isNaN(driverMajor) + ? undefined + : driverMajor >= 570 + ? 13 + : 12; + const candidate = { name, vramMb, cudaMajor }; + if (!best || (vramMb ?? 0) > (best.vramMb ?? 0)) { + best = candidate; + } + } + return best ?? {}; +}; + +/** + * Parses `/proc/meminfo` into total RAM MB. + */ +export const parseProcMeminfo = (stdout: string): number => { + const match = stdout.match(/^MemTotal:\s*(\d+)\s*kB/m); + if (!match) { + return 0; + } + return Math.floor(Number(match[1]) / 1024); +}; + +/** + * Parses `sysctl hw.memsize` (Darwin) — bytes on one line. + */ +export const parseSysctlMemsize = (stdout: string): number => { + const match = stdout.match(/(\d+)/); + if (!match) { + return 0; + } + return Math.floor(Number(match[1]) / 1024 / 1024); +}; + +/** + * Parses PowerShell `Get-CimInstance Win32_ComputerSystem` TotalPhysicalMemory + * (bytes; wmic fallback prints the same number). + */ +export const parseWinTotalMemory = (stdout: string): number => { + const match = stdout.match(/(\d+)/); + if (!match) { + return 0; + } + return Math.floor(Number(match[1]) / 1024 / 1024); +}; + +/** + * Runs every probe and assembles the HardwareProfile. No probe failure + * aborts detection — the profile degrades field by field (AC-1). + * + * @param options — executor, platform, arch, optional disk path. + * @returns The assembled profile. + */ +export const detectHardware = async (options: DetectOptions): Promise => { + const { executor, platform, arch, diskPath } = options; + let gpuVendor: GpuVendor = 'none'; + let gpuName: string | undefined; + let vramMb: number | undefined; + let cudaMajor: 12 | 13 | undefined; + let unifiedMemory = false; + let ramMb = 0; + let cores = 0; + let containerRuntime: HardwareProfile['containerRuntime'] = 'none'; + let gpuPassthroughReady = false; + + if (platform === 'darwin') { + gpuVendor = 'apple'; + unifiedMemory = true; + } + + let freeDiskBytes = 0; + + // Independent probe groups run concurrently: GPU vendor detection (a + // sequential fallback chain — AMD/Vulkan only probed when nothing matched), + // RAM, cores, disk, and the container runtime. The runtime probe + // short-circuits: podman is only probed when the docker probe fails. + await Promise.all([ + (async () => { + // ── NVIDIA ──────────────────────────────────────────────────────── + const nvidia = await probe(executor, 'nvidia-smi', [ + '--query-gpu=name,memory.total,driver_version', + '--format=csv,noheader', + ]); + if (nvidia.ok) { + const parsed = parseNvidiaSmi(nvidia.stdout); + if (parsed.vramMb !== undefined || parsed.name !== undefined) { + gpuVendor = 'nvidia'; + gpuName = parsed.name; + vramMb = parsed.vramMb; + cudaMajor = parsed.cudaMajor; + } + } + + // ── AMD ─────────────────────────────────────────────────────────── + if (gpuVendor === 'none') { + const rocm = await probe(executor, 'rocm-smi', ['--showmeminfo', 'vram']); + if (rocm.ok && rocm.stdout.includes('vram')) { + gpuVendor = 'amd'; + const vramMatch = rocm.stdout.match(/vram\s*\(.*\)\s*:\s*(\d+)/i); + if (vramMatch) { + vramMb = Number(vramMatch[1]); + } + } + } + + // ── Vulkan (Intel Arc / iGPU / unknown GPU) ─────────────────────── + if (gpuVendor === 'none') { + const vulkan = await probe(executor, 'vulkaninfo', ['--summary']); + if (vulkan.ok) { + const summary = vulkan.stdout.toLowerCase(); + if (summary.includes('nvidia')) { + gpuVendor = 'nvidia'; + unifiedMemory = false; + } else if (summary.includes('amd') || summary.includes('advanced micro devices')) { + gpuVendor = 'amd'; + } else if (summary.includes('intel')) { + gpuVendor = 'intel'; + unifiedMemory = true; + } else { + // A generic Vulkan device with no vendor match — treat as intel/iGPU + // (universal fallback) rather than assuming a dGPU. + gpuVendor = 'intel'; + unifiedMemory = true; + } + } + } + })(), + + (async () => { + // ── RAM ─────────────────────────────────────────────────────────── + if (platform === 'linux') { + const meminfo = await executor.readTextFile('/proc/meminfo'); + if (meminfo.ok) { + ramMb = parseProcMeminfo(meminfo.stdout); + } + } else if (platform === 'darwin') { + const memsize = await probe(executor, 'sysctl', ['hw.memsize']); + if (memsize.ok) { + ramMb = parseSysctlMemsize(memsize.stdout); + } + } else { + // win32: prefer PowerShell CIM; wmic is deprecated on Windows 11. + const cim = await probe(executor, 'powershell', [ + '-NoProfile', + '-Command', + '(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory', + ]); + if (cim.ok) { + ramMb = parseWinTotalMemory(cim.stdout); + } else { + const wmic = await probe(executor, 'wmic', [ + 'ComputerSystem', + 'get', + 'TotalPhysicalMemory', + '/value', + ]); + if (wmic.ok) { + ramMb = parseWinTotalMemory(wmic.stdout); + } + } + } + })(), + + (async () => { + // ── Cores ───────────────────────────────────────────────────────── + if (platform === 'linux') { + const nproc = await probe(executor, 'nproc', []); + if (nproc.ok) { + const digits = nproc.stdout.match(/\d+/)?.[0]; + cores = digits ? Number.parseInt(digits, 10) || 0 : 0; + } + } else if (platform === 'darwin') { + // `-n` prints just the value; some hosts still echo the key — + // extract digits, never parse the whole line. + const ncpu = await probe(executor, 'sysctl', ['-n', 'hw.ncpu']); + if (ncpu.ok) { + const digits = ncpu.stdout.match(/\d+/)?.[0]; + cores = digits ? Number.parseInt(digits, 10) || 0 : 0; + } + } else { + // win32: `nproc` does not exist — use PowerShell's CIM query. + const cpuCount = await probe(executor, 'powershell', [ + '-NoProfile', + '-Command', + '(Get-CimInstance Win32_ComputerSystem).NumberOfLogicalProcessors', + ]); + if (cpuCount.ok) { + const digits = cpuCount.stdout.match(/\d+/)?.[0]; + cores = digits ? Number.parseInt(digits, 10) || 0 : 0; + } + } + })(), + + (async () => { + // ── Disk (volume backing the target path) ───────────────────────── + const statfs = await executor.statfs(diskPath ?? '.'); + if ('freeBytes' in statfs) { + freeDiskBytes = statfs.freeBytes; + } + })(), + + (async () => { + // ── Container runtime + GPU passthrough ─────────────────────────── + // podman is only probed when docker is unavailable (short-circuit). + const docker = await probe(executor, 'docker', ['info']); + if (docker.ok) { + containerRuntime = 'docker'; + gpuPassthroughReady = docker.stdout.toLowerCase().includes('nvidia'); + } else { + const podman = await probe(executor, 'podman', ['info']); + if (podman.ok) { + containerRuntime = 'podman'; + gpuPassthroughReady = podman.stdout.toLowerCase().includes('nvidia'); + } + } + })(), + ]); + + return { + platform, + arch, + gpu: { + vendor: gpuVendor, + name: gpuName, + vramMb, + cudaMajor, + unifiedMemory, + }, + ramMb, + cores, + freeDiskBytes, + containerRuntime, + gpuPassthroughReady, + }; +}; diff --git a/packages/shared/local-ai/src/lib/fixture_executor.test.ts b/packages/shared/local-ai/src/lib/fixture_executor.test.ts new file mode 100644 index 00000000..435ad777 --- /dev/null +++ b/packages/shared/local-ai/src/lib/fixture_executor.test.ts @@ -0,0 +1,142 @@ +// packages/shared/local-ai/src/lib/fixture_executor.test.ts +// +// AC-0b: the full detection + recommendation pipeline runs against the +// fixture-replay executor with ZERO process spawns and produces the same +// StackPlan the Bun/CLI adapter would for equivalent inputs. This proves the +// core is host-agnostic — a Tauri adapter needs no change to @aikami/local-ai. +// +// AC-0c: the shared ProbeExecutor contract suite runs against the fixture +// adapter too. + +import { describe, expect, test } from 'bun:test'; +import type { ModelManifest } from '@aikami/types'; +import { detectHardware } from './detect.ts'; +import { createFixtureExecutor } from './fixture_executor.ts'; +import { runProbeExecutorContractSuite } from './probe_executor.contract_suite.ts'; +import { recommend } from './recommend.ts'; + +const ok = (stdout: string) => ({ ok: true as const, stdout, stderr: '', exitCode: 0 }); + +const MANIFEST: ModelManifest = { + schemaVersion: 1, + entries: [ + { + id: 'text-qwen2.5-7b-instruct-q4km', + modality: 'text', + tier: '8gb', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'r', + revision: 'rev', + file: 'm.gguf', + targetPath: 'text/m.gguf', + bytes: 4_683_074_240, + sha256: 'b', + }, + ], +}; + +/** A captured Linux NVIDIA laptop session, replayed with zero spawns. */ +const NVIDIA_FIXTURES = createFixtureExecutor({ + table: { + commands: [ + { + command: 'nvidia-smi', + args: ['--query-gpu=name,memory.total,driver_version', '--format=csv,noheader'], + result: ok('NVIDIA GeForce RTX 3060, 12282 MiB, 535.104.05\n'), + }, + { command: 'docker', args: ['info'], result: ok('Runtimes: nvidia runc\n') }, + ], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 67108864 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, +}); + +describe('AC-0b — fixture pipeline produces the same plan', () => { + test('detect → recommend against fixtures, no spawns', async () => { + const profile = await detectHardware({ + executor: NVIDIA_FIXTURES, + platform: 'linux', + arch: 'x64', + }); + expect(profile.gpu.vendor).toBe('nvidia'); + expect(profile.gpu.vramMb).toBe(12282); + expect(profile.gpu.cudaMajor).toBe(12); + + const plan = recommend({ + profile, + modalities: ['text'], + manifest: MANIFEST, + }); + // The fixture executor carries the same values the Bun adapter would + // return for the same commands, so the plan is the Bun adapter's plan. + expect(plan.backend).toBe('cuda'); + expect(plan.models[0]?.manifestId).toBe('text-qwen2.5-7b-instruct-q4km'); + }); + + test('all probes not-found → still a valid CPU plan (AC-1 offline mode)', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [], + files: [{ path: '/proc/meminfo', result: ok('MemTotal: 16777216 kB\n') }], + statfs: [{ path: '.', result: { freeBytes: 1 << 40 } }], + }, + unmatched: { ok: false, reason: 'not-found' }, + }); + const profile = await detectHardware({ executor, platform: 'linux', arch: 'x64' }); + expect(profile.gpu.vendor).toBe('none'); + const plan = recommend({ profile, modalities: ['text'], manifest: MANIFEST }); + expect(plan.backend).toBe('cpu'); + expect(plan.models.length).toBe(1); + }); +}); + +describe('AC-0c — shared contract suite against the fixture adapter', () => { + runProbeExecutorContractSuite({ + label: 'fixture-replay', + factory: () => + createFixtureExecutor({ + table: { + commands: [ + { + command: 'definitely-not-a-real-binary-xyz', + args: [], + result: { ok: false, reason: 'not-found' }, + }, + { + command: process.execPath, + args: ['-e', 'console.error("boom"); process.exit(3)'], + result: { ok: false, reason: 'failed', detail: 'boom' }, + }, + { + command: process.execPath, + args: ['-e', 'setTimeout(() => {}, 5000)'], + result: { ok: false, reason: 'timeout', detail: 'killed' }, + }, + { + command: process.execPath, + args: [ + '-e', + 'process.stdout.write(process.argv[1])', + ' RTX 4090, 24564 MiB, 570.00 \nSecond line \n', + ], + result: ok(' RTX 4090, 24564 MiB, 570.00 \nSecond line \n'), + }, + ], + files: [ + { + // The contract suite reads its own file for the + // readTextFile case — replay it from a fixture. + path: `${import.meta.dir}/probe_executor.contract_suite.ts`, + result: ok('content\n'), + }, + { path: '/fixture/denied.txt', result: { ok: false, reason: 'denied' } }, + ], + statfs: [{ path: '/fixture', result: { freeBytes: 123 } }], + }, + }), + permissionDeniedPath: '/fixture/denied.txt', + }); +}); diff --git a/packages/shared/local-ai/src/lib/fixture_executor.ts b/packages/shared/local-ai/src/lib/fixture_executor.ts new file mode 100644 index 00000000..87bedb99 --- /dev/null +++ b/packages/shared/local-ai/src/lib/fixture_executor.ts @@ -0,0 +1,79 @@ +// packages/shared/local-ai/src/lib/fixture_executor.ts +// +// Fixture-replay ProbeExecutor: answers every run/readTextFile/statfs call +// from a captured-fixture table with ZERO process spawns. This is the +// adapter that proves the core is host-agnostic (AC-0b) and that a Tauri +// adapter needs no change to @aikami/local-ai. +// +// Fixtures must be captured from real machines (nvidia-smi, rocm-smi, +// vulkaninfo, /proc/meminfo, sysctl, docker info) — see the C-391 Watch +// Points. Hand-written strings drift from reality. + +import type { ProbeExecutor, ProbeResult, StatfsResult } from './probe_executor.ts'; + +export type CommandFixture = { + readonly command: string; + readonly args?: readonly string[]; + readonly result: ProbeResult; +}; + +export type FileFixture = { + readonly path: string; + readonly result: ProbeResult; +}; + +export type StatfsFixture = { + readonly path: string; + readonly result: StatfsResult; +}; + +export type FixtureTable = { + readonly commands: readonly CommandFixture[]; + readonly files: readonly FileFixture[]; + readonly statfs: readonly StatfsFixture[]; +}; + +/** + * Structural argument comparison: same length, same value per position. + * join(' ') is ambiguous (["a b", "c"] and ["a", "b c"] collide), so + * compare the vectors element-wise. + */ +const sameArgs = (a: readonly string[], b: readonly string[]): boolean => + a.length === b.length && a.every((value, index) => value === b[index]); + +/** + * Builds a fixture-replay executor. Every probe returns the matching fixture + * result; an unmatched probe resolves to a not-found ProbeResult so a + * missing fixture behaves like a missing binary rather than throwing. + * + * @param table — Captured fixtures. + * @param unmatched — Optional default result for unmatched probes. + */ +export const createFixtureExecutor = (options: { + readonly table: FixtureTable; + readonly unmatched?: ProbeResult; +}): ProbeExecutor => { + const { table, unmatched } = options; + const fallback: ProbeResult = unmatched ?? { + ok: false, + reason: 'not-found', + detail: 'no fixture for this probe', + }; + + return { + async run(command, args) { + const fixture = table.commands.find( + (entry) => entry.command === command && sameArgs(entry.args ?? [], [...args]), + ); + return fixture?.result ?? fallback; + }, + async readTextFile(path) { + const fixture = table.files.find((entry) => entry.path === path); + return fixture?.result ?? fallback; + }, + async statfs(path) { + const fixture = table.statfs.find((entry) => entry.path === path); + return fixture?.result ?? { ok: false }; + }, + }; +}; diff --git a/packages/shared/local-ai/src/lib/manifest.test.ts b/packages/shared/local-ai/src/lib/manifest.test.ts new file mode 100644 index 00000000..896f2b84 --- /dev/null +++ b/packages/shared/local-ai/src/lib/manifest.test.ts @@ -0,0 +1,250 @@ +// packages/shared/local-ai/src/lib/manifest.test.ts +import { describe, expect, test } from 'bun:test'; +import { createFixtureExecutor } from './fixture_executor.ts'; +import { loadManifest, parseManifest } from './manifest.ts'; + +const MANIFEST_JSON = JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-qwen2.5-1.5b-instruct-q4km', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'bartowski/Qwen2.5-1.5B-Instruct-GGUF', + revision: 'rev', + file: 'model.gguf', + targetPath: 'text/qwen.gguf', + bytes: 986048768, + sha256: 'a'.repeat(64), + }, + ], +}); + +describe('parseManifest', () => { + test('parses a valid manifest', () => { + const manifest = parseManifest(MANIFEST_JSON); + expect(manifest.schemaVersion).toBe(1); + expect(manifest.entries).toHaveLength(1); + expect(manifest.entries[0]?.id).toBe('text-qwen2.5-1.5b-instruct-q4km'); + }); + + test('rejects invalid JSON', () => { + expect(() => parseManifest('{not json')).toThrow(/invalid manifest JSON/); + }); + + test('rejects a wrong schema version', () => { + expect(() => parseManifest('{"schemaVersion":2,"entries":[]}')).toThrow(/does not match/); + }); + + test('rejects an entry missing required fields', () => { + expect(() => + parseManifest(JSON.stringify({ schemaVersion: 1, entries: [{ id: 'x' }] })), + ).toThrow(/does not match/); + }); + + test('rejects a file entry without a source (no url and no repo coordinates)', () => { + expect(() => + parseManifest( + JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-no-source', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + targetPath: 'text/m.gguf', + bytes: 100, + sha256: 'a'.repeat(64), + }, + ], + }), + ), + ).toThrow(/does not match/); + }); + + test('rejects a file entry with partial repo coordinates', () => { + expect(() => + parseManifest( + JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-partial-repo', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'owner/repo', + revision: 'rev', + targetPath: 'text/m.gguf', + bytes: 100, + sha256: 'a'.repeat(64), + }, + ], + }), + ), + ).toThrow(/does not match/); + }); + + test('rejects an archive entry without a url', () => { + expect(() => + parseManifest( + JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'voice-no-url', + modality: 'tts', + tier: 'any', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'archive', + targetPath: 'tts/kokoro', + bytes: 100, + sha256: 'a'.repeat(64), + }, + ], + }), + ), + ).toThrow(/does not match/); + }); + + test('rejects negative bytes', () => { + expect(() => + parseManifest( + JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-negative-bytes', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'owner/repo', + revision: 'rev', + file: 'm.gguf', + targetPath: 'text/m.gguf', + bytes: -1, + sha256: 'a'.repeat(64), + }, + ], + }), + ), + ).toThrow(/does not match/); + }); + + test('rejects a non-integer byte count', () => { + expect(() => + parseManifest( + JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-float-bytes', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'owner/repo', + revision: 'rev', + file: 'm.gguf', + targetPath: 'text/m.gguf', + bytes: 1.5, + sha256: 'a'.repeat(64), + }, + ], + }), + ), + ).toThrow(/does not match/); + }); + + test('rejects a sha256 that is not 64 hex characters', () => { + expect(() => + parseManifest( + JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-bad-sha', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'owner/repo', + revision: 'rev', + file: 'm.gguf', + targetPath: 'text/m.gguf', + bytes: 100, + sha256: 'zz-not-hex', + }, + ], + }), + ), + ).toThrow(/does not match/); + }); + + test('accepts a file entry with a direct url (no repo coordinates)', () => { + const raw = JSON.stringify({ + schemaVersion: 1, + entries: [ + { + id: 'text-url-only', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + url: 'https://example.com/model.gguf', + targetPath: 'text/m.gguf', + bytes: 100, + sha256: 'a'.repeat(64), + }, + ], + }); + const manifest = parseManifest(raw); + expect(manifest.entries[0]?.id).toBe('text-url-only'); + }); +}); + +describe('loadManifest', () => { + test('reads through the executor seam', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [], + files: [ + { + path: '/m/manifest.json', + result: { ok: true, stdout: MANIFEST_JSON, stderr: '', exitCode: 0 }, + }, + ], + statfs: [], + }, + }); + const manifest = await loadManifest({ executor, path: '/m/manifest.json' }); + expect(manifest.entries[0]?.id).toBe('text-qwen2.5-1.5b-instruct-q4km'); + }); + + test('throws when the read fails', async () => { + const executor = createFixtureExecutor({ + table: { + commands: [], + files: [{ path: '/m/missing.json', result: { ok: false, reason: 'not-found' } }], + statfs: [], + }, + }); + await expect(loadManifest({ executor, path: '/m/missing.json' })).rejects.toThrow( + /manifest read failed/, + ); + }); +}); diff --git a/packages/shared/local-ai/src/lib/manifest.ts b/packages/shared/local-ai/src/lib/manifest.ts new file mode 100644 index 00000000..430adf6e --- /dev/null +++ b/packages/shared/local-ai/src/lib/manifest.ts @@ -0,0 +1,54 @@ +// packages/shared/local-ai/src/lib/manifest.ts +// +// Loads and validates C-390's models.manifest.json. Pure: accepts a JSON +// string or reads through the injected ProbeExecutor seam (never node:fs). +// The manifest is C-390's — this contract only reads it. + +import { ModelManifestSchema } from '@aikami/schemas'; +import type { ModelManifest } from '@aikami/types'; +import { Value } from 'typebox/value'; +import type { ProbeExecutor, ProbeResult } from './probe_executor.ts'; + +/** + * Parses and validates raw manifest JSON. + * + * @param raw — Raw JSON text of models.manifest.json. + * @returns The validated manifest. + * @throws Error when the JSON is invalid or does not match the manifest schema. + */ +export const parseManifest = (raw: string): ModelManifest => { + let parsed: unknown; + try { + parsed = JSON.parse(raw); + } catch (error) { + throw new Error( + `invalid manifest JSON: ${error instanceof Error ? error.message : String(error)}`, + ); + } + const checked = Value.Check(ModelManifestSchema, parsed); + if (!checked) { + // TypeBox 1.3.12 reports errors as an array; the first error's + // instancePath identifies the failing field (e.g. /entries/0/sha256). + const firstError = [...Value.Errors(ModelManifestSchema, parsed)][0]; + const detail = firstError ? ` (${firstError.instancePath || '/'}: ${firstError.message})` : ''; + throw new Error(`manifest does not match the C-390 models.manifest.json schema${detail}`); + } + return parsed as ModelManifest; +}; + +/** + * Loads the manifest through a ProbeExecutor's readTextFile seam. The + * adapter owns the actual filesystem; the core stays portable. + * + * @param options — executor and the manifest path. + */ +export const loadManifest = async (options: { + readonly executor: ProbeExecutor; + readonly path: string; +}): Promise => { + const result: ProbeResult = await options.executor.readTextFile(options.path); + if (!result.ok) { + throw new Error(`manifest read failed (${result.reason}): ${options.path}`); + } + return parseManifest(result.stdout); +}; diff --git a/packages/shared/local-ai/src/lib/probe_executor.contract_suite.ts b/packages/shared/local-ai/src/lib/probe_executor.contract_suite.ts new file mode 100644 index 00000000..af1e5d0b --- /dev/null +++ b/packages/shared/local-ai/src/lib/probe_executor.contract_suite.ts @@ -0,0 +1,139 @@ +// packages/shared/local-ai/src/lib/probe_executor.contract_suite.ts +// +// Shared ProbeExecutor contract suite (AC-0c). Run against EVERY adapter: +// the Bun/CLI one (local-stack tests) and the fixture-replay one (this +// package's tests). Exercises the six guarantees from the C-391 design +// reference: +// +// - missing binary → resolves { ok:false, reason:'not-found' } +// - non-zero exit → resolves { ok:false, reason:'failed' } +// - hanging process → settles within timeoutMs and is killed +// - permission denial → resolves { ok:false, reason:'denied' } +// - byte-faithful stdout → returned undecorated +// - never throws → every case resolves, never rejects +// +// Platform independence: command fixtures use the current process +// executable (`process.execPath` + `-e`) instead of POSIX-only sh/sleep/ +// printf, so the suite runs identically on Linux, macOS, and Windows. The +// permission-denied case is driven by an explicit `permissionDeniedPath` +// capability; adapters that cannot produce a deterministic denial omit it +// and that test is skipped. + +import { describe, expect, test } from 'bun:test'; +import type { ProbeExecutor } from './probe_executor.ts'; + +export type ExecutorFactory = () => ProbeExecutor; + +// The suite reads its own source file for the readTextFile case. Computed +// without node:path so the AC-0 boundary test (no Node builtins in the +// public graph) stays green; both the Bun/CLI adapter and the fixture +// adapter live in this same directory. +const SELF_PATH = `${import.meta.dir}/probe_executor.contract_suite.ts`; + +/** + * Runs the full contract suite against one adapter. Call this from each + * adapter's own test file. + * + * @param label — Adapter label for describe(). + * @param factory — Builds a fresh executor per test. + * @param permissionDeniedPath — Deterministic path whose read yields + * { ok:false, reason:'denied' }. Omit to skip the permission-denied test. + */ +export const runProbeExecutorContractSuite = (options: { + readonly label: string; + readonly factory: ExecutorFactory; + readonly permissionDeniedPath?: string; +}): void => { + const { label, factory, permissionDeniedPath } = options; + + describe(`ProbeExecutor contract — ${label}`, () => { + test('missing binary resolves not-found, never rejects', async () => { + const executor = factory(); + const result = await executor.run('definitely-not-a-real-binary-xyz', [], { + timeoutMs: 500, + }); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.reason).toBe('not-found'); + } + }); + + test('non-zero exit resolves failed with byte-faithful stderr', async () => { + const executor = factory(); + const result = await executor.run( + process.execPath, + ['-e', 'console.error("boom"); process.exit(3)'], + { timeoutMs: 500 }, + ); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.reason).toBe('failed'); + expect(result.detail).toBeString(); + } + }); + + test('hanging process settles within timeoutMs and is killed', async () => { + const executor = factory(); + const start = Date.now(); + const result = await executor.run(process.execPath, ['-e', 'setTimeout(() => {}, 5000)'], { + timeoutMs: 300, + }); + const elapsed = Date.now() - start; + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.reason).toBe('timeout'); + } + // Allow generous scheduling slack, but nowhere near the 5 s sleep. + expect(elapsed).toBeLessThan(2500); + }); + + test('stdout is returned byte-faithfully', async () => { + const executor = factory(); + const payload = ' RTX 4090, 24564 MiB, 570.00 \nSecond line \n'; + const result = await executor.run( + process.execPath, + ['-e', 'process.stdout.write(process.argv[1])', payload], + { timeoutMs: 500 }, + ); + expect(result.ok).toBe(true); + if (result.ok) { + expect(result.stdout).toBe(payload); + } + }); + + test('readTextFile returns contents without trimming', async () => { + const executor = factory(); + // This file is guaranteed to exist and end with a newline wherever the + // suite runs (both the Bun/CLI adapter and the fixture adapter). + const result = await executor.readTextFile(SELF_PATH); + // Assert ok FIRST: a failed read must fail the test rather than + // skipping the newline assertion below. + expect(result.ok).toBe(true); + if (result.ok) { + expect(result.stdout.endsWith('\n')).toBe(true); + } + }); + + test('statfs resolves a shape, never throws', async () => { + const executor = factory(); + const result = await executor.statfs('/fixture'); + if ('freeBytes' in result) { + expect(result.freeBytes).toBeGreaterThanOrEqual(0); + } else { + expect(result.ok).toBe(false); + } + }); + + test.skipIf(permissionDeniedPath === undefined)( + 'permission-denied path resolves denied, not failed', + async () => { + const executor = factory(); + const result = await executor.readTextFile(permissionDeniedPath ?? ''); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.reason).toBe('denied'); + } + }, + ); + }); +}; diff --git a/packages/shared/local-ai/src/lib/probe_executor.ts b/packages/shared/local-ai/src/lib/probe_executor.ts new file mode 100644 index 00000000..68ff7493 --- /dev/null +++ b/packages/shared/local-ai/src/lib/probe_executor.ts @@ -0,0 +1,61 @@ +// packages/shared/local-ai/src/lib/probe_executor.ts +// +// The injected boundary between the portable planning core and a host's +// process/filesystem access. The core NEVER spawns a process itself — it +// declares this interface and receives an implementation (Bun/CLI, Tauri, +// fixture-replay). See the contract table in the C-391 design reference: +// +// - Signature: run(command, args, { timeoutMs }): Promise +// - No shell: command and args are a fixed array; an adapter never builds +// or evaluates a shell string. +// - Never throws: missing binary, non-zero exit, timeout, permission +// denial all resolve to a ProbeResult with ok:false and a discriminated +// reason. Rejection is reserved for adapter bugs. +// - Honours the timeout: the promise settles within timeoutMs; the child +// is killed, not abandoned. +// - Byte-faithful: stdout/stderr returned undecorated — no trimming, +// locale translation, or colour stripping. +// - Side-effect free: probes are read-only. +// - Filesystem reads (readTextFile, statfs) are part of the same seam so +// /proc/meminfo and free-disk checks are stubbable identically. + +/** + * Discriminated result of a single probe. `ok: false` never throws — the + * caller decides how to degrade. + */ +export type ProbeResult = + | { + readonly ok: true; + readonly stdout: string; + readonly stderr: string; + readonly exitCode: number; + } + | { + readonly ok: false; + readonly reason: 'not-found' | 'timeout' | 'denied' | 'failed'; + readonly detail?: string; + }; + +/** + * Free-space result against the volume backing a path. + */ +export type StatfsResult = { readonly freeBytes: number } | { readonly ok: false }; + +/** + * The injected boundary every host adapter must satisfy. + */ +export type ProbeExecutor = { + run( + command: string, + args: readonly string[], + options: { readonly timeoutMs: number }, + ): Promise; + /** Probing /proc/meminfo and friends — stubbable on the same seam. */ + readTextFile(path: string): Promise; + /** Free-space check against the volume backing a given path. */ + statfs(path: string): Promise; +}; + +/** Convenience: true when a probe succeeded. */ +export const isOk = (result: ProbeResult): result is Extract => + result.ok; diff --git a/packages/shared/local-ai/src/lib/recommend.test.ts b/packages/shared/local-ai/src/lib/recommend.test.ts new file mode 100644 index 00000000..73027d27 --- /dev/null +++ b/packages/shared/local-ai/src/lib/recommend.test.ts @@ -0,0 +1,395 @@ +// packages/shared/local-ai/src/lib/recommend.test.ts +import { describe, expect, test } from 'bun:test'; +import type { HardwareProfile, ModelManifest } from '@aikami/types'; +import { recommend } from './recommend.ts'; + +/** + * A manifest shaped like C-390's models.manifest.json, with the text tiers + * needed for the VRAM table (AC-3) and one image + voice + stt entry each. + */ +const MANIFEST: ModelManifest = { + schemaVersion: 1, + entries: [ + { + id: 'text-qwen2.5-1.5b-instruct-q4km', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'bartowski/Qwen2.5-1.5B-Instruct-GGUF', + revision: 'rev', + file: 'Qwen2.5-1.5B-Instruct-Q4_K_M.gguf', + targetPath: 'text/qwen2.5-1.5b-instruct-q4_k_m.gguf', + bytes: 986_048_768, // ~0.92 GiB — fits any usable budget + sha256: 'a', + }, + { + id: 'text-qwen2.5-7b-instruct-q4km', + modality: 'text', + tier: '8gb', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'bartowski/Qwen2.5-7B-Instruct-GGUF', + revision: 'rev', + file: 'Qwen2.5-7B-Instruct-Q4_K_M.gguf', + targetPath: 'text/qwen2.5-7b-instruct-q4_k_m.gguf', + bytes: 4_683_074_240, // ~4.36 GiB — fits an 8 GB card's usable 5.6 GB + sha256: 'b', + }, + { + id: 'text-mistral-nemo-12b-instruct-q4km', + modality: 'text', + tier: '16gb', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'bartowski/Mistral-Nemo-Instruct-2407-GGUF', + revision: 'rev', + file: 'Mistral-Nemo-Instruct-2407-Q4_K_M.gguf', + targetPath: 'text/mistral-nemo-instruct-2407-q4_k_m.gguf', + bytes: 7_477_208_192, // ~6.96 GiB — fits a 12 GB card's usable 8.4 GB + sha256: 'c', + }, + { + id: 'image-flux1-schnell-q4k', + modality: 'image', + tier: '8gb', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'leejet/FLUX.1-schnell-gguf', + revision: 'rev', + file: 'flux1-schnell-q4_k.gguf', + targetPath: 'image/flux1-schnell-q4_k.gguf', + bytes: 6_884_606_880, // ~6.41 GiB + sha256: 'd', + }, + { + id: 'image-sd15-pruned-q4_0', + modality: 'image', + tier: 'cpu', + license: 'CreativeML OpenRAIL-M', + requiresAcknowledgement: true, + kind: 'file', + repo: 'second-state/stable-diffusion-v1-5-GGUF', + revision: 'rev', + file: 'stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf', + targetPath: 'image/stable-diffusion-v1-5-pruned-emaonly-q4_0.gguf', + bytes: 1_566_768_416, // ~1.46 GiB + sha256: 'e', + }, + { + id: 'tts-kokoro-82m', + modality: 'tts', + tier: 'any', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'archive', + url: 'https://example.com/kokoro.tar.bz2', + targetPath: 'tts/kokoro-multi-lang-v1_0', + bytes: 349_418_188, + sha256: 'f', + }, + { + id: 'stt-moonshine-tiny-en-int8', + modality: 'stt', + tier: 'any', + license: 'MIT', + requiresAcknowledgement: false, + kind: 'archive', + url: 'https://example.com/moonshine.tar.bz2', + targetPath: 'stt/sherpa-onnx-moonshine-tiny-en-int8', + bytes: 107_600_538, + sha256: 'g', + }, + ], +}; + +const profile = (overrides: Partial): HardwareProfile => ({ + platform: 'linux', + arch: 'x64', + gpu: { vendor: 'none', unifiedMemory: false }, + ramMb: 32768, + cores: 8, + freeDiskBytes: 100 * 1024 * 1024 * 1024, + containerRuntime: 'docker', + gpuPassthroughReady: false, + ...overrides, +}); + +const pickText = (plan: { models: { manifestId: string }[] }): string => + plan.models.find((m) => m.modality === 'text')?.manifestId ?? 'none'; + +describe('AC-3 — tier selection respects usable VRAM, not total', () => { + test('4 GB VRAM → cpu tier (Qwen 1.5B)', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 3050', vramMb: 4096, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(pickText(plan)).toBe('text-qwen2.5-1.5b-instruct-q4km'); + }); + + test('8 GB VRAM → 8gb tier (Qwen 7B)', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 3060', vramMb: 8192, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(pickText(plan)).toBe('text-qwen2.5-7b-instruct-q4km'); + }); + + test('12 GB VRAM → 8gb tier unless the 16gb entry fits usable 8.4 GB (top-tier fallback warns)', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 4070', vramMb: 12288, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + // 12 GB * 0.7 = 8.4 GB usable. Mistral (6.96 GiB) fits inside 8.4 GB → + // the 16gb-tier entry is selected, and the top-tier fallback warns. + expect(pickText(plan)).toBe('text-mistral-nemo-12b-instruct-q4km'); + expect(plan.warnings.some((w) => w.includes('nominal 8gb') || w.includes('tight fit'))).toBe( + true, + ); + }); + + test('24 GB VRAM → 16gb tier', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 4090', vramMb: 24576, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(pickText(plan)).toBe('text-mistral-nemo-12b-instruct-q4km'); + // 24 GB is nominally 16gb — no top-tier fallback warning. + expect(plan.warnings.some((w) => w.includes('nominal'))).toBe(false); + }); +}); + +describe('tierOverride — explicit tier pinning (--tier)', () => { + test('tierOverride cpu on a 24 GB VRAM profile selects the CPU-tier model without a nominal-tier warning', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 4090', vramMb: 24576, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + tierOverride: 'cpu', + }); + // 24 GB is nominally 16gb, but the override pins selection to the cpu + // tier (Qwen 1.5B) and suppresses the top-tier fallback warning. + expect(pickText(plan)).toBe('text-qwen2.5-1.5b-instruct-q4km'); + expect(plan.warnings.some((w) => w.includes('nominal') || w.includes('tight fit'))).toBe(false); + }); +}); + +describe('backendOverride metal on a non-macOS host', () => { + test('emits the expected warning while setting nativeEngines to true', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 4070', vramMb: 12288, unifiedMemory: false }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + backendOverride: 'metal', + }); + expect(plan.backend).toBe('metal'); + expect(plan.nativeEngines).toBe(true); + expect(plan.warnings.some((w) => w.includes('--backend metal requested'))).toBe(true); + }); +}); + +describe('AC-2 — NVIDIA detection selects the matching CUDA image', () => { + test('CUDA 12 driver → backend cuda, cudaMajor 12', () => { + const plan = recommend({ + profile: profile({ + gpu: { + vendor: 'nvidia', + name: 'RTX 4070', + vramMb: 12288, + cudaMajor: 12, + unifiedMemory: false, + }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(plan.backend).toBe('cuda'); + expect(plan.nativeEngines).toBe(false); + }); + + test('CUDA 13 driver → backend cuda, cudaMajor 13', () => { + const plan = recommend({ + profile: profile({ + gpu: { + vendor: 'nvidia', + name: 'RTX 5070', + vramMb: 12288, + cudaMajor: 13, + unifiedMemory: false, + }, + gpuPassthroughReady: true, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(plan.backend).toBe('cuda'); + }); +}); + +describe('AC-12 — missing GPU passthrough is caught, not assumed', () => { + test('NVIDIA GPU without toolkit falls back to cpu with a warning', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'nvidia', name: 'RTX 4070', vramMb: 12288, unifiedMemory: false }, + gpuPassthroughReady: false, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(plan.backend).toBe('cpu'); + expect(plan.warnings.some((w) => w.includes('NVIDIA Container Toolkit'))).toBe(true); + }); + + test('explicit --backend cuda on no NVIDIA GPU obeys with a loud warning', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'none', unifiedMemory: false }, + gpuPassthroughReady: false, + }), + modalities: ['text'], + manifest: MANIFEST, + backendOverride: 'cuda', + }); + expect(plan.backend).toBe('cuda'); + expect(plan.warnings.some((w) => w.includes('--backend cuda requested'))).toBe(true); + }); +}); + +describe('AC-4 — unified memory is not treated as VRAM', () => { + test('Apple Silicon 16 GB unified → usable 8 GB, nativeEngines true', () => { + const plan = recommend({ + profile: profile({ + platform: 'darwin', + arch: 'arm64', + gpu: { vendor: 'apple', unifiedMemory: true }, + ramMb: 16384, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + // 50% of 16 GB = 8 GB usable. Mistral (6.96 GiB) fits inside 8 GB, so + // it is selected — but only because the 50% rule says 8 GB usable, and + // 8 GB usable is nominally the 8gb tier, so the 16gb pick warns as a + // top-tier fallback. Had the planner treated all 16 GB as free, usable + // would be 11.2 GB (70%), nominal tier would be 16gb, and no warning + // would appear. The warning therefore proves the 50% rule held. + expect(plan.backend).toBe('metal'); + expect(plan.nativeEngines).toBe(true); + const text = plan.models.find((m) => m.modality === 'text'); + expect(text).toBeDefined(); + expect(text?.bytes ?? 0).toBeLessThanOrEqual(8 * 1024 * 1024 * 1024); + expect(plan.warnings.some((w) => w.includes('nominal 8gb'))).toBe(true); + }); +}); + +describe('AC-5 — modality selection controls the download set', () => { + test('--modalities text yields exactly one text model and COMPOSE_PROFILES text', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'none', unifiedMemory: false }, + ramMb: 16384, + }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(plan.models).toHaveLength(1); + expect(plan.models[0]?.modality).toBe('text'); + expect(plan.modalities).toEqual(['text']); + expect(plan.models.some((m) => m.modality === 'image')).toBe(false); + expect(plan.models.some((m) => m.modality === 'voice')).toBe(false); + }); + + test('voice modality selects the tts entry', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'none', unifiedMemory: false }, + ramMb: 16384, + }), + modalities: ['voice'], + manifest: MANIFEST, + }); + expect(plan.models).toHaveLength(1); + expect(plan.models[0]?.manifestId).toBe('tts-kokoro-82m'); + expect(plan.models[0]?.modality).toBe('voice'); + }); + + test('multiple modalities accumulate models and total download', () => { + const plan = recommend({ + profile: profile({ + gpu: { vendor: 'none', unifiedMemory: false }, + ramMb: 8192, + }), + modalities: ['text', 'voice'], + manifest: MANIFEST, + }); + expect(plan.models).toHaveLength(2); + // 8 GB RAM → usable 4 GB → cpu tier → Qwen 1.5B + Kokoro. + const expected = MANIFEST.entries + .filter((e) => e.id === 'text-qwen2.5-1.5b-instruct-q4km' || e.id === 'tts-kokoro-82m') + .reduce((sum, e) => sum + e.bytes, 0); + expect(plan.totalDownloadBytes).toBe(expected); + }); +}); + +describe('AC-1 — no GPU degrades to CPU without error', () => { + test('no GPU tooling → gpu.vendor none, backend cpu, valid plan', () => { + const plan = recommend({ + profile: profile({ gpu: { vendor: 'none', unifiedMemory: false }, ramMb: 8192 }), + modalities: ['text'], + manifest: MANIFEST, + }); + expect(plan.backend).toBe('cpu'); + expect(plan.nativeEngines).toBe(false); + expect(plan.models.length).toBeGreaterThan(0); + }); + + test('image on CPU-only selects the cpu-tier SD1.5 and surfaces its licence', () => { + const plan = recommend({ + profile: profile({ gpu: { vendor: 'none', unifiedMemory: false }, ramMb: 8192 }), + modalities: ['image'], + manifest: MANIFEST, + }); + expect(plan.models[0]?.manifestId).toBe('image-sd15-pruned-q4_0'); + expect(plan.models[0]?.requiresAcknowledgement).toBe(true); + expect(plan.models[0]?.license).toBe('CreativeML OpenRAIL-M'); + }); +}); + +describe('web/ollama/comfyui modalities add no models', () => { + test('web adds no download entries', () => { + const plan = recommend({ + profile: profile({ gpu: { vendor: 'none', unifiedMemory: false }, ramMb: 8192 }), + modalities: ['web'], + manifest: MANIFEST, + }); + expect(plan.models).toHaveLength(0); + expect(plan.totalDownloadBytes).toBe(0); + }); +}); diff --git a/packages/shared/local-ai/src/lib/recommend.ts b/packages/shared/local-ai/src/lib/recommend.ts new file mode 100644 index 00000000..aca51f8d --- /dev/null +++ b/packages/shared/local-ai/src/lib/recommend.ts @@ -0,0 +1,283 @@ +// packages/shared/local-ai/src/lib/recommend.ts +// +// Pure recommendation: (HardwareProfile, StackModality[], ModelManifest) → +// StackPlan. No I/O, no process spawning, no container assumptions — the +// tier table and headroom rules from the C-391 design reference. +// +// Selection rule: usable bytes come from the headroom rule (70% of VRAM for +// dedicated GPUs, 50% of RAM for unified memory / CPU-only); a manifest +// entry is eligible only when bytes ≤ usable; the selection is the largest +// manifest tier whose entry fits. Selecting a tier above the machine's +// nominal tier (per TIER_TABLE) warns as a top-tier fallback — the model +// fits, but barely, and the user deserves to know before downloading 7 GB. + +import type { + GpuVendor, + HardwareProfile, + ModelManifest, + StackModality, + StackPlan, +} from '@aikami/types'; +import type { TierLabel } from './tier_table.ts'; +import { tierForUsable, tierRank, usableBytesForProfile } from './tier_table.ts'; + +export type RecommendOptions = { + readonly profile: HardwareProfile; + readonly modalities: readonly StackModality[]; + readonly manifest: ModelManifest; + /** Explicit backend override (--backend). Defaults to auto-detection. */ + readonly backendOverride?: StackPlan['backend']; + /** + * Explicit tier override (--tier cpu|8gb|16gb). When set, the largest + * entry at or below that tier is selected regardless of the machine's + * nominal tier (CI / power users pinning a specific size). + */ + readonly tierOverride?: 'cpu' | '8gb' | '16gb'; +}; + +/** User-facing modality → manifest modality. `web`/`ollama`/`comfyui` have no models. */ +const MANIFEST_MODALITY: Readonly> = { + text: 'text', + image: 'image', + voice: 'tts', + stt: 'stt', + web: undefined, + ollama: undefined, + comfyui: undefined, +} as const; + +const TIER_ORDER: readonly Extract[] = ['16gb', '8gb', 'cpu']; + +/** Warning text when no container runtime was detected. */ +const NO_RUNTIME_WARNING = + 'No container runtime detected (docker or podman). The stack needs one to run engines; install Docker before `docker compose up`.'; + +/** Converts a size to a human "X.X GB" string for rationale lines. */ +const formatGb = (bytes: number): string => `${(bytes / 1024 / 1024 / 1024).toFixed(1)} GB`; + +/** True when the profile is a unified-memory system (Apple Silicon, iGPU). */ +const isUnifiedMemory = (profile: HardwareProfile): boolean => + profile.gpu.unifiedMemory || profile.gpu.vendor === 'apple'; + +/** + * Picks the best entry for one modality from the manifest under the headroom + * rule. Returns the entry and the tier it resolved to, plus a warning when + * the selection sits above the machine's nominal tier. + */ +const selectEntry = (options: { + readonly manifest: ModelManifest; + readonly manifestModality: string; + readonly usableBytes: number; + readonly nominalTier: TierLabel; + readonly profileName: string; + /** When set, never select above this tier (--tier override). */ + readonly tierCap?: 'cpu' | '8gb' | '16gb'; +}): + | { + readonly entry: ModelManifest['entries'][number]; + readonly warning?: string; + } + | undefined => { + const { manifestModality, usableBytes, nominalTier } = options; + const entries = options.manifest.entries.filter((entry) => entry.modality === manifestModality); + if (entries.length === 0) { + return undefined; + } + + // Any-tier entries (tiny voice/stt archives) are always eligible and are + // the natural pick for their modality when nothing bigger fits. A sole + // oversized any-tier entry must warn exactly like the fallback path below, + // so the user learns about the tight fit before the download starts. + const anyEntry = entries.find((entry) => entry.tier === 'any'); + if (anyEntry && (entries.length === 1 || anyEntry.bytes <= usableBytes)) { + if (anyEntry.bytes > usableBytes) { + return { + entry: anyEntry, + warning: `${options.profileName} model ${anyEntry.id} (${formatGb(anyEntry.bytes)}) does not comfortably fit ${formatGb(usableBytes)} usable — selecting it anyway as the smallest available.`, + }; + } + return { entry: anyEntry }; + } + + // Largest-tier entry that fits inside usable bytes. + for (const tier of TIER_ORDER) { + if (options.tierCap && tierRank(tier) > tierRank(options.tierCap)) { + continue; + } + const fitting = entries + .filter((entry) => entry.tier === tier && entry.bytes <= usableBytes) + .sort((a, b) => b.bytes - a.bytes); + const best = fitting[0]; + if (best) { + if (tierRank(tier) > tierRank(nominalTier)) { + return { + entry: best, + warning: `${options.profileName} model ${best.id} (${formatGb(best.bytes)}) fits in ${formatGb(usableBytes)} usable, but that is above this machine's nominal ${nominalTier} tier — expect a tight fit.`, + }; + } + return { entry: best }; + } + } + + // Nothing fits usable bytes — fall back to the smallest entry and warn. + const smallest = [...entries].sort((a, b) => a.bytes - b.bytes)[0]; + if (smallest) { + return { + entry: smallest, + warning: `${options.profileName} model ${smallest.id} (${formatGb(smallest.bytes)}) does not comfortably fit ${formatGb(usableBytes)} usable — selecting it anyway as the smallest available.`, + }; + } + return undefined; +}; + +/** Selects the backend from the profile unless the user overrode it. */ +const selectBackend = (options: { + readonly profile: HardwareProfile; + readonly override?: StackPlan['backend']; +}): { readonly backend: StackPlan['backend']; readonly warnings: readonly string[] } => { + const { profile, override } = options; + const warnings: string[] = []; + + // The container-runtime warning applies to EVERY container-based backend + // (cpu/cuda/rocm/vulkan/intel/musa), not only CPU-only profiles: a CUDA + // pick with no docker/podman would fail at `up` just the same. Metal is + // the native runtime and never needs the warning. + const warnIfNoRuntime = (backend: StackPlan['backend']): void => { + if (backend !== 'metal' && profile.containerRuntime === 'none') { + warnings.push(NO_RUNTIME_WARNING); + } + }; + + if (override) { + if (override === 'cuda' && profile.gpu.vendor !== 'nvidia' && profile.platform !== 'win32') { + warnings.push( + `--backend cuda requested on a ${profile.gpu.vendor === 'none' ? 'machine with no NVIDIA GPU' : `${profile.gpu.vendor} GPU`} — obeying the override, but the container will not have GPU access.`, + ); + } + if (override === 'metal' && profile.platform !== 'darwin') { + warnings.push( + '--backend metal requested on a non-macOS host — native launchers will not run here.', + ); + } + warnIfNoRuntime(override); + return { backend: override, warnings }; + } + + if (profile.platform === 'darwin') { + return { backend: 'metal', warnings }; + } + + let backend: StackPlan['backend']; + switch (profile.gpu.vendor) { + case 'nvidia': { + if (!profile.gpuPassthroughReady) { + warnings.push( + 'NVIDIA GPU detected but the NVIDIA Container Toolkit is not wired into the container runtime — GPU containers would fail at `up`. Falling back to CPU. Install the toolkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html', + ); + backend = 'cpu'; + } else { + backend = 'cuda'; + } + break; + } + case 'amd': + backend = 'rocm'; + break; + case 'intel': + backend = 'vulkan'; + break; + case 'apple': + backend = 'metal'; + break; + case 'none': + backend = 'cpu'; + break; + } + warnIfNoRuntime(backend); + return { backend, warnings }; +}; + +/** + * Computes the plan: backend, per-modality model picks with rationale, + * total download, and warnings. + * + * @param options — profile, requested modalities, manifest, optional override. + * @returns The validated plan. + */ +export const recommend = (options: RecommendOptions): StackPlan => { + const { profile, modalities, manifest, backendOverride, tierOverride } = options; + + const usableBytes = usableBytesForProfile({ + gpuVendor: profile.gpu.vendor, + vramMb: profile.gpu.vramMb, + ramMb: profile.ramMb, + unifiedMemory: isUnifiedMemory(profile), + }); + const nominalTier = tierOverride ?? tierForUsable(usableBytes); + const warnings: string[] = []; + + const { backend, warnings: backendWarnings } = selectBackend({ + profile, + override: backendOverride, + }); + warnings.push(...backendWarnings); + + const models: StackPlan['models'] = []; + for (const modality of modalities) { + const manifestModality = MANIFEST_MODALITY[modality]; + if (!manifestModality) { + continue; // web / ollama / comfyui — no models to download + } + const picked = selectEntry({ + manifest, + manifestModality, + usableBytes, + nominalTier, + profileName: modality, + tierCap: tierOverride, + }); + if (!picked) { + warnings.push(`no manifest entry for modality ${modality}`); + continue; + } + if (picked.warning) { + warnings.push(picked.warning); + } + const entry = picked.entry; + models.push({ + manifestId: entry.id, + modality, + bytes: entry.bytes, + license: entry.license, + requiresAcknowledgement: entry.requiresAcknowledgement, + rationale: + entry.tier === 'any' + ? 'universal tier — fits anywhere' + : `${formatGb(usableBytes)} usable → tier ${entry.tier} (${formatGb(entry.bytes)})`, + }); + } + + if (models.length === 0 && modalities.length > 0) { + warnings.push('no models selected for the requested modalities'); + } + + const totalDownloadBytes = models.reduce((sum, model) => sum + model.bytes, 0); + const nativeEngines = backend === 'metal'; + + return { + backend, + modalities: [...modalities], + models, + totalDownloadBytes, + warnings, + nativeEngines, + }; +}; + +/** Convenience type guard for the GPU vendor union. */ +export const isGpuVendor = (value: string): value is GpuVendor => + value === 'nvidia' || + value === 'amd' || + value === 'intel' || + value === 'apple' || + value === 'none'; diff --git a/packages/shared/local-ai/src/lib/tier_table.test.ts b/packages/shared/local-ai/src/lib/tier_table.test.ts new file mode 100644 index 00000000..35663558 --- /dev/null +++ b/packages/shared/local-ai/src/lib/tier_table.test.ts @@ -0,0 +1,75 @@ +// packages/shared/local-ai/src/lib/tier_table.test.ts +import { describe, expect, test } from 'bun:test'; +import { TIER_TABLE, tierForUsable, usableBytesForProfile } from './tier_table.ts'; + +describe('TIER_TABLE', () => { + test('is sorted ascending by minUsableBytes', () => { + for (let i = 1; i < TIER_TABLE.length; i += 1) { + expect(TIER_TABLE[i]?.minUsableBytes ?? 0).toBeGreaterThan( + TIER_TABLE[i - 1]?.minUsableBytes ?? 0, + ); + } + }); + + test('starts at cpu with zero usable bytes', () => { + expect(tierForUsable(0)).toBe('cpu'); + }); + + const Gib = 1024 * 1024 * 1024; + + test('exactly 4 GiB usable reaches the 8gb tier', () => { + expect(tierForUsable(4 * Gib)).toBe('8gb'); + }); + + test('exactly 10 GiB usable reaches the 16gb tier', () => { + expect(tierForUsable(10 * Gib)).toBe('16gb'); + }); + + test('one byte below 10 GiB remains in the 8gb tier', () => { + expect(tierForUsable(10 * Gib - 1)).toBe('8gb'); + }); +}); + +describe('usableBytesForProfile', () => { + test('dedicated GPU uses 70% of VRAM', () => { + const usable = usableBytesForProfile({ + gpuVendor: 'nvidia', + vramMb: 12282, + ramMb: 32768, + unifiedMemory: false, + }); + // 12282 MiB * 0.7 + expect(usable).toBe(Math.floor(12282 * 1024 * 1024 * 0.7)); + }); + + test('unified memory uses 50% of total RAM', () => { + const usable = usableBytesForProfile({ + gpuVendor: 'apple', + ramMb: 16384, + unifiedMemory: true, + }); + // 16 GiB * 0.5 + expect(usable).toBe(Math.floor(16384 * 1024 * 1024 * 0.5)); + }); + + test('CPU-only falls back to unified-memory sizing on system RAM', () => { + const usable = usableBytesForProfile({ + gpuVendor: 'none', + ramMb: 16384, + unifiedMemory: false, + }); + expect(usable).toBe(Math.floor(16384 * 1024 * 1024 * 0.5)); + }); + + test('NVIDIA profile with vramMb 0 falls back to unified-memory sizing on system RAM', () => { + const usable = usableBytesForProfile({ + gpuVendor: 'nvidia', + vramMb: 0, + ramMb: 16384, + unifiedMemory: false, + }); + // A 0-MiB VRAM report (probe failed / driverless) must not size models + // against 0 usable bytes — fall back to the RAM calculation. + expect(usable).toBe(Math.floor(16384 * 1024 * 1024 * 0.5)); + }); +}); diff --git a/packages/shared/local-ai/src/lib/tier_table.ts b/packages/shared/local-ai/src/lib/tier_table.ts new file mode 100644 index 00000000..dfc9b861 --- /dev/null +++ b/packages/shared/local-ai/src/lib/tier_table.ts @@ -0,0 +1,104 @@ +// packages/shared/local-ai/src/lib/tier_table.ts +// +// Tier thresholds live IN CODE as a typed constant, per the C-391 design +// reference — C-390's manifest keeps its per-entry `tier` labels and is not +// edited by this contract. The manifest vocabulary (`cpu` / `8gb` / `16gb` / +// `any`) is what this table maps onto. +// +// Headroom rule: usable VRAM = 70% of reported VRAM for dedicated GPUs; +// usable memory = 50% of total RAM for unified-memory systems (Apple +// Silicon, iGPUs). A manifest entry is eligible only when its file size ≤ +// usable bytes. The thresholds below define when a given usable size reaches +// each tier; the manifest entry itself still has to fit inside usable bytes. + +const GIB = 1024 * 1024 * 1024; + +export type TierLabel = 'cpu' | '8gb' | '16gb' | 'any'; + +export type TierRow = { + /** Minimum usable bytes for this tier to be reachable. */ + readonly minUsableBytes: number; + readonly tier: Extract; +}; + +/** + * Usable-memory thresholds for each model tier. Sorted ascending. + * + * Calibrated so a card is nominally in the tier whose smallest model it can + * hold comfortably: 4 GiB usable (≈ an 8 GB card after 70% headroom, and + * enough for the smallest 8gb-tier model) reaches `8gb`; 10 GiB usable + * (≈ a 16 GB card, comfortably above the largest shipped 16gb-tier model) + * reaches `16gb`. A 12 GB card (8.4 GB usable) stays nominal `8gb`, so a + * 16gb-tier pick there warns as a top-tier fallback (AC-3). + */ +export const TIER_TABLE: readonly TierRow[] = [ + { minUsableBytes: 0, tier: 'cpu' }, + { minUsableBytes: 4 * GIB, tier: '8gb' }, + { minUsableBytes: 10 * GIB, tier: '16gb' }, +] as const; + +/** Fraction of reported VRAM treated as usable on dedicated GPUs. */ +export const DEDICATED_GPU_HEADROOM = 0.7; +/** Fraction of total memory treated as usable on unified-memory systems. */ +export const UNIFIED_MEMORY_HEADROOM = 0.5; + +/** + * Maps usable bytes to the largest tier reachable at or below that size. + * The `any` tier is always reachable (tiny voice/stt models). + * + * @param usableBytes — Usable VRAM or usable unified memory. + * @returns The tier label the usable size nominally supports. + */ +export const tierForUsable = (usableBytes: number): TierLabel => { + let tier: TierLabel = 'cpu'; + for (const row of TIER_TABLE) { + if (usableBytes >= row.minUsableBytes) { + tier = row.tier; + } + } + return tier; +}; + +/** + * Orders tier labels for "largest first" iteration. `any` sorts below the + * fixed tiers — a dedicated entry always wins over the universal fallback. + */ +export const tierRank = (tier: TierLabel): number => { + switch (tier) { + case '16gb': + return 3; + case '8gb': + return 2; + case 'cpu': + return 1; + case 'any': + return 0; + } +}; + +/** + * Usable bytes for a hardware profile: dedicated GPUs use 70% of VRAM + * (headroom for the compositor and a busy desktop); unified-memory systems + * use 50% of total RAM (shared with the OS). A CPU-only profile (no GPU) + * sizes models against usable system RAM so the CPU backend still picks a + * sane default. + * + * @returns Usable bytes for model sizing. + */ +export const usableBytesForProfile = (options: { + readonly gpuVendor: 'nvidia' | 'amd' | 'intel' | 'apple' | 'none'; + readonly vramMb?: number; + readonly ramMb: number; + readonly unifiedMemory: boolean; +}): number => { + const { gpuVendor, vramMb, ramMb, unifiedMemory } = options; + if (unifiedMemory) { + return Math.floor(ramMb * 1024 * 1024 * UNIFIED_MEMORY_HEADROOM); + } + if (gpuVendor !== 'none' && vramMb !== undefined && vramMb > 0) { + return Math.floor(vramMb * 1024 * 1024 * DEDICATED_GPU_HEADROOM); + } + // CPU-only: models live in system RAM; use the same unified headroom so a + // 16 GB machine does not try to fit a 12 GB model into swap. + return Math.floor(ramMb * 1024 * 1024 * UNIFIED_MEMORY_HEADROOM); +}; diff --git a/packages/shared/local-ai/tsconfig.json b/packages/shared/local-ai/tsconfig.json new file mode 100644 index 00000000..9694404c --- /dev/null +++ b/packages/shared/local-ai/tsconfig.json @@ -0,0 +1,17 @@ +{ + "$schema": "https://json.schemastore.org/tsconfig", + "display": "Local AI", + "extends": "../../../config/tsconfig/tsconfig.base.json", + "compilerOptions": { + "rootDir": "..", + "outDir": "dist", + "paths": { + "@aikami/types": ["../types/src/index.ts"], + "@aikami/types/*": ["../types/src/lib/*"], + "@aikami/schemas": ["../schemas/src/index.ts"], + "@aikami/schemas/*": ["../schemas/src/lib/*"] + } + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist", "**/*.test.ts", "**/*.test.tsx", "**/*.test.js"] +} diff --git a/packages/shared/schemas/src/index.ts b/packages/shared/schemas/src/index.ts index 3fcb3527..78881cd7 100644 --- a/packages/shared/schemas/src/index.ts +++ b/packages/shared/schemas/src/index.ts @@ -53,6 +53,10 @@ export * from './lib/game/relationship_state.ts'; export * from './lib/game/rules_command.ts'; export * from './lib/game/status_effect.ts'; export * from './lib/game/swarm_handoff.ts'; +export * from './lib/local_ai/hardware_profile.ts'; +export * from './lib/local_ai/model_manifest.ts'; +export * from './lib/local_ai/stack_backend.ts'; +export * from './lib/local_ai/stack_plan.ts'; export * from './lib/logging/index.ts'; export * from './lib/media/audio_track_catalog.ts'; export * from './lib/media/image_engine.ts'; diff --git a/packages/shared/schemas/src/lib/local_ai/hardware_profile.ts b/packages/shared/schemas/src/lib/local_ai/hardware_profile.ts new file mode 100644 index 00000000..3ca0a59a --- /dev/null +++ b/packages/shared/schemas/src/lib/local_ai/hardware_profile.ts @@ -0,0 +1,50 @@ +// packages/shared/schemas/src/lib/local_ai/hardware_profile.ts +import Type from 'typebox'; +import { StackBackendSchema, StackModalitySchema } from './stack_backend.ts'; + +export const GpuVendorSchema = Type.Union([ + Type.Literal('nvidia'), + Type.Literal('amd'), + Type.Literal('intel'), + Type.Literal('apple'), + Type.Literal('none'), +]); + +export const PlatformSchema = Type.Union([ + Type.Literal('linux'), + Type.Literal('darwin'), + Type.Literal('win32'), +]); + +export const ArchSchema = Type.Union([Type.Literal('x64'), Type.Literal('arm64')]); + +export const ContainerRuntimeSchema = Type.Union([ + Type.Literal('docker'), + Type.Literal('podman'), + Type.Literal('none'), +]); + +export const CudaMajorSchema = Type.Union([Type.Literal(12), Type.Literal(13)]); + +export const HardwareProfileSchema = Type.Object({ + platform: PlatformSchema, + arch: ArchSchema, + gpu: Type.Object({ + vendor: GpuVendorSchema, + name: Type.Optional(Type.String()), + vramMb: Type.Optional(Type.Number()), + /** NVIDIA only — decides server-cuda vs server-cuda13. */ + cudaMajor: Type.Optional(CudaMajorSchema), + /** True when the GPU shares system RAM (Apple Silicon, iGPU). */ + unifiedMemory: Type.Boolean(), + }), + ramMb: Type.Number(), + cores: Type.Number(), + freeDiskBytes: Type.Number(), + containerRuntime: ContainerRuntimeSchema, + /** NVIDIA Container Toolkit detected — GPU containers will actually work. */ + gpuPassthroughReady: Type.Boolean(), +}); + +export const StackModalitiesInputSchema = Type.Array(StackModalitySchema); +export const StackBackendInputSchema = Type.Optional(StackBackendSchema); diff --git a/packages/shared/schemas/src/lib/local_ai/model_manifest.test.ts b/packages/shared/schemas/src/lib/local_ai/model_manifest.test.ts new file mode 100644 index 00000000..5b9183e7 --- /dev/null +++ b/packages/shared/schemas/src/lib/local_ai/model_manifest.test.ts @@ -0,0 +1,113 @@ +// packages/shared/schemas/src/lib/local_ai/model_manifest.test.ts +// +// Schema validation tests for C-391's manifest entry schema: source +// requirements (file needs url or full repo coordinates; archive needs +// url), byte-count integrity (non-negative integer), and sha256 format +// (exactly 64 hex characters). + +import { describe, expect, test } from 'bun:test'; +import { Value } from 'typebox/value'; +import { ModelManifestSchema } from './model_manifest.ts'; + +const VALID_FILE_ENTRY = { + id: 'text-qwen', + modality: 'text', + tier: 'cpu', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'file', + repo: 'owner/repo', + revision: 'rev', + file: 'model.gguf', + targetPath: 'text/model.gguf', + bytes: 100, + sha256: 'a'.repeat(64), +}; + +const check = (entry: unknown): boolean => + Value.Check(ModelManifestSchema, { schemaVersion: 1, entries: [entry] }); + +describe('ModelManifestEntrySchema — source requirements', () => { + test('accepts a file entry with full repo coordinates', () => { + expect(check(VALID_FILE_ENTRY)).toBe(true); + }); + + test('accepts a file entry with a direct url', () => { + expect(check({ ...VALID_FILE_ENTRY, url: 'https://example.com/model.gguf' })).toBe(true); + }); + + test('accepts an archive entry with a url', () => { + expect( + check({ + id: 'voice-kokoro', + modality: 'tts', + tier: 'any', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'archive', + url: 'https://example.com/kokoro.tar.bz2', + targetPath: 'tts/kokoro', + bytes: 100, + sha256: 'a'.repeat(64), + }), + ).toBe(true); + }); + + test('rejects a file entry with no url and no repo coordinates', () => { + expect( + check({ ...VALID_FILE_ENTRY, repo: undefined, revision: undefined, file: undefined }), + ).toBe(false); + }); + + test('rejects a file entry with only partial repo coordinates', () => { + expect(check({ ...VALID_FILE_ENTRY, revision: undefined })).toBe(false); + }); + + test('rejects an archive entry without a url', () => { + expect( + check({ + id: 'voice-kokoro', + modality: 'tts', + tier: 'any', + license: 'Apache-2.0', + requiresAcknowledgement: false, + kind: 'archive', + targetPath: 'tts/kokoro', + bytes: 100, + sha256: 'a'.repeat(64), + }), + ).toBe(false); + }); +}); + +describe('ModelManifestEntrySchema — byte-count integrity', () => { + test('rejects negative bytes', () => { + expect(check({ ...VALID_FILE_ENTRY, bytes: -1 })).toBe(false); + }); + + test('rejects a non-integer byte count', () => { + expect(check({ ...VALID_FILE_ENTRY, bytes: 1.5 })).toBe(false); + }); + + test('accepts zero bytes', () => { + expect(check({ ...VALID_FILE_ENTRY, bytes: 0 })).toBe(true); + }); +}); + +describe('ModelManifestEntrySchema — sha256 integrity', () => { + test('accepts exactly 64 lowercase hex characters', () => { + expect(check({ ...VALID_FILE_ENTRY, sha256: 'a'.repeat(64) })).toBe(true); + }); + + test('accepts exactly 64 uppercase hex characters', () => { + expect(check({ ...VALID_FILE_ENTRY, sha256: 'A'.repeat(64) })).toBe(true); + }); + + test('rejects a sha256 that is not 64 characters', () => { + expect(check({ ...VALID_FILE_ENTRY, sha256: 'a'.repeat(63) })).toBe(false); + }); + + test('rejects a sha256 with non-hex characters', () => { + expect(check({ ...VALID_FILE_ENTRY, sha256: `${'a'.repeat(63)}z` })).toBe(false); + }); +}); diff --git a/packages/shared/schemas/src/lib/local_ai/model_manifest.ts b/packages/shared/schemas/src/lib/local_ai/model_manifest.ts new file mode 100644 index 00000000..4a7a746c --- /dev/null +++ b/packages/shared/schemas/src/lib/local_ai/model_manifest.ts @@ -0,0 +1,68 @@ +// packages/shared/schemas/src/lib/local_ai/model_manifest.ts +import Type from 'typebox'; + +/** + * C-390's `models.manifest.json` schema (schemaVersion 1). The manifest is + * owned by C-390 — C-391 reads it, never edits it. Tier labels are the + * vocabulary the C-391 tier table maps onto: `cpu` / `8gb` / `16gb` / `any`. + */ +export const ManifestEntryModalitySchema = Type.Union([ + Type.Literal('text'), + Type.Literal('image'), + Type.Literal('tts'), + Type.Literal('stt'), +]); + +export const ManifestEntryTierSchema = Type.Union([ + Type.Literal('cpu'), + Type.Literal('8gb'), + Type.Literal('16gb'), + Type.Literal('any'), +]); + +/** + * Source discriminated by kind. A `file` entry must declare either a direct + * `url` or the full HuggingFace repo coordinates (repo + revision + file); + * an `archive` entry must declare a direct `url`. Extra properties are + * tolerated (e.g. a file with both url and repo coords), but a source that + * satisfies none of the variants fails validation. + */ +const ManifestEntrySourceSchema = Type.Union([ + Type.Object({ + kind: Type.Literal('file'), + /** file kind: HuggingFace repo (repo/revision/file) or direct url override */ + repo: Type.String(), + revision: Type.String(), + file: Type.String(), + url: Type.Optional(Type.String()), + }), + Type.Object({ + kind: Type.Literal('file'), + /** file kind: direct url override */ + url: Type.String(), + }), + Type.Object({ + kind: Type.Literal('archive'), + /** archive kind: direct download url */ + url: Type.String(), + }), +]); + +export const ModelManifestEntrySchema = Type.Intersect([ + Type.Object({ + id: Type.String(), + modality: ManifestEntryModalitySchema, + tier: ManifestEntryTierSchema, + license: Type.String(), + requiresAcknowledgement: Type.Boolean(), + targetPath: Type.String(), + bytes: Type.Integer({ minimum: 0 }), + sha256: Type.String({ pattern: '^[0-9a-fA-F]{64}$' }), + }), + ManifestEntrySourceSchema, +]); + +export const ModelManifestSchema = Type.Object({ + schemaVersion: Type.Literal(1), + entries: Type.Array(ModelManifestEntrySchema), +}); diff --git a/packages/shared/schemas/src/lib/local_ai/stack_backend.ts b/packages/shared/schemas/src/lib/local_ai/stack_backend.ts new file mode 100644 index 00000000..509bd65a --- /dev/null +++ b/packages/shared/schemas/src/lib/local_ai/stack_backend.ts @@ -0,0 +1,51 @@ +// packages/shared/schemas/src/lib/local_ai/stack_backend.ts +import Type from 'typebox'; + +/** + * Hardware backend value set introduced by C-391. Value set matches C-390's + * `.env.example` `COMPOSE_FILE` backends (`compose.cpu.yaml`, + * `compose.cuda.yaml`, `compose.rocm.yaml`, `compose.vulkan.yaml`, + * `compose.intel.yaml`, `compose.musa.yaml`) plus `metal` kept from the + * C-390 design reference for the native macOS plan. + */ +export const STACK_BACKENDS = ['cpu', 'cuda', 'rocm', 'vulkan', 'intel', 'musa', 'metal'] as const; + +export type StackBackendValue = (typeof STACK_BACKENDS)[number]; + +/** + * Modality value set introduced by C-391. Matches C-390's `.env.example` + * `COMPOSE_PROFILES` (`text`, `image`, `voice`, `stt`, `web`, `ollama`, + * `comfyui`). Note the manifest (C-390) labels the voice model `tts`; the + * user-facing modality `voice` maps to manifest modality `tts`. + */ +export const STACK_MODALITIES = [ + 'text', + 'image', + 'voice', + 'stt', + 'web', + 'ollama', + 'comfyui', +] as const; + +export type StackModalityValue = (typeof STACK_MODALITIES)[number]; + +// TypeBox's Static inference needs a literal TUPLE inside Type.Union, not +// an array — `.map()` on a const tuple widens to `T[]` and Static collapses +// to `never`. This recursive tuple helper preserves the literal order. +type LiteralTupleOf = T extends readonly [ + infer First extends string, + ...infer Rest extends string[], +] + ? [ReturnType>, ...LiteralTupleOf] + : []; + +const backendSchemas = STACK_BACKENDS.map((backend) => Type.Literal(backend)) as LiteralTupleOf< + typeof STACK_BACKENDS +>; +export const StackBackendSchema = Type.Union(backendSchemas); + +const modalitySchemas = STACK_MODALITIES.map((modality) => + Type.Literal(modality), +) as LiteralTupleOf; +export const StackModalitySchema = Type.Union(modalitySchemas); diff --git a/packages/shared/schemas/src/lib/local_ai/stack_plan.ts b/packages/shared/schemas/src/lib/local_ai/stack_plan.ts new file mode 100644 index 00000000..02a46e6a --- /dev/null +++ b/packages/shared/schemas/src/lib/local_ai/stack_plan.ts @@ -0,0 +1,23 @@ +// packages/shared/schemas/src/lib/local_ai/stack_plan.ts +import Type from 'typebox'; +import { StackBackendSchema, StackModalitySchema } from './stack_backend.ts'; + +export const StackPlanModelSchema = Type.Object({ + manifestId: Type.String(), + modality: StackModalitySchema, + bytes: Type.Number(), + license: Type.String(), + requiresAcknowledgement: Type.Boolean(), + /** One-line human justification shown in the plan. */ + rationale: Type.String(), +}); + +export const StackPlanSchema = Type.Object({ + backend: StackBackendSchema, + modalities: Type.Array(StackModalitySchema), + models: Type.Array(StackPlanModelSchema), + totalDownloadBytes: Type.Number(), + warnings: Type.Array(Type.String()), + /** True when engines must run natively rather than in containers (macOS). */ + nativeEngines: Type.Boolean(), +}); diff --git a/packages/shared/types/src/index.ts b/packages/shared/types/src/index.ts index f41e6c67..51334ae9 100644 --- a/packages/shared/types/src/index.ts +++ b/packages/shared/types/src/index.ts @@ -66,6 +66,10 @@ export * from './lib/game/rules_command.ts'; export * from './lib/game/status_effect.ts'; export * from './lib/game/swarm_handoff.ts'; export * from './lib/game/world_gen.ts'; +export * from './lib/local_ai/hardware_profile.ts'; +export * from './lib/local_ai/model_manifest.ts'; +export * from './lib/local_ai/stack_backend.ts'; +export * from './lib/local_ai/stack_plan.ts'; export * from './lib/media/image_engine.ts'; export * from './lib/media/image_style_profile.ts'; export * from './lib/media/music.ts'; diff --git a/packages/shared/types/src/lib/local_ai/hardware_profile.ts b/packages/shared/types/src/lib/local_ai/hardware_profile.ts new file mode 100644 index 00000000..b482c76f --- /dev/null +++ b/packages/shared/types/src/lib/local_ai/hardware_profile.ts @@ -0,0 +1,8 @@ +// packages/shared/types/src/lib/local_ai/hardware_profile.ts + +import type { HardwareProfileSchema } from '@aikami/schemas'; +import type { Static } from 'typebox'; + +export type HardwareProfile = Static; +export type GpuVendor = HardwareProfile['gpu']['vendor']; +export type CudaMajor = NonNullable; diff --git a/packages/shared/types/src/lib/local_ai/model_manifest.ts b/packages/shared/types/src/lib/local_ai/model_manifest.ts new file mode 100644 index 00000000..64759c5e --- /dev/null +++ b/packages/shared/types/src/lib/local_ai/model_manifest.ts @@ -0,0 +1,9 @@ +// packages/shared/types/src/lib/local_ai/model_manifest.ts + +import type { ModelManifestSchema } from '@aikami/schemas'; +import type { Static } from 'typebox'; + +export type ModelManifest = Static; +export type ModelManifestEntry = Static['entries'][number]; +export type ManifestEntryModality = ModelManifestEntry['modality']; +export type ManifestEntryTier = ModelManifestEntry['tier']; diff --git a/packages/shared/types/src/lib/local_ai/stack_backend.ts b/packages/shared/types/src/lib/local_ai/stack_backend.ts new file mode 100644 index 00000000..62f8b8f8 --- /dev/null +++ b/packages/shared/types/src/lib/local_ai/stack_backend.ts @@ -0,0 +1,7 @@ +// packages/shared/types/src/lib/local_ai/stack_backend.ts + +import type { StackBackendSchema, StackModalitySchema } from '@aikami/schemas'; +import type { Static } from 'typebox'; + +export type StackBackend = Static; +export type StackModality = Static; diff --git a/packages/shared/types/src/lib/local_ai/stack_plan.ts b/packages/shared/types/src/lib/local_ai/stack_plan.ts new file mode 100644 index 00000000..d648325d --- /dev/null +++ b/packages/shared/types/src/lib/local_ai/stack_plan.ts @@ -0,0 +1,7 @@ +// packages/shared/types/src/lib/local_ai/stack_plan.ts + +import type { StackPlanSchema } from '@aikami/schemas'; +import type { Static } from 'typebox'; + +export type StackPlan = Static; +export type StackPlanModel = StackPlan['models'][number]; diff --git a/packages/shared/types/src/lib/runtime/runtime_engine_config.ts b/packages/shared/types/src/lib/runtime/runtime_engine_config.ts index 5a4f9580..4f126378 100644 --- a/packages/shared/types/src/lib/runtime/runtime_engine_config.ts +++ b/packages/shared/types/src/lib/runtime/runtime_engine_config.ts @@ -3,8 +3,8 @@ // Derived runtime engine config types (C-389). The TypeBox schema in // `@aikami/schemas` is the single source of truth; these types are inferred // via `Static<>` so runtime validation and TypeScript stay in lockstep. -import type { Static } from 'typebox'; -import { + +import type { ImageEngineSchema, RuntimeEngineConfigSchema, RuntimeImageConfigSchema, @@ -15,6 +15,7 @@ import { RuntimeVoiceTtsConfigSchema, TtsModeSchema, } from '@aikami/schemas'; +import type { Static } from 'typebox'; export type RuntimeEngineConfig = Static; export type RuntimeTextConfig = Static;