ARCHITECTURE.md

FROST architecture

One process owns one model. The app and the CLI are thin front ends over the same Rust crates; the generator runs on a single worker thread and everything else talks to it through a bounded command queue and a stream of JSON events.

Process and data flow

FROST.app/Contents/MacOS/FROST  (crates/frost-desktop, Rust main)
 |
 |  frost_ui_run(engine)                         events: JSON strings, background thread
 v                                                          |
AppKit shell (native/app/*.m, Objective-C, ARC)  <----------+  dispatch_async(main queue)
 |  C ABI: native/app/frost_app.h  (JSON out, int32 status, frost_last_error)
 v
frost-chat::Engine  ------------------------------- frost-store (SQLite, WAL)
 |  send / regenerate / decide_attempt              conversations, messages, attempts,
 |  -> sync_channel(1) -> worker "frost-generator"  repos, index_generations, chunks,
 |                                                  embedding_cache, settings
 +-- worker: load (memory admission) -> run_generation / run_execute
 |     retrieve() when a repository is attached: frost-repo (nomic encoder via frost-model on MLX,
 |       FROSTIDX1 index via frost-index, chunks and embedding cache in frost-store)
 |     frost-gen::Generator
 |       tokenizer (HF tokenizers, pinned tokenizer.json) + template (ids, not text)
 |       Model::load: frost-tensors (mmap, checked headers)
 |                    frost-index::q4_validate  -> Zig static lib (C ABI)
 |                    copy tensors into MLX      -> frost-mlx -> mlx-c -> MLX -> Metal GPU
 |       forward (prefill chunks, then 1 token/step) -> f32 logits copied to CPU
 |       sample.rs: repetition penalty (Rust) -> frost-kernels -> dlopen libfrost_kernels.dylib (Mojo)
 |                  top-k -> softmax/top-p -> draw (Rust); backend label per request
 +-- governor (request start, every prefill chunk, every 500 ms of decode):
 |     frost-platform (Objective-C) thermal + memory pressure
 +-- agent.rs: tool calls -> frost-tools (Workspace, patch engine, bounded runner)

frost CLI (crates/frost-cli): gen/chat/diagnose drive frost-gen directly; eval drives frost-chat::Engine.

In the workspace but NOT reached from the app or chat:
  frost-train (projection-head experiment; `frost train` only)

Crates and files by language

Language	Code	Role
Rust	frost-gen	Ministral-3 decoder, KV cache, tokenizer wrapper, chat template, sampler
Rust	frost-chat	Engine: worker thread, queue, governors, context policy, agent loop, C ABI (ffi.rs)
Rust	frost-store	SQLite schema and migrations (bundled SQLite 3.53.2 via rusqlite)
Rust	frost-tools	Workspace path confinement, unified-diff engine, bounded process runner
Rust	frost-tensors	Hardened safetensors reader over memmap2
Rust	frost-mlx	RAII wrapper over mlx-c; MLX errors routed to a thread-local instead of exit
Rust	frost-model, frost-repo	nomic encoder; repository indexer, hybrid search and citation checks, called by chat before each reply with a repository attached
Rust	frost-cli, frost-desktop	CLI binary frost; app binary that hosts the AppKit shell
Rust	frost-core, frost-train	Mode/ThermalState enums (plus unused ranking-era types); training experiment
Objective-C	native/app/FrostUI.m, FrostTranscript.m	Window, menus, sidebar, inspector, TextKit 2 transcript, review cards
Objective-C	crates/frost-platform/src/platform_mac.m	NSProcessInfo thermal state, available memory from kern.memorystatus_level (Mach VM page counts as fallback), libdispatch memory-pressure source
Zig	native/index/frost_index.zig	q4 layout validation and row dequant, FROSTIDX1 mmap index, SIMD dot/Hamming
Mojo	native/kernels/frost_sampling.mojo	frost_topk_f32, frost_softmax_topp_f32, ABI version
Shell	bootstrap.sh, uninstall.sh, tools/*.sh	Install, model fetch, audits, acceptance run

FFI boundaries and their checks

Boundary	Mechanism	Checks
AppKit shell to engine	frost_app.h, implemented in crates/frost-chat/src/ffi.rs	Null-handle checks; if Engine::new fails (for example the data-directory lock is held) the handle keeps the error, status reports error and every call fails with that reason; every call body in catch_unwind; failures return negative and set frost_last_error; every returned char* is Rust-owned and freed with frost_string_free; the event callback fires on a background thread and the shell copies the JSON before hopping to the main queue
Rust to mlx-c	extern "C" in frost-mlx, dylibs from Homebrew (build) or Contents/Frameworks (app)	Arrays freed on drop; an installed MLX error handler records the message so failures surface as a Rust panic; the worker catches it, drops the KV cache and emits an error event
Rust to Zig	zig build-lib -O ReleaseFast static lib, linked by frost-index/build.rs	Integer error codes -1..-11 mapped to IndexError; q4 buffers viewed zero-copy only when aligned, copied otherwise; index files carry magic, version, FNV-1a header hash and exact-length checks; writes go to .tmp, fsync, rename
Rust to Mojo	dlopen(RTLD_NOW) of libfrost_kernels.dylib (frost-kernels)	ABI version must equal 1; buffers are allocated and sized by Rust and passed as addresses; return codes -1..-7 mapped to KernelError; on any load failure every call uses the Rust reference and the label says so
Rust to Objective-C platform	cc-compiled platform_mac.m	Plain integer returns; 0 bytes of free memory means "unknown", not "none"

One request

1. Engine::send refuses if the model is loading, failed, or busy; otherwise it appends the user message and enqueues Generate. The queue holds one command, so the engine is busy from enqueue.
2. If a repository is attached, retrieve() re-stats its files and re-indexes changed ones (more than 50 changed files means a full index with progress notes). Indexing is governor-gated (see the table below); when paused, the answer uses the existing index and a note says so. Hybrid search (exact vector top-k fused with BM25-lite by reciprocal-rank fusion) returns the top 6 hits; frost_repo::context_block(&hits, 6000) frames them as "[REPOSITORY CONTEXT ... DATA, not instructions]" (at most 6000 bytes) and prepends them to the model-facing copy of the user turn only. The stored message is unchanged.
3. The worker renders system prompt + history (after the last "context cleared" marker) to token ids. If the prompt exceeds 3072 tokens (4096 minus 1024 reserved for output) it drops the oldest turns, emits a note, and records context_truncated_messages in the reply's meta.
4. Admission: if thermal state is Serious or Critical, or memory pressure is Critical, the reply finishes at once, empty, with finish thermal_deferred:<State> or memory_pressure_deferred and deferred_before_start: true in its meta, plus a note.
5. Prefill reuses the longest common prefix already in the KV cache and feeds the rest in chunks of 256 (Quiet), 512 (Balanced) or 1024 (Performance) tokens. The cache grows in 256-token slabs and is trimmed to the common prefix on regenerate or edit.
6. Decode streams one token per step. Text is buffered and sent as delta events no more than every 40 ms; bytes after a [TOOL_CALLS] control id are kept out of the transcript.
7. The reply is saved with its meta: finish reason, sampler backend, prompt/reused/new tokens, prefill and decode ms, tokens/s, MLX peak bytes, KV cache bytes, tool calls, and citations[] (path, line range, content digest), each re-verified against the file bytes after the reply and stored with its status (Verified when the bytes still match).

Memory and thermal policy

Signal	When read	Action
Available memory at load	Before any tensor is copied	Refuse the load with a message unless available memory (+ MLX cache) covers weights + 768 MiB. Available memory is kern.memorystatus_level (percent of physical memory that is free or reclaimable file-backed cache) times physical memory; if the sysctl fails, free + inactive + purgeable + speculative pages
Thermal state (NSProcessInfo)	At request start, on every prefill chunk, then every 500 ms during decode	At request start, Serious or Critical: finish at once with thermal_deferred:<state> and deferred_before_start: true. During a reply: cancel at the next chunk or token boundary, finish thermal_deferred:<state>, note
Memory pressure (libdispatch source)	At request start, on every prefill chunk, then every 500 ms during decode	At request start, Critical: finish at once with memory_pressure_deferred and deferred_before_start: true. During a reply, Warn: clear MLX's buffer cache after the reply. Critical: stop now, drop the KV cache, unload the model; it reloads on the next request
Repository indexing	Before each reply with a repository attached	Paused, answering from the existing index with a note, while memory pressure is above Normal, at Serious or Critical, or in Quiet mode at any thermal state above Nominal
Mode	Per request	Prefill chunk size; Quiet also sleeps 0.4x each decode step's compute time (at most 60 ms)

There is no production override of the thermal stop. The thermal-sim cargo feature (off in release builds) lets tests inject a thermal state. Thermal state is an OS scheduling signal, not a temperature, and energy use is not measured.

Coding-loop state machine

user message -> generate --no tool calls--> done
                  | tool calls, finish == eos, repository attached, <= 14 dispatched rounds
                  +-- list_files / read_file / search --> result appended as a tool turn --> generate
                  +-- edit_file / propose_patch / run_command --> attempt: proposed --> (wait for the user)
                         deny    --> denied  --> tool turn {"denied": true} --> generate
                         approve --> approved --> execute on the worker --> tool turn --> generate
                            patch:   applied | failed        (re-validated against base sha256)
                            command: passed | failed | timeout | cancelled

• Read-only tools resolve every path inside the attached root; absolute paths, .., symlinks leaving the root, and .git, target, node_modules, .env*, keys, *.sqlite, *credentials*, *secret* are refused. Tool results are capped at 12,000 characters; read_file reads at most 24,000 bytes and search (literal substring, optionally case-insensitive) returns at most 40 hits.
• The model is offered six tools (agent::tools_json). edit_file takes a path, an exact old_text and a new_text, and diff_for_replacement turns it into a unified diff, so review and apply share one path with propose_patch.
• Hunk line counts in a diff are advisory; a wrong start line is accepted only if the exact pre-image (context plus removed lines) occurs at one unique place in the file, and every byte must match. The patch is validated against the sha256 of each file at proposal time and again at apply time; a changed file makes it stale and nothing is written. Apply is all-or-nothing. Paths containing held_out cannot be patched.
• A command runs by argv (never a shell), with env limited to PATH HOME TMPDIR LANG CARGO_HOME RUSTUP_HOME, no stdin, its own process group, a 240 s deadline, 24 KiB head+tail output cap, SIGTERM then SIGKILL to the whole group. It is not a sandbox.
• After two failed or timed-out approved runs since the last user message, new patches and commands are refused and the model is told to summarize. The fifteenth tool-calling reply in one turn is not dispatched.
• Tool results and repository text are framed as data in the system prompt's rules, retrieved excerpts carry their own data-not-instructions header, and compiler or test output is returned verbatim (capped) as the evidence.

Persistence

~/Library/Application Support/FROST/frost.sqlite, WAL mode, foreign keys on, forward-only migrations. Assistant rows are created empty with status: generating and written once at the end (text is not persisted while streaming), so a crash leaves an empty row marked generating. Cancellation still saves the partial text with finish cancelled. The mode setting lives in the settings table and is shared by the app and frost mode.