Skip to content

Feed both writers' decision bodies through tiles, retiring WriterBuffer - #93

Merged
dpmills merged 1 commit into
mainfrom
dmills/tile-only-writer-input
Aug 26, 2026
Merged

Feed both writers' decision bodies through tiles, retiring WriterBuffer#93
dpmills merged 1 commit into
mainfrom
dmills/tile-only-writer-input

Conversation

@dpmills

@dpmills dpmills commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Both mutation writers fed their decision body by pushing rows onto a WriterBuffer that the body's input leaf also held, so a value computed by one producer reached another without traversing an edge: the dependency was real but invisible to the producer graph, and the ordering it needed came from the driver's own code rather than from the dataflow. This splits each writer so the body's input is an ordinary tile, and gates the rule in CI — it was already documented in src/interpreter/CLAUDE.md, and broken anyway.

The split

Each writer becomes a drive that produces the body input and a store/writer that consumes decisions. InductionDrive reads the changelog back through a FanOut::new_cyclic and holds no part of the recurrence: the store's decided frontier is the next position to iterate, and the previous accumulator is that key's value at it, so a row is a pure function of the store tile and the source tile. TransactDrive is the same shape over the transaction source, and owning that source makes it the writer's completeness signal. WriterBuffer, BodyInputBuffer and BodyInputSource are deleted; the mechanism is in design-operators.md. A shared DriveWindow holds everything the two drives do the same way, leaving each with only its own rule for which row to emit next.

One position per outer pull

The cyclic fan serves a snapshot taken before the traversal began, so a position decided during a pull is invisible until the next. Induction drove every arrived position in one pull because it owned the engine; it now runs at the rate every other cyclic operator here does, which the doc argues as a retention problem rather than a rate one. StoreDenseRead gains a decided-positions filter so a Memo above it cannot latch a fold the store has not decided.

The gate

./ci.sh shared_state flags shared mutable state under src/interpreter/ unless its inner type is a known-legitimate kind or the site says why. It cannot prove an absence, only stop one being added silently — the checker's own docstring argues the rest. CycleSlot collapses the late-wiring cell both stores hand-rolled and is the shape the gate recognises; shared_consumer / forwarding_consumer give the notification wiring one spelling.

Tests

The mutability, transaction and async-source suites are the gate and needed no behavioural changes. a_contended_item_keeps_the_drive_window_flat is new: the first operator-level test of the real drive / writer / commit-operator cycle, with six writers contending on one key so an item loses five times before winning. Two alternating writers make the loser retry once, so the end-to-end suite cannot reach that depth.

@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch from 31d1080 to fb17c97 Compare August 13, 2026 05:39
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch 2 times, most recently from f179cc3 to c02dd74 Compare August 13, 2026 21:57
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch from c02dd74 to fc254ba Compare August 13, 2026 23:42
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch from fc254ba to 0eb0370 Compare August 14, 2026 19:49
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch from 0eb0370 to a3c21e5 Compare August 14, 2026 21:34
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch from a3c21e5 to 5ae0ab6 Compare August 14, 2026 22:00
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch 2 times, most recently from 3f34eaa to 120c9e2 Compare August 15, 2026 03:51
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch from 0828012 to b4cea42 Compare August 17, 2026 23:42
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` is a copy of [#93](#93 single commit, carried here so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than against the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges, and review it there rather than here.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's sources and sinks, and inherits the operator behind every binding whose computation is unchanged, so an edit to one part of a program swaps in place while the rest keeps serving with what it has accumulated.

## Endpoints outlive a version, and one guard replaces the endpoint rule

`EndpointRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **drops state**: `LiveProgram::update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key the new version has not got. Everything else is allowed, including adding endpoints, which works — the added route serves as soon as the swap completes. Dropping a variable is refused because it is the one failure an author cannot observe: the program carries on answering and only the accumulated history is gone.

Removing an endpoint is a known gap rather than a decision: it is accepted, and the route it leaves behind still buffers requests nobody reads. [live-update.md](src/ccl/design/live-update.md#removing-an-endpoint-orphans-its-route) records it and what the fix is.

## Operator identity is the term it computes

`Let` conversion goes through `OpConversionContext::bind_let`, which keys operators by the `resolved_hash` of the bound term and adopts the fan-out a previous version built for the same computation. That key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update rather than settling in over several. Heredity is tracked separately, by the set of bindings this compilation rebuilt: `reads_only_adopted` declines any term with a free name among them, so an adopted operator never reads a rebuilt subgraph. A binding under an iteration (`BindingKind::Aligned`) is always rebuilt.

A program's mutable variables live in a `Transact` store, and every read of one is a projection off that store's `__reg` binder, so `bind_store` gives the store a class on the same terms. Without one `__reg` is free in every such term and hashes by its bare spelling, so a term reading a store hashes the same however the recurrence was edited — the defect [live-update.md](src/ccl/design/live-update.md#stores-are-bindings-too) records and `an_edit_to_the_accumulating_loop_takes_effect` pins.

## A subscription ends with its producer

A store carries the program's mutable variables, so a rebuilt one resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so the next compilation reads each carried key off `FanOut::cached_tile` and seeds the variables it still declares — no second channel out of the operator. State is keyed by `(scope, runtime key)`: an induction store's scope is the source its loop reads, because an accumulator's own key is its position within its store and two loops each carrying one both call it `acc0`; a commit store scopes under `__txn`, where keys are the variables' own spellings. A resuming store seeds its engine at the position it resumes from and its drive bases `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection where its subscriber stopped.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` first, so a rejected version leaves the running one serving, then tears down and recompiles with `Endpoints::Inherited`. `main.rs` becomes a driver over `LiveProgram`.

## Tests and docs

`tests/programs/live_update/` drives two base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. `guestbook` puts both loops in one causal group and `two-loops` gives each its own store — the difference between state surviving by replay and a store being adopted outright. Four of the thirteen cases are regressions for defects found while building this, and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, and `a_transactional_variable_survives_an_edit_to_its_writer`. `src/control_port.rs` unit-tests request parsing. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Endpoints outlive a version, and one guard replaces the endpoint rule

`EndpointRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **drops state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key it has not got. Everything else is allowed. Dropping a variable is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Removing an endpoint is a known gap rather than a decision: it is accepted and orphans the route, which [live-update.md](src/ccl/design/live-update.md#removing-an-endpoint-orphans-its-route) records.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(scope, runtime key)`, an induction store scoping by the source its loop reads and a commit store under `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Four of the thirteen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, and `a_transactional_variable_survives_an_edit_to_its_writer`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **drops state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key it has not got. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Dropping a variable is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)`, an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Five of the fifteen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, and `a_version_that_stops_serving_a_route_retires_it`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **drops state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key it has not got. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Dropping a variable is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Six of the seventeen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, and `reordering_two_accumulators_does_not_cross_their_state`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, or declares at a different type. A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Seven of the nineteen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, and `an_update_may_not_change_the_type_of_held_state`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Nine of the twenty-two cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-six cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Two cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 22, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 25, 2026
Drop this commit when rebasing after #93 merges. It is a copy of that PR's
single commit, carried here so the work above it can be written against the
structures it introduces (`InductionDrive`/`TransactDrive`, `DriveWindow`)
rather than against the `WriterBuffer` it deletes.
dpmills added a commit that referenced this pull request Aug 25, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 25, 2026
Drop this commit when rebasing after #93 merges. It is a copy of that PR's
single commit, carried here so the work above it can be written against the
structures it introduces (`InductionDrive`/`TransactDrive`, `DriveWindow`)
rather than against the `WriterBuffer` it deletes.
dpmills added a commit that referenced this pull request Aug 25, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

## Sources and sinks outlive a version, and one guard replaces the endpoint rule

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

## What survives, and how it is identified

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

## A subscription ends with its producer

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

## Driving one version at a time

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

## Tests

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
Comment thread src/interpreter/CLAUDE.md Outdated
Comment thread src/interpreter/design-operators.md
Data flows between operators only as tiles, pulled by `get` — except that both
mutation writers fed their decision body by pushing rows onto a `WriterBuffer`
the body's input leaf also held. A value computed by one producer reached another
without traversing an edge: the dependency was real but invisible to the producer
graph, and the ordering it needed (append, *then* pull) came from the writer's own
code rather than from the dataflow. This splits both writers so the body's input
is an ordinary tile, and gates the rule in CI so it cannot be broken silently
again.

## The split

Each writer becomes a **driver** that produces the body input and a
**store**/**writer** that consumes decisions. `InductionDriver` reads the changelog
back through a `FanOut::new_cyclic` and holds no part of the recurrence: the
store's decided frontier *is* the next position to iterate, and the previous
accumulator is that key's value at it, so a row is a pure function of the store
tile and the source tile. `TransactDriver` is the same shape over the transaction
source, and owning that source makes it the writer's completeness signal too.
`WriterBuffer`, `BodyInputBuffer` and `BodyInputSource` are deleted; the mechanism
is in [design-operators.md](src/interpreter/design-operators.md#the-commit-operator-interpretercommit_operatorrs).

The two drivers share a `DriverWindow`: the row window, its absolute positions, the
body-input tile it renders, and what a release reclaims. What differs between them
is only which row to emit next — the induction driver takes it from the store's
decided frontier, the transaction driver from its acked item cursor. Both go through
`body_input_fields`, so the field layout a driver declares in its tiling and the one
it renders in its tile are the same definition.

**The name.** `driver` is the operator that supplies the body's input row.
`operator_conversion` used the word for the *writer* at two `Memo` sites; both now say
the writer re-pulls the body once per proposal, which is the fact the `Memo` rests on.
`design-operators.md`'s convergence heading said "the writer drives" and now says what
that section claims: the writer re-arms.

## Where to be suspicious: the ack

A transaction's next item is not decided by the store frontier — a commit is what
*moves* it — so the item cursor advances on the commit-ack, delivered as a release.
A release from the body alone is wrong, and measurably so: a body releases a row
the moment it *consumes* it, which advanced the driver past an item still in flight.
The driver therefore sits behind a `FanOut` whose branches are the body and the
writer, and advances on their **intersection** — consumed *and* finished. That is
why `TransactWriter` holds a driver branch it barely reads.

The writer decides only the driver's newest live position, abandoning older ones.
That is supersession, not a leak, and it rests on the driver's live window belonging
to one item — asserted where it is established, since two attempts really can be
live at once.

There is one item cursor, and it is the driver's. The writer names an attempt by the
driver position it came from, which is the identity the driver itself uses; a copy of
the item index in the writer would be a second cursor advanced by a different rule,
free to disagree with the real one. The induction store likewise reads its decided
count off its engine's watermark rather than counting alongside it — that watermark
is the same number the driver folds out of the published frontier.

## One position per outer pull

The cyclic fan serves a snapshot taken before the traversal began, so a position
decided *during* a pull is invisible until the next. Induction drove every arrived
position in one pull because it owned the engine; it now runs at the rate every
other cyclic operator here does. The doc argues the resulting re-rendering as a
retention problem. Measured on a 3200-iteration loop, the split is not slower.

## The gate

The invariant was documented in `src/interpreter/CLAUDE.md` and broken anyway, so
`./ci.sh shared_state` now flags shared mutable state under `src/interpreter/`
unless its inner type is a known-legitimate kind or the site says why. It cannot
prove an absence, only stop one being added silently. `CycleSlot` collapses the
late-wiring cell both stores hand-rolled, and is the shape the gate recognises —
no allowlisted *kind* spells a slot, so a hand-rolled one has to argue for itself.

`shared_consumer` / `forwarding_consumer` give the notification wiring one
spelling. Three had accumulated, one of which compiled only because the local's
type had not been erased yet.

## Tests

The mutability, transaction and async-source suites are the gate and needed no
behavioural changes. Unit fixtures now converge the cycle before asserting — the
pull rate showing up — and `AddIfBody` propagates its domain predicate and forwards
release, as a compiled body chain does.

`a_contended_item_keeps_the_drive_window_flat` is the first operator-level test of
the real driver/writer/commit-operator cycle. Six writers contend on one key, so an
item loses five times before winning; the driver's live window must stay at
`MAX_LIVE_ATTEMPTS`. Without the writer's supersession release it reaches six — one
row per retry. The end-to-end suite cannot reach that condition: two alternating
writers make the loser retry exactly once.

Both halves of the release intersection are load-bearing for that bound. A compiled
body fans its input through a `Memo`, which releases each row as it consumes it; a
body chain that released only on its own output would leave the intersection at the
writer's ack, and the window grows one row per retry with supersession still in
place. That is an obligation on the body chain, alongside forwarding
`domain_predicate` so the store's frontier can close at all.
@dpmills
dpmills force-pushed the dmills/tile-only-writer-input branch 3 times, most recently from 07e5ce7 to 592b6d1 Compare August 26, 2026 19:00
@dpmills
dpmills merged commit b89c570 into main Aug 26, 2026
1 check passed
@dpmills
dpmills deleted the dmills/tile-only-writer-input branch August 26, 2026 19:51
dpmills added a commit that referenced this pull request Aug 26, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 26, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 27, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 27, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills added a commit that referenced this pull request Aug 27, 2026
… and swaps it in place

> **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there.

Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving.

`SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not.

The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone.

Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt.

A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently.

Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection.

Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`.

`tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants