Feed both writers' decision bodies through tiles, retiring WriterBuffer - #93
Merged
Conversation
Contributor
Author
This was referenced Aug 13, 2026
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
from
August 13, 2026 05:39
31d1080 to
fb17c97
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
2 times, most recently
from
August 13, 2026 21:57
f179cc3 to
c02dd74
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
from
August 13, 2026 23:42
c02dd74 to
fc254ba
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
from
August 14, 2026 19:49
fc254ba to
0eb0370
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
from
August 14, 2026 21:34
0eb0370 to
a3c21e5
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
from
August 14, 2026 22:00
a3c21e5 to
5ae0ab6
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
2 times, most recently
from
August 15, 2026 03:51
3f34eaa to
120c9e2
Compare
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
from
August 17, 2026 23:42
0828012 to
b4cea42
Compare
This was referenced Aug 22, 2026
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` is a copy of [#93](#93 single commit, carried here so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than against the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges, and review it there rather than here. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's sources and sinks, and inherits the operator behind every binding whose computation is unchanged, so an edit to one part of a program swaps in place while the rest keeps serving with what it has accumulated. ## Endpoints outlive a version, and one guard replaces the endpoint rule `EndpointRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **drops state**: `LiveProgram::update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key the new version has not got. Everything else is allowed, including adding endpoints, which works — the added route serves as soon as the swap completes. Dropping a variable is refused because it is the one failure an author cannot observe: the program carries on answering and only the accumulated history is gone. Removing an endpoint is a known gap rather than a decision: it is accepted, and the route it leaves behind still buffers requests nobody reads. [live-update.md](src/ccl/design/live-update.md#removing-an-endpoint-orphans-its-route) records it and what the fix is. ## Operator identity is the term it computes `Let` conversion goes through `OpConversionContext::bind_let`, which keys operators by the `resolved_hash` of the bound term and adopts the fan-out a previous version built for the same computation. That key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update rather than settling in over several. Heredity is tracked separately, by the set of bindings this compilation rebuilt: `reads_only_adopted` declines any term with a free name among them, so an adopted operator never reads a rebuilt subgraph. A binding under an iteration (`BindingKind::Aligned`) is always rebuilt. A program's mutable variables live in a `Transact` store, and every read of one is a projection off that store's `__reg` binder, so `bind_store` gives the store a class on the same terms. Without one `__reg` is free in every such term and hashes by its bare spelling, so a term reading a store hashes the same however the recurrence was edited — the defect [live-update.md](src/ccl/design/live-update.md#stores-are-bindings-too) records and `an_edit_to_the_accumulating_loop_takes_effect` pins. ## A subscription ends with its producer A store carries the program's mutable variables, so a rebuilt one resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so the next compilation reads each carried key off `FanOut::cached_tile` and seeds the variables it still declares — no second channel out of the operator. State is keyed by `(scope, runtime key)`: an induction store's scope is the source its loop reads, because an accumulator's own key is its position within its store and two loops each carrying one both call it `acc0`; a commit store scopes under `__txn`, where keys are the variables' own spellings. A resuming store seeds its engine at the position it resumes from and its drive bases `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection where its subscriber stopped. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` first, so a rejected version leaves the running one serving, then tears down and recompiles with `Endpoints::Inherited`. `main.rs` becomes a driver over `LiveProgram`. ## Tests and docs `tests/programs/live_update/` drives two base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. `guestbook` puts both loops in one causal group and `two-loops` gives each its own store — the difference between state surviving by replay and a store being adopted outright. Four of the thirteen cases are regressions for defects found while building this, and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, and `a_transactional_variable_survives_an_edit_to_its_writer`. `src/control_port.rs` unit-tests request parsing. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Endpoints outlive a version, and one guard replaces the endpoint rule `EndpointRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **drops state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key it has not got. Everything else is allowed. Dropping a variable is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Removing an endpoint is a known gap rather than a decision: it is accepted and orphans the route, which [live-update.md](src/ccl/design/live-update.md#removing-an-endpoint-orphans-its-route) records. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(scope, runtime key)`, an induction store scoping by the source its loop reads and a commit store under `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Four of the thirteen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, and `a_transactional_variable_survives_an_edit_to_its_writer`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **drops state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key it has not got. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Dropping a variable is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)`, an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Five of the fifteen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, and `a_version_that_stops_serving_a_route_retires_it`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **drops state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds a value under a key it has not got. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Dropping a variable is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Six of the seventeen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, and `reordering_two_accumulators_does_not_cross_their_state`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, or declares at a different type. A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Seven of the nineteen cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, and `an_update_may_not_change_the_type_of_held_state`. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Nine of the twenty-two cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-six cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Two cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 22, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 25, 2026
Drop this commit when rebasing after #93 merges. It is a copy of that PR's single commit, carried here so the work above it can be written against the structures it introduces (`InductionDrive`/`TransactDrive`, `DriveWindow`) rather than against the `WriterBuffer` it deletes.
dpmills
added a commit
that referenced
this pull request
Aug 25, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 25, 2026
Drop this commit when rebasing after #93 merges. It is a copy of that PR's single commit, carried here so the work above it can be written against the structures it introduces (`InductionDrive`/`TransactDrive`, `DriveWindow`) rather than against the `WriterBuffer` it deletes.
dpmills
added a commit
that referenced
this pull request
Aug 25, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. ## Sources and sinks outlive a version, and one guard replaces the endpoint rule `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. ## What survives, and how it is identified Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. ## A subscription ends with its producer Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. ## Driving one version at a time Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. ## Tests `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
sortalongo
approved these changes
Aug 26, 2026
Data flows between operators only as tiles, pulled by `get` — except that both mutation writers fed their decision body by pushing rows onto a `WriterBuffer` the body's input leaf also held. A value computed by one producer reached another without traversing an edge: the dependency was real but invisible to the producer graph, and the ordering it needed (append, *then* pull) came from the writer's own code rather than from the dataflow. This splits both writers so the body's input is an ordinary tile, and gates the rule in CI so it cannot be broken silently again. ## The split Each writer becomes a **driver** that produces the body input and a **store**/**writer** that consumes decisions. `InductionDriver` reads the changelog back through a `FanOut::new_cyclic` and holds no part of the recurrence: the store's decided frontier *is* the next position to iterate, and the previous accumulator is that key's value at it, so a row is a pure function of the store tile and the source tile. `TransactDriver` is the same shape over the transaction source, and owning that source makes it the writer's completeness signal too. `WriterBuffer`, `BodyInputBuffer` and `BodyInputSource` are deleted; the mechanism is in [design-operators.md](src/interpreter/design-operators.md#the-commit-operator-interpretercommit_operatorrs). The two drivers share a `DriverWindow`: the row window, its absolute positions, the body-input tile it renders, and what a release reclaims. What differs between them is only which row to emit next — the induction driver takes it from the store's decided frontier, the transaction driver from its acked item cursor. Both go through `body_input_fields`, so the field layout a driver declares in its tiling and the one it renders in its tile are the same definition. **The name.** `driver` is the operator that supplies the body's input row. `operator_conversion` used the word for the *writer* at two `Memo` sites; both now say the writer re-pulls the body once per proposal, which is the fact the `Memo` rests on. `design-operators.md`'s convergence heading said "the writer drives" and now says what that section claims: the writer re-arms. ## Where to be suspicious: the ack A transaction's next item is not decided by the store frontier — a commit is what *moves* it — so the item cursor advances on the commit-ack, delivered as a release. A release from the body alone is wrong, and measurably so: a body releases a row the moment it *consumes* it, which advanced the driver past an item still in flight. The driver therefore sits behind a `FanOut` whose branches are the body and the writer, and advances on their **intersection** — consumed *and* finished. That is why `TransactWriter` holds a driver branch it barely reads. The writer decides only the driver's newest live position, abandoning older ones. That is supersession, not a leak, and it rests on the driver's live window belonging to one item — asserted where it is established, since two attempts really can be live at once. There is one item cursor, and it is the driver's. The writer names an attempt by the driver position it came from, which is the identity the driver itself uses; a copy of the item index in the writer would be a second cursor advanced by a different rule, free to disagree with the real one. The induction store likewise reads its decided count off its engine's watermark rather than counting alongside it — that watermark is the same number the driver folds out of the published frontier. ## One position per outer pull The cyclic fan serves a snapshot taken before the traversal began, so a position decided *during* a pull is invisible until the next. Induction drove every arrived position in one pull because it owned the engine; it now runs at the rate every other cyclic operator here does. The doc argues the resulting re-rendering as a retention problem. Measured on a 3200-iteration loop, the split is not slower. ## The gate The invariant was documented in `src/interpreter/CLAUDE.md` and broken anyway, so `./ci.sh shared_state` now flags shared mutable state under `src/interpreter/` unless its inner type is a known-legitimate kind or the site says why. It cannot prove an absence, only stop one being added silently. `CycleSlot` collapses the late-wiring cell both stores hand-rolled, and is the shape the gate recognises — no allowlisted *kind* spells a slot, so a hand-rolled one has to argue for itself. `shared_consumer` / `forwarding_consumer` give the notification wiring one spelling. Three had accumulated, one of which compiled only because the local's type had not been erased yet. ## Tests The mutability, transaction and async-source suites are the gate and needed no behavioural changes. Unit fixtures now converge the cycle before asserting — the pull rate showing up — and `AddIfBody` propagates its domain predicate and forwards release, as a compiled body chain does. `a_contended_item_keeps_the_drive_window_flat` is the first operator-level test of the real driver/writer/commit-operator cycle. Six writers contend on one key, so an item loses five times before winning; the driver's live window must stay at `MAX_LIVE_ATTEMPTS`. Without the writer's supersession release it reaches six — one row per retry. The end-to-end suite cannot reach that condition: two alternating writers make the loser retry exactly once. Both halves of the release intersection are load-bearing for that bound. A compiled body fans its input through a `Memo`, which releases each row as it consumes it; a body chain that released only on its own output would leave the intersection at the writer's ack, and the window grows one row per retry with supersession still in place. That is an obligation on the body chain, alongside forwarding `domain_predicate` so the store's frontier can close at all.
dpmills
force-pushed
the
dmills/tile-only-writer-input
branch
3 times, most recently
from
August 26, 2026 19:00
07e5ce7 to
592b6d1
Compare
dpmills
added a commit
that referenced
this pull request
Aug 26, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 26, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 27, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 27, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
dpmills
added a commit
that referenced
this pull request
Aug 27, 2026
… and swaps it in place > **First commit is temporary.** `TEMPORARY: vendor PR #93` copies [#93](#93 single commit so the work above it is written against the structures that PR introduces (`InductionDrive` / `TransactDrive` / `DriveWindow`) rather than the `WriterBuffer` it deletes. Drop it when rebasing after #93 merges; review it there. Changing a running program means restarting it, dropping its open sockets and everything its operators accumulated. This adds a control port (`--control`, default 8081) serving `/diff` and `/update`: a new version inherits the running program's endpoints and may add to them, keeps the operator behind every binding whose computation is unchanged, and resumes each mutable variable from the value it was holding — so an edit swaps in place while the rest keeps serving. `SourceSinkRegistry` (`src/ccl/context.rs`) holds the open sources, each route's reply sink, and each bound port's listener on `GlobalContext` rather than per compilation, so a replacement binds a route the registry already holds — keeping its listener and everything buffered behind it — and opens one it does not. The only refused update is one that **cannot take over the state**: `update` reads the variables the new version declares off its planned tree and refuses if the running program holds one it no longer declares, declares at a different type, or declares under a different store (a variable moved between loops). A retype has to be caught here — allowed through, the store is built around a constant of the wrong extent and the process dies on the next pull, taking every endpoint with it. Everything else is allowed. A version may add a route, which serves as soon as the swap completes; one that stops serving a route retires it from its `SharedHttpServer`, so the address answers 404 rather than buffering requests for a reader that no longer exists. Losing a value is refused because it is the one failure an author cannot observe — the program carries on answering and only the accumulated history is gone. Conversion keys each `Let` binding and each `Transact` store by the `resolved_hash` of the term it realizes and adopts the matching one. The key is stable across compilations, so an unchanged part of a program is recognized on a program's *first* update; heredity is tracked separately, by the set of bindings this compilation rebuilt. A rebuilt store resumes rather than restarting. Its value already rides its own cyclic fan as a `Tile::Store`, so `live_state` reads each carried key off `FanOut::cached_tile` — no second channel out of the operator, which is what keeps this off the ledger `ci_shared_state` maintains. State is keyed by `(store id, runtime key)` — the runtime key being the variable's own spelling, since a write set is keyed by the variable written — with an induction store identified by the source its loop reads and a commit store by `__txn`, because planning labels an accumulator by its position within its store and two loops each carrying one both call it `acc0`. A resuming store seeds its engine at the position it resumes from and bases its `DriveWindow` there — positions are absolute, so leaving that base at `0` makes the decision lookup miss and the drive stall silently. Sharing an operator across a swap requires knowing which subscriptions are still real, so a subscription now lasts exactly as long as its subscriber: `FanOutShared::subscribers` and `Scheduler::add_source_handle` hold `Weak` references, and `SinkConsumer::detach` clears the producer slot. This changes behavior for every program, not only an updated one — a dead fan-out slot no longer pins the release-guard intersection. Nothing on the interpreter side is `Send`, so the `src/control_port.rs` server thread only parses a request and blocks; the main loop services one per tick. `LiveProgram::update` (`src/live_program.rs`) compiles to `CompileStage::Planned` and runs the state guard before touching the running program, then tears down and recompiles. `main.rs` becomes a driver over `LiveProgram`. `tests/programs/live_update/` drives three base programs over real HTTP, each with variants named for the one edit that distinguishes them; its module doc tabulates which sequence expects what. Eleven of the twenty-nine cases are regressions for defects found while building this and are worth reading first: `an_edit_to_the_accumulating_loop_takes_effect`, `reuse_does_not_depend_on_how_many_updates_came_before`, `a_store_resumes_however_far_its_source_has_advanced`, `a_transactional_variable_survives_an_edit_to_its_writer`, `a_version_that_stops_serving_a_route_retires_it`, `reordering_two_accumulators_does_not_cross_their_state`, `an_update_may_not_change_the_type_of_held_state`, `a_loop_may_gain_an_accumulator`, and `moving_a_variable_to_another_loop_is_refused`. The design doc tabulates every edit shape the guard was measured against. Five cases drive the binary as a subprocess over `stdin`, covering a program whose output is its `main` value rather than a sink: an element-wise transformation splits exactly at the swap, an accumulator carries across it, and a fed accumulator shows it happening mid-stream. Design: [live-update.md](src/ccl/design/live-update.md).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both mutation writers fed their decision body by pushing rows onto a
WriterBufferthat the body's input leaf also held, so a value computed by one producer reached another without traversing an edge: the dependency was real but invisible to the producer graph, and the ordering it needed came from the driver's own code rather than from the dataflow. This splits each writer so the body's input is an ordinary tile, and gates the rule in CI — it was already documented insrc/interpreter/CLAUDE.md, and broken anyway.The split
Each writer becomes a drive that produces the body input and a store/writer that consumes decisions.
InductionDrivereads the changelog back through aFanOut::new_cyclicand holds no part of the recurrence: the store's decided frontier is the next position to iterate, and the previous accumulator is that key's value at it, so a row is a pure function of the store tile and the source tile.TransactDriveis the same shape over the transaction source, and owning that source makes it the writer's completeness signal.WriterBuffer,BodyInputBufferandBodyInputSourceare deleted; the mechanism is in design-operators.md. A sharedDriveWindowholds everything the two drives do the same way, leaving each with only its own rule for which row to emit next.One position per outer pull
The cyclic fan serves a snapshot taken before the traversal began, so a position decided during a pull is invisible until the next. Induction drove every arrived position in one pull because it owned the engine; it now runs at the rate every other cyclic operator here does, which the doc argues as a retention problem rather than a rate one.
StoreDenseReadgains a decided-positions filter so aMemoabove it cannot latch a fold the store has not decided.The gate
./ci.sh shared_stateflags shared mutable state undersrc/interpreter/unless its inner type is a known-legitimate kind or the site says why. It cannot prove an absence, only stop one being added silently — the checker's own docstring argues the rest.CycleSlotcollapses the late-wiring cell both stores hand-rolled and is the shape the gate recognises;shared_consumer/forwarding_consumergive the notification wiring one spelling.Tests
The mutability, transaction and async-source suites are the gate and needed no behavioural changes.
a_contended_item_keeps_the_drive_window_flatis new: the first operator-level test of the real drive / writer / commit-operator cycle, with six writers contending on one key so an item loses five times before winning. Two alternating writers make the loser retry once, so the end-to-end suite cannot reach that depth.