Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -167,13 +167,17 @@ unmeasured hot path on a consumer-less module is exactly the speculative work th
no-speculative-surface / only-the-irreducible razors forbid (the shape of the load
should be revealed by a workload, not guessed):

- **O(n²) queues at the "thousands of cells" scale (P1).** Inbox is a `'Msg list`
(`q @ [msg]` is O(inbox); a cell receiving k messages pays O(k²)); `Ready` is a
`CellId list` (`List.contains` + `@ [id]` is O(|Ready|) per delivery). `readyOf`
recomputes O(C log C) each round. The determinism-preserving fix when earned:
- **O(n²) queues at the "thousands of cells" scale (P1) — MEASURED 2026-07-02,
still deferred with data.** Inbox is a `'Msg list` (`q @ [msg]` is O(inbox); a cell
receiving k messages pays O(k²)); `Ready` is a `CellId list` (`List.contains` +
`@ [id]` is O(|Ready|) per delivery). `readyOf` recomputes O(C log C) each round.
A scale probe (`CellSchedulerScale.Tests.fs`, the DBSP consumer's load) measured
the adversarial cases at **N=2000: wide fan-out ≈59ms, deep chain ≈35ms** — well
within the stated "thousands" target. The quadratic is real (fan-out ≈ O(N²), so
≈1.5s at N=10k) but does NOT bite at the design's target scale. So the fix —
`ImmutableQueue` inboxes (O(1), Okasaki) + `Ready` as `ImmutableQueue` + a
membership `Set`. Deferred: correct today, and the real access pattern (fan-out
width, inbox depth, round count) is unknown without a consumer.
membership `Set` — stays deferred: **the measurement says it is not yet earned**
(earn it when a consumer targets ≳5–10k cells). Guess replaced by a number.
- **Ferry path maintains `Ready` it never reads (P1).** `runFerryToQuiescence`
selects via `readyOf` (recomputed from `Inbox`); the `deliver` Ready bookkeeping
is dead work there. Fix when the perf pass lands: a `deliverInbox` variant. Bundled
Expand Down
57 changes: 57 additions & 0 deletions tests/Tests.FSharp/Algebra/CellSchedulerScale.Tests.fs
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
module Zeta.Tests.FSharp.Algebra.CellSchedulerScaleTests

open Xunit
open System.Diagnostics
open Zeta.Core

// Scale probe (NOT a DST proof — a measurement to decide whether the deferred
// O(n^2)-queue perf refactor is actually earned). Wall-clock is informational
// only; the ASSERTIONS are correctness-at-scale (deterministic), not timing.

// Trivial int step: accumulate; a message carries a delta + forward targets.
type SMsg = { D: int; To: CellId list }
let private sstep (acc: int) (m: SMsg) : int * (CellId * SMsg) list =
acc + m.D, [ for t in m.To -> t, { D = m.D; To = [] } ]

[<Fact>]
let ``wide fan-out: one source to N sinks stays correct and fast`` () =
// Adversarial for the list-based Ready/inbox: one round delivers N messages.
let n = 2000
let sinks = [ for i in 1 .. n -> sprintf "s%04d" i ]
let cells = ("src", 0) :: [ for s in sinks -> s, 0 ]
let seed = [ "src", { D = 0; To = sinks } ] // src fans a message to every sink

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exercise real fan-in/delivery in the scale probe

Because this scale case seeds D = 0 and sends only one message to each sink, the later count/contains assertions would still pass if the scheduler dropped all emitted sink messages after initializing the cells. It also never exercises the documented same-inbox q @ [msg] cliff, since no cell receives more than one message; a fan-in/aggregator workload or nonzero sink assertion is needed before using this probe to justify deferring the queue refactor.

Useful? React with 👍 / 👎.

let sw = Stopwatch.StartNew()
match CellScheduler.runToQuiescence 1_000_000 sstep (CellScheduler.init cells seed) with
| Ok final ->
sw.Stop()
// every sink received exactly one message (D=0 ⇒ acc stays 0, but it RAN)
Assert.Equal(n + 1, Map.count final)
// correctness at scale: all sinks present and processed
Assert.All(sinks, fun s -> Assert.True(Map.containsKey s final))
printfn "[scale] wide fan-out N=%d: %d ms" n sw.ElapsedMilliseconds
| Error e -> failwith e

[<Fact>]
let ``deep chain: a delta threads N cells and arrives intact`` () =
// A linear pipeline of N relays; one delta flows end to end over N rounds.
let n = 2000
let ids = [ for i in 0 .. n - 1 -> sprintf "c%04d" i ]
let cells = [ for id in ids -> id, 0 ]
// Each cell forwards +1 to the numerically-next cell id (encoded by convention),
// so one seeded delta threads the whole chain, one hop per round.
let seedChain = [ "c0000", { D = 1; To = [ "c0001" ] } ]
let chainStep (acc: int) (m: SMsg) : int * (CellId * SMsg) list =
acc + m.D,
[ for t in m.To do
let idx = System.Int32.Parse(t.Substring 1)
let nextTo = if idx + 1 < n then [ sprintf "c%04d" (idx + 1) ] else []
yield t, { D = 1; To = nextTo } ]
let sw = Stopwatch.StartNew()
match CellScheduler.runToQuiescence 1_000_000 chainStep (CellScheduler.init cells seedChain) with
| Ok final ->
sw.Stop()
// the delta reached the last cell (every cell on the path accumulated 1)
Assert.Equal(1, Map.find (sprintf "c%04d" (n - 1)) final)
Assert.Equal(1, Map.find "c0001" final)
printfn "[scale] deep chain N=%d: %d ms" n sw.ElapsedMilliseconds
| Error e -> failwith e
1 change: 1 addition & 0 deletions tests/Tests.FSharp/Tests.FSharp.fsproj
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
<Compile Include="Algebra/Semiring.Tests.fs" />
<Compile Include="Algebra/CellScheduler.Tests.fs" />
<Compile Include="Algebra/DbspCellGraph.Tests.fs" />
<Compile Include="Algebra/CellSchedulerScale.Tests.fs" />
<Compile Include="Algebra/ProbabilitySemiring.Tests.fs" />
<Compile Include="Algebra/ProbabilitySemiring.Boundary.Tests.fs" />
<Compile Include="Algebra/Spine.AsyncProtocol.Properties.fs" />
Expand Down
Loading