FE-1314: Run experiment runs in parallel across workers

.changeset/parallel-experiment-runs.md

@@ -0,0 +1,14 @@
+---
+"@hashintel/petrinaut-core": patch
+"@hashintel/petrinaut": patch
+---
+
+Run an experiment's runs in parallel across several Web Workers.
+
+An experiment used to run every one of its runs in a single worker, using one core however many the machine had. Runs are independent, so they now split across one worker per logical core (minus one, so the editor stays responsive), capped at the run count — measured at ~4x on 8 shards on a 10-core machine.
+
+Sharding cannot change what an experiment reports. Per-run seeds derive from the run's **global** index rather than its position within a shard, so run *i* gets the same seed whichever worker owns it, and each worker's per-frame statistics recombine through the metric accumulator monoids (`empty`/`merge`) — output is byte-identical at every shard count. A frame is only finalised once every still-running shard has reported it, with finished shards dropped from that watermark rather than blocking it.
+
+Scalar metric frames now carry their pre-reduction accumulator state, because `frameValue` is already reduced and a mean of means is not a mean.
+
+Hosts can cap or pin parallelism with `experimentShardCount` on `ExperimentsProvider`, or `shardCount` on `createMonteCarloExperiment`.

libs/@hashintel/petrinaut-core/.oxlintrc.json

@@ -144,6 +144,7 @@
     "dist/**",
     "build/**",
     "coverage/**",
+    "benchmarks/**",
     "*.gen.*",
     "*.tsbuildinfo",
     ".turbo/**"

libs/@hashintel/petrinaut-core/benchmarks/.gitignore

@@ -0,0 +1 @@
+.node-worker-bundle.mjs

libs/@hashintel/petrinaut-core/benchmarks/README.md

@@ -0,0 +1,70 @@
+# Simulation benchmarks
+
+Measurement harnesses backing
+[`../docs/simulation-performance.md`](../docs/simulation-performance.md). These
+are investigation tools, not tests — nothing here asserts, and none of it runs
+in CI.
+
+They run against the **built** package, so build first:
+
+```bash
+yarn build
+```
+
+Then, from this directory:
+
+```bash
+node monte-carlo-throughput.mjs
+```
+
+## What each one measures
+
+| Script                       | Question it answers                                                                                                    |
+| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
+| `monte-carlo-throughput.mjs` | Baseline ns/run-frame of the current engine, per-run construction cost, and what metric aggregation adds               |
+| `flat-stepper-ceiling.mjs`   | What the same net costs as hand-written flat typed-array code — i.e. the headroom available without leaving JavaScript |
+| `coloured-enumeration.mjs`   | How per-frame cost scales with token count when a coloured input arc has weight 2 (the `∏ C(n, w)` blow-up)            |
+| `shard-main.mjs`             | Whether sharding runs across worker threads scales, **and whether it changes results** (simulator level)               |
+| `sharded-experiment.mjs`     | The same question against the production `createMonteCarloExperiment` runtime with real worker threads                 |
+
+`flat-stepper-ceiling.mjs` needs no build — it imports nothing from the package.
+
+## Reading the numbers
+
+Results are reported per **run-frame** (one run advanced by one frame) rather
+than per experiment, because runs finish at different times: a completed run
+stops consuming budget, so wall clock alone conflates "faster engine" with
+"runs deadlocked earlier". `runFrames` is summed from run summaries.
+
+Absolute figures are machine-specific. The ratios are the point.
+
+## The sharding checks
+
+Two scripts cover sharding at different levels.
+
+`sharded-experiment.mjs` is the one that matters: it drives the shipped
+`createMonteCarloExperiment` over real worker threads at several shard counts.
+It bundles the worker for Node first, because the `dist` build wraps the worker
+in an inline Blob that only a browser can load. It exits non-zero if any shard
+count produces different results.
+
+`shard-main.mjs` predates the production implementation and spawns
+`shard-worker.mjs` directly against the **unmodified** `MonteCarloSimulator`. It
+is kept because it isolates the simulator from the experiment runtime, so a
+regression can be attributed to one or the other.
+
+Both check two things:
+
+1. **Scaling** — wall clock against shard count.
+2. **Result preservation** — every frame's merged histogram is fingerprinted and
+   compared across shard counts. This must print `identical to 1 shard: YES` on
+   every row. If it ever prints `NO`, the sharding design is wrong, not the
+   benchmark.
+
+Two details carry that guarantee, and both must survive into any production
+implementation:
+
+- Seeds derive from the **global** run index, so run _i_ gets the same seed
+  regardless of which shard owns it.
+- Metric state is merged with the accumulator monoid's `merge`, which is
+  associative and commutative, so shard completion order does not matter.

libs/@hashintel/petrinaut-core/benchmarks/coloured-enumeration.mjs

@@ -0,0 +1,104 @@
+/**
+ * Measures how per-frame cost scales with token count for a transition whose
+ * coloured input arc has weight 2.
+ *
+ * `enumerateWeightedMarkingIndicesGenerator` materialises the full per-place
+ * combination list up front, so the expectation is O(C(n, 2)) = O(n^2) work and
+ * allocation per transition evaluation per frame.
+ */
+import { performance } from "node:perf_hooks";
+
+const DIST = "../dist";
+
+const { createMonteCarloSimulator } = await import(`${DIST}/index.js`);
+const { compileHirArtifacts } = await import(`${DIST}/hir.js`);
+
+/** Net: one coloured place `pool`, one transition consuming 2 pool tokens. */
+const sdcpn = {
+  types: [
+    {
+      id: "t-item",
+      name: "Item",
+      iconSlug: "circle",
+      displayColor: "#00FF00",
+      elements: [{ elementId: "v", name: "v", type: "real" }],
+    },
+  ],
+  places: [
+    {
+      id: "pool",
+      name: "Pool",
+      colorId: "t-item",
+      dynamicsEnabled: false,
+      differentialEquationId: null,
+      x: 0,
+      y: 0,
+    },
+    {
+      id: "sink",
+      name: "Sink",
+      colorId: "t-item",
+      dynamicsEnabled: false,
+      differentialEquationId: null,
+      x: 100,
+      y: 0,
+    },
+  ],
+  transitions: [
+    {
+      id: "pair",
+      name: "Pair",
+      inputArcs: [{ placeId: "pool", weight: 2, type: "standard" }],
+      outputArcs: [{ placeId: "sink", weight: 1 }],
+      lambdaType: "predicate",
+      // Never fires, so token counts stay constant and we measure pure
+      // enablement/enumeration cost at a fixed marking size.
+      lambdaCode: "export default Lambda(() => false);",
+      transitionKernelCode:
+        "export default TransitionKernel(() => ({ Sink: [{ v: 1 }] }));",
+      x: 50,
+      y: 0,
+    },
+  ],
+  differentialEquations: [],
+  parameters: [],
+};
+
+const artifacts = compileHirArtifacts(sdcpn).artifacts;
+
+process.stdout.write(
+  "coloured place, input arc weight 2, transition never fires\n" +
+    "tokens   C(n,2)      ns/run-frame\n",
+);
+
+for (const tokens of [10, 25, 50, 100, 200, 400]) {
+  const simulator = createMonteCarloSimulator({
+    sdcpn,
+    initialMarking: {
+      pool: Array.from({ length: tokens }, (_, index) => ({ v: index })),
+      sink: [],
+    },
+    parameterValues: {},
+    seed: 1,
+    dt: 0.1,
+    maxTime: 5,
+    runCount: 20,
+    hirArtifacts: artifacts,
+    metrics: [],
+  });
+
+  const start = performance.now();
+  simulator.runUntilComplete();
+  const ms = performance.now() - start;
+
+  let frames = 0;
+  for (const summary of simulator.getSummaries()) {
+    frames += summary.frameNumber;
+  }
+
+  const combinations = (tokens * (tokens - 1)) / 2;
+  process.stdout.write(
+    `${String(tokens).padStart(6)}   ${String(combinations).padStart(7)}   ` +
+      `${((ms / frames) * 1e6).toFixed(0).padStart(12)}\n`,
+  );
+}

libs/@hashintel/petrinaut-core/benchmarks/flat-stepper-ceiling.mjs

@@ -0,0 +1,134 @@
+/**
+ * Measures the achievable ceiling for the SIR Monte Carlo workload in plain JS:
+ * a flat, allocation-free, structure-of-arrays stepper equivalent to what a
+ * codegen backend (JS or WASM) would emit for the same net.
+ *
+ * Mirrors the semantics of the current engine for this net:
+ *  - 3 uncoloured places (S, I, R), 2 transitions (infection, recovery)
+ *  - per transition per frame: one RNG draw, exp(-lambda * timeSinceLastFiring)
+ *    acceptance test, structural enablement on input arc weights
+ *  - infection: S-1, I+1 (consumes 1 S + 1 I, emits 2 I)
+ *  - recovery:  I-1, R+1
+ *  - deadlock when no transition is structurally enabled
+ */
+import { performance } from "node:perf_hooks";
+
+const RUNS = 4000;
+const DT = 0.1;
+const MAX_TIME = 60;
+const MAX_FRAMES = Math.round(MAX_TIME / DT);
+const INFECTION_RATE = 0.4;
+const RECOVERY_RATE = 0.1;
+
+const S0 = 500;
+const I0 = 5;
+
+// ---- Structure of arrays: one lane per run, no per-run objects. -------------
+const s = new Int32Array(RUNS).fill(S0);
+const i = new Int32Array(RUNS).fill(I0);
+const r = new Int32Array(RUNS);
+// Elapsed frames since last firing, per transition per run.
+const elapsed0 = new Int32Array(RUNS);
+const elapsed1 = new Int32Array(RUNS);
+const rng = new Uint32Array(RUNS);
+const frameNumber = new Int32Array(RUNS);
+const active = new Uint8Array(RUNS).fill(1);
+
+for (let run = 0; run < RUNS; run++) {
+  rng[run] = (42 + run * 2654435761) >>> 0;
+}
+
+// mulberry32-style step, matching the shape of the engine's seeded RNG:
+// one u32 state, one float out.
+const start = performance.now();
+
+let advancedTotal = 0;
+let activeCount = RUNS;
+
+for (let frame = 0; frame < MAX_FRAMES && activeCount > 0; frame++) {
+  for (let run = 0; run < RUNS; run++) {
+    if (active[run] === 0) {
+      continue;
+    }
+
+    const sv = s[run];
+    const iv = i[run];
+
+    // Deadlock check: infection needs S>=1 && I>=1, recovery needs I>=1.
+    if (iv === 0) {
+      active[run] = 0;
+      activeCount--;
+      continue;
+    }
+
+    let state = rng[run];
+    let fired0 = 0;
+    let fired1 = 0;
+
+    // --- transition 0: infection (S>=1, I>=1) ---
+    if (sv >= 1 && iv >= 1) {
+      state = (state + 0x6d2b79f5) >>> 0;
+      let t = state;
+      t = Math.imul(t ^ (t >>> 15), t | 1);
+      t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
+      const u = ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+      const lambda = INFECTION_RATE * (elapsed0[run] * DT);
+      if (Math.exp(-lambda) <= u) {
+        fired0 = 1;
+      }
+    }
+
+    // --- transition 1: recovery (I>=1) ---
+    if (iv >= 1) {
+      state = (state + 0x6d2b79f5) >>> 0;
+      let t = state;
+      t = Math.imul(t ^ (t >>> 15), t | 1);
+      t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
+      const u = ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+      const lambda = RECOVERY_RATE * (elapsed1[run] * DT);
+      if (Math.exp(-lambda) <= u) {
+        fired1 = 1;
+      }
+    }
+
+    rng[run] = state;
+
+    if (fired0 === 1) {
+      s[run] = sv - 1;
+      i[run] = iv + 1;
+      elapsed0[run] = 0;
+    } else {
+      elapsed0[run]++;
+    }
+
+    if (fired1 === 1) {
+      i[run] = i[run] - 1;
+      r[run] = r[run] + 1;
+      elapsed1[run] = 0;
+    } else {
+      elapsed1[run]++;
+    }
+
+    frameNumber[run]++;
+    advancedTotal++;
+
+    if (frameNumber[run] >= MAX_FRAMES) {
+      active[run] = 0;
+      activeCount--;
+    }
+  }
+}
+
+const ms = performance.now() - start;
+process.stdout.write(
+  `flat SoA stepper: ${ms.toFixed(0)} ms, ${advancedTotal} run-frames, ${(
+    (ms / advancedTotal) *
+    1e6
+  ).toFixed(0)} ns/run-frame\n`,
+);
+
+let sumR = 0;
+for (let run = 0; run < RUNS; run++) {
+  sumR += r[run];
+}
+process.stdout.write(`mean recovered: ${(sumR / RUNS).toFixed(1)}\n`);