Add SDKConfigBenchmark: legacy vs config endpoint benchmark suite - #7174

Draft

facumenzella wants to merge 35 commits into

mainfrom

facu/sdk-config-benchmark-v2

Commits on Jul 9, 2026

other(benchmarks): scaffold SDKConfigBenchmark tuist project

facumenzella
and
claude
committed
other(benchmarks): add seeded rng and rc-container encoder

facumenzella
and
claude
committed
other(benchmarks): add payload factory producing real sdk wire shapes

facumenzella
and
claude
committed
other(benchmarks): add simulated network transport and fixture server

facumenzella
and
claude
committed
other(benchmarks): wire real manager-level sdk stacks per benchmark mode

facumenzella
and
claude
committed
other(benchmarks): add benchmark runner with phase metrics and jsonl output

facumenzella
and
claude
committed
other(benchmarks): add matrix runner, compare script and benchmark docs

facumenzella
and
claude
committed
feat(benchmarks): add live transport pinned to the stress-test project

facumenzella
and
claude
committed
other(benchmarks): apply review-loop fixes for measurement validity

facumenzella
and
claude
committed
other(benchmarks): simplify pass: shared request classifier and sdk helper reuse

facumenzella
and
claude
committed
other(benchmarks): apply second review pass fixes

facumenzella
and
claude
committed
other(benchmarks): apply third review pass fixes

facumenzella
and
claude
committed
other(benchmarks): apply fourth review pass fixes

facumenzella
and
claude
committed
other(benchmarks): apply fifth review pass fixes

facumenzella
and
claude
committed
other(benchmarks): wait for the iteration's config event and refresh numbers

facumenzella
and
claude
committed
other(benchmarks): apply final review pass fixes

facumenzella
and
claude
committed
other(benchmarks): document live inline blob delivery

facumenzella
and
claude
committed
feat(benchmarks): swappable live target project

facumenzella
and
claude
committed
other(benchmarks): resolve live API keys at run time instead of in source

facumenzella
and
claude
committed
feat(benchmarks): add app-host launch measurement app

facumenzella
and
claude
committed
feat(benchmarks): add app-launch xcuitest runner

facumenzella
and
claude
committed
feat(benchmarks): add app-launch variant runner and docs

facumenzella
and
claude
committed
feat(benchmarks): inject TUIST_BENCH_API_KEY into the app scheme run environment

facumenzella
and
claude
committed
feat(benchmarks): register blobs and config-path phases in both tiers

facumenzella
and
claude
committed
other(benchmarks): exclude Tuist-only benchmark targets from the xcodeproj sync check

facumenzella
and
claude
committed
fix(benchmarks): make the app-launch variant switch survive cached derived data

facumenzella
and
claude
committed
fix(benchmarks): require a successful config outcome and quarantine failed-run rows

facumenzella
and
claude
committed
other(benchmarks): apply simplify-pass cleanups to the app-launch tier

facumenzella
and
claude
committed
fix(benchmarks): validate cold config exchanges and warm priming

facumenzella
and
claude
committed
fix(benchmarks): require an explicit project label with externally supplied keys

facumenzella
and
claude
committed
feat(benchmarks): make configure + getOfferings the app-launch headline

facumenzella
and
claude
committed
other(benchmarks): ignore python bytecode from compare.py runs

facumenzella
and
claude
committed
feat(benchmarks): attribute stored blobs to their config topics

facumenzella
and
claude
committed
fix(benchmarks): retry launches that produce nothing, once, with a visible budget

facumenzella
and
claude
committed

Commits on Jul 10, 2026

Merge branch 'main' into facu/sdk-config-benchmark-v2
facumenzella
authored