benchmark-evaluation

When Better Means Less: Quantifying What Benchmarks Miss Between Model Generations. 2,310 controlled comparisons show GPT-5 series lost 6.7x creativity and gained 4.4x false refusals vs chatgpt-4o-latest — invisible to standard benchmarks.

Research Knowledge Acquisition Engine — 5 autonomous campaigns (literature-survey, patent-mining, benchmark-archaeology, meta-analysis, baseline-establishment) with quantitative budget enforcement

Claim-relative identified sets across benchmark evaluation specifications

Measure how much of a machine-learning benchmark score survives deployment. Grouped bootstrap, frozen thresholds, gap attribution.

Code, data, and results for "The Correlation Mirage: Benchmark Dependence Collapses for Top-Performing LLMs" (EMNLP 2026). Copula-based tail dependence analysis of LLM benchmark suites.

Does a benchmark overstate operational skill when the distribution shift is ordinary and physical? Houston ground-level ozone exceedance forecasting — the control domain in a multi-domain study of benchmark-vs-operational ML performance.

How much of a phishing benchmark's score is detection skill, and how much is an artifact of how the dataset was built? A zero-parameter regex scores TSS 0.99 on PhiUSIIL; the trained model scores 0.000 on live data.

Key-free replication artifact for mutation testing of task-scoped state oracles in software-agent benchmarks

Research record and qualification toolkit for the NeuroTopo architecture-candidate line: pre-registered protocols, reproducible runs, negative results

How Hard is Microservice Anomaly Detection?

SPIRAL: structured supervision harvesting and self-refining inference for surgical video understanding. ECCV 2026 MedVidU (Oral).

Audit of BixBench, FutureHouse's bioinformatics-agent benchmark: replicated runs, replicated grading, re-derived answer keys — and what a single pooled score hides.

Accepted to Findings of EMNLP 2026. Artifact for attributing function-calling gains to interface alignment versus procedural transfer.

Deterministic params-replay audit harness for the GRIP-V video benchmark

Replication package for: The Score Is Not the Model - four protocol failure modes in Open Graph Benchmark reporting

Re-evaluating HINT & MEXA-CTP on the TOP benchmark: reported gains lie below the noise floor; a base-rate lookup matches every model. Reproducible code + data + paper.