I came to AI from psychology, and I keep asking the same question I learned there. How sure can we be that a measurement means what it says? Today I ask it about AI agents, benchmarks and the numbers people quote about them.
What I've made so far
Agent Reliability Calculator shows why adding more verifiers stops helping after a point and why retrying an agent buys less than the usual formula promises. The defaults come from 131,332 real agent runs on four public benchmarks. You can try it in the browser.
Honest agent scores asks whether broken benchmark tasks or repeated runs change which AI agent comes out on top. On 3,300 Terminal-Bench 4.0 runs the answer was mostly no. The surprise was that broken tasks can make agents look different in reliability when they are not.
The Signature Explorer explains digital signatures from the one in your crypto wallet to the post-quantum ones. You can sign a message, break it and even forge a signature yourself, and every fact on the page links to its original source. Open the site.
How I work
I write down the analysis plan before I run the numbers. If the result is negative, I publish it anyway. Every figure in my projects can be traced back to where it came from.