feat: add opt-in VEGAS error reporting (return_error) - #245
Conversation
VEGAS already computes a standard deviation, chi-squared and goodness-of-fit internally but discarded them, returning only the bare integral. Passing return_error=True now returns a VEGASResult bundling all of them. - New VEGASResult dataclass (integral, sdev, chi2, dof, Q, nr_of_fevals) with a compact "value +/- sdev (chi2/dof, Q)" repr; Q via scipy's regularized upper incomplete gamma. Exposed as torchquad.VEGASResult. - return_error defaults to False, so the existing bare-value return is unchanged (no breaking change). Tests (numpy + torch): result fields well-formed and self-consistent, error estimate brackets the true error, backward-compatible default return, and numpy bit-for-bit reproducibility.
Coverage Report
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- Narrow the differentiability claim: only integral is on the gradient path; sdev/chi2 come from detached variances and are not differentiable. - Guard _get_chisq against zero-variance iterations (as _get_result/_get_error do) so the now-user-facing chi2/Q cannot become inf/nan. - Pin Q in the test to an independent gammaincc oracle.
|
Claude: Thanks — all three addressed in 467d25f:
Re-ran vegas_test.py + vegas_result_test.py: 8 passed. |
|
Claude: ### Accuracy comparison: Monte Carlo vs Sobol QMC vs VEGAS Relative error against the analytic value at an equal sample budget of N = 2¹⁴ = 16384, numpy backend, float64, fixed
Takeaways
Caveat: VEGAS's |
|
Claude: Added an error-calibration test against the analytic collection (c3b18c2) — thanks, the earlier tests only exercised one integrand.
Measured Runs on numpy and torch. Related tracked guidance is in the REVIEW.md update on #244. |
Add a calibration test that, for every real scalar-integrand analytic test function (1-D and 3-D), asserts VEGAS's reported sdev brackets the true error against the closed-form value (true_error < 8*sdev; worst observed ratio ~3.4) and that VEGAS converges (<5%). Validates the error estimate against ground truth rather than only range-checking it.
c3b18c2 to
32d8d4a
Compare
…eporting # Conflicts: # CHANGELOG.md # torchquad/integration/vegas.py
…eporting # Conflicts: # CHANGELOG.md
What
VEGAS already computes a standard deviation, chi-squared and goodness-of-fit
internally, then threw them away and returned only the bare integral. This adds
an opt-in
return_error=Truethat surfaces them:VEGASResultdataclass (integral,sdev,chi2,dof,Q,nr_of_fevals) with a compactvalue +/- sdev (chi2/dof, Q)repr;Qis thechi-squared goodness-of-fit p-value via SciPy's regularized upper incomplete
gamma. Exposed as
torchquad.VEGASResult.integralis on thegradient path;
sdev/chi2derive from VEGAS's detached variances and arenot differentiable.
return_errordefaults toFalse→ the existing bare-value return isunchanged (no breaking change).
Test plan
vegas_result_test.py(numpy + torch): result fields well-formed andself-consistent, error estimate brackets the true error, backward-compatible
default return; numpy bit-for-bit reproducibility (torch bit-reproducibility
isn't asserted — GPU reductions aren't deterministic)
vegas_test.pystill greenruff/pydoclint/vultureclean;sphinx-build -Wbuilds(
VEGASResultauto-documented via__all__, per-field attribute docstrings)Roadmap F2.