Proposal
Two undocumented divergences from the reference implementation, in sparse_probing.py and docs/source/content/sparse_probing.md.
Motivation
No std floor under preprocess="standardize" (:307). Only an exactly-zero standard deviation maps to scale 1; the reference floors at max(std, 1e-3). A near-constant column is therefore standardized by its own noise:
g = torch.Generator().manual_seed(0)
X = torch.randn(200, 8, generator=g)
X[:, 0] = 1.0 + torch.randn(200, generator=g) * 1e-6
y = (torch.rand(200, generator=g) < 0.5).long()
fit_sparse_probe(X, y, k=8, preprocess="standardize", seed=0)
# train std 9.930e-07 -> preprocess_scale 9.922e-07, where the reference would use 1e-3 (1008x larger)
# constant_features is False, so nothing in the result flags it
The column reaches the fit as pure noise at unit scale. This is what #1728 specified, so it may be a guide note rather than a change, but it's written down nowhere today.
max_eval is public only as a string (:458). The LBFGS evaluation budget is max_iter * 5 // 4, torch's default. A caller who gets stop_reason == "max_eval" can't tell from the signature or the guide what budget was hit, and for max_iter <= 3 it equals max_iter.
Pitch
- Either expose a
floor defaulting to the reference's 1e-3, or name the amplification in the guide's divergence paragraph; a near-constant-column test either way.
- State the derived evaluation budget in the
max_iter docstring.
Acceptance:
Checklist
Proposal
Two undocumented divergences from the reference implementation, in
sparse_probing.pyanddocs/source/content/sparse_probing.md.Motivation
No std floor under
preprocess="standardize"(:307). Only an exactly-zero standard deviation maps to scale 1; the reference floors atmax(std, 1e-3). A near-constant column is therefore standardized by its own noise:The column reaches the fit as pure noise at unit scale. This is what #1728 specified, so it may be a guide note rather than a change, but it's written down nowhere today.
max_evalis public only as a string (:458). The LBFGS evaluation budget ismax_iter * 5 // 4, torch's default. A caller who getsstop_reason == "max_eval"can't tell from the signature or the guide what budget was hit, and formax_iter <= 3it equalsmax_iter.Pitch
floordefaulting to the reference's1e-3, or name the amplification in the guide's divergence paragraph; a near-constant-column test either way.max_iterdocstring.Acceptance:
max_iterdocstring names the evaluation budgetmake unit-testpassesuv run mypy .passesChecklist