Repository navigation
feat: column scoping — scan --columns / --exclude (0.3.0) - #51
Conversation
Restrict detection to a chosen set of columns (allowlist) or to everything but a set (denylist), applied before any detector runs and to the baseline too. This is the answer to identifier noise on wide corpora: on a real 20k journald capture, `point` flags ~10k mostly-meaningless outliers across JOB_ID/PIDs/timestamps/counters; --exclude of those fields collapses it to a couple hundred findings that matter, without touching detector config. The scope is explicit, never heuristic — on-ethos with the project's "never guess" rule, and avoids the obvious auto-skip footgun (dropping near-unique columns would discard exactly the near-unique numeric measurements the marquee detectors rely on: packet durationNanos, span durations, latencies). An unknown column name on the primary corpus is a hard error (exit 2) so a typo can't silently scope a scan to nothing and read as "clean"; the baseline is projected leniently. - ax-core: RecordSet::select / RecordSet::without projection primitives. - anomalyx: --columns/--exclude parsing (mutually exclusive, non-empty), scope_columns applied in cmd_scan and cmd_explain, usage() updated. - No envelope or config_version change — scope is an input-side projection, so the determinism contract is unchanged. - Gates: proptest + cargo-mutants 0-missed on both changed files (ax-core record.rs 21/0; anomalyx main.rs 43/0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 4 minutes and 58 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
What
Adds explicit, deterministic column scoping to
scan/explain:--columns C,..— analyze only these columns (allowlist)--exclude C,..— analyze every column except these (denylist)Mutually exclusive. The projection is applied before any detector runs and to the
--baselinetoo, so drift comparison stays consistent.Why
This is the answer to identifier noise on wide corpora, surfaced by running the installed binary against this machine's real systemd journal. On a 20k-entry
journalctl -o jsoncapture, thepointdetector dutifully flags ~10k statistical outliers across journald's many ID/counter/timestamp fields (JOB_ID, PIDs,__REALTIME_TIMESTAMP,N_RESTARTS) — each real but operationally meaningless. Scoping collapses that:Design: explicit, never heuristic
A silent auto-skip ("drop near-unique columns") was rejected as off-ethos — it would itself be a guess (the project's #1 non-negotiable is never guess), and the obvious heuristic would wrongly discard exactly the near-unique numeric measurements the marquee detectors depend on (packet
durationNanos, span durations, latencies). You name the scope; the result stays deterministic.An unknown column name on the primary corpus is a hard error (exit
2) so a typo can't silently scope a scan to nothing and read as "clean". The baseline is projected leniently (different corpus, need not carry every scoped column).Contract impact
None. Column scope is an input-side projection — no envelope field, no
config_versionchange. Same input + same flags ⇒ byte-identical output.Gates
proptest/unit: full workspace green.cargo-mutants0 missed on both changed files:ax-core/record.rs21 mutants (17 caught / 4 unviable),anomalyx/main.rs43 mutants (38 caught / 5 unviable).🤖 Generated with Claude Code