-
Notifications
You must be signed in to change notification settings - Fork 0
Benchmark devirt on real-world dispatch: tantivy 3.5× faster, tracing ~0% #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
6317de1
463c0f7
01712fe
180fdbb
ee2eba8
2b355d9
5074abc
261ddb6
8afa6b9
db32f18
d32ea09
cf915f0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,6 @@ | ||
| [workspace] | ||
| members = ["crates/*", "fuzz"] | ||
| exclude = ["benchmarks"] | ||
| resolver = "3" | ||
|
|
||
| [workspace.package] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| target/ | ||
| Cargo.lock |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # Real-World Devirt Benchmarks | ||
|
|
||
| Benchmarks measuring `devirt`'s impact on dispatch patterns from real Rust projects. | ||
|
|
||
| ## tantivy search engine — scorer dispatch (`tantivy-devirt/`) | ||
|
|
||
| Reproduces tantivy's per-document scoring loop. tantivy iterates over | ||
| matching documents calling `scorer.score()` (BM25: ~10 arithmetic ops) | ||
| and `scorer.advance()` through `&mut dyn Scorer` — mandatory dynamic | ||
| dispatch since the scorer type comes from the user's search query parsed | ||
| at runtime. | ||
|
|
||
| Reference: [tantivy `src/query/weight.rs`](https://github.com/quickwit-oss/tantivy/blob/main/src/query/weight.rs) | ||
|
|
||
| Note: tantivy already manually devirtualizes its hottest paths | ||
| (`TermWeight::for_each` calls the concrete `TermScorer` directly). | ||
| The `dyn Scorer` fallback runs for uncommon query types (phrase, regex, | ||
| fuzzy). This benchmark measures the speedup devirt would provide on that | ||
| fallback path, and demonstrates what tantivy achieves manually that devirt | ||
| could automate. | ||
|
|
||
| ### Results (TermScorer with BM25 scoring) | ||
|
|
||
| | n documents | devirt | plain vtable | Speedup | | ||
| |-------------|--------|-------------|---------| | ||
| | 1,000 | 3.1 µs | 9.9 µs | **3.2×** | | ||
| | 10,000 | 27.8 µs | 98.9 µs | **3.6×** | | ||
| | 100,000 | 276 µs | 961 µs | **3.5×** | | ||
| | 1,000,000 | 3.54 ms | 10.3 ms | **2.9×** | | ||
|
|
||
| Shuffled 80/20 hot/cold (100 scorers × 1000 docs): 333 µs vs 821 µs — **2.5×** | ||
|
|
||
| ### Running | ||
|
|
||
| ```bash | ||
| cd benchmarks/tantivy-devirt | ||
| cargo bench | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,19 @@ | ||||||
| [package] | ||||||
| name = "tantivy-devirt-bench" | ||||||
| version = "0.0.0" | ||||||
| edition = "2024" | ||||||
| publish = false | ||||||
|
|
||||||
| [dependencies] | ||||||
| devirt = { path = "../../crates/core" } | ||||||
|
|
||||||
| [dev-dependencies] | ||||||
| criterion = { version = "0.5", features = ["html_reports"] } | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Criterion version mismatch with workspace. The workspace specifies Consider updating to match the workspace version: -criterion = { version = "0.5", features = ["html_reports"] }
+criterion = { version = "0.8", features = ["html_reports"] }📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
|
|
||||||
| [[bench]] | ||||||
| name = "scorer" | ||||||
| harness = false | ||||||
|
|
||||||
| [profile.bench] | ||||||
| lto = "thin" | ||||||
| codegen-units = 1 | ||||||
|
Comment on lines
+17
to
+19
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial Profile configuration duplicates workspace settings. These benchmark profile settings are identical to the workspace's 🤖 Prompt for AI Agents |
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Document the tracing benchmarks and ConstScorer results mentioned in the PR objectives.
The PR objectives explicitly describe two benchmark case studies:
Additionally, the PR mentions that "ConstScorer (very cheap score()) shows ~0% difference," which is valuable context missing from the README.
The negative results are just as important as the positive ones—they help users understand when devirt is and isn't beneficial. Consider adding:
🤖 Prompt for AI Agents