Implement new storage to improve performance #3
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Báo cáo hiệu năng storage backend | ||
|
|
||
| So sánh 3 backend mà `codegraph-graph` hỗ trợ cho việc persist index: | ||
|
|
||
| - `in_memory` — `GraphIndex::in_memory()` (baseline RAM, không persist) | ||
| - `sqlite` — backend hiện tại, qua `sqlx` (`sqlite://<dir>/db.sqlite`) | ||
| - `lmdb` — backend mới thêm, qua `lmdb-rkv` (`lmdb://<dir>`) | ||
|
|
||
| > Redis bị loại khỏi phạm vi vì đã chạy trên RAM, không phải "disk-backed". | ||
|
|
||
| ## Cách đo | ||
|
|
||
| Benchmark chạy **đúng pipeline thật** như `codspeed.rs` (extract → index → query) | ||
| thay vì micro-benchmark gọi trực tiếp từng `Storage`. Với mỗi repo: | ||
|
|
||
| 1. **extract** một lần (`codegraph-extract`: walk + parse → `Vec<ParseResult>`). | ||
| 2. **index**: với mỗi backend, mỗi iteration dựng **storage mới** (tempdir/file | ||
| mới) rồi `GraphIndex::open(dsn)` + `ingest` — đo chi phí open+ingest, không bị | ||
| tích luỹ giữa các iteration. Backend được chọn bằng **DSN scheme** | ||
| (`sqlite://` / `lmdb://` / `None` = in-memory), đúng cơ chế | ||
| `GraphIndex::open(dsn)` trong `lib.rs`. | ||
| 3. **query**: chạy bộ truy vấn mẫu trên index in-memory sau ingest (engine query | ||
| nằm in-memory, backend không ảnh hưởng phase này). | ||
|
|
||
| Repo đo: toàn bộ `crates/` (chính workspace này). Lệnh: | ||
|
|
||
| ```bash | ||
| cargo bench -p codegraph-bench --bench storage | ||
| ``` | ||
|
|
||
| ## Kết quả | ||
|
|
||
| ### index: open + ingest (mỗi iteration storage mới) | ||
|
|
||
| | Backend | lần 1 (median) | lần 2 (median) | lần 3 (median) | ghi chú | | ||
| |-------------|---------------|----------------|----------------|---------| | ||
| | `in_memory` | 13.75 µs | 12.09 µs | 8.99 µs | không persist, không I/O | | ||
| | `sqlite` | 42.73 ms | 40.55 ms | 13.46 ms | biến động cao | | ||
| | `lmdb` | 20.01 ms | 28.40 ms | 15.88 ms | biến động cao | | ||
|
|
||
| **Nhận xét**: biến động giữa các lần chạy lớn (máy đo còn chia tải). Trung bình | ||
| LMDB nhanh hơn SQLite khoảng **1.4–2.1×**; có lần chạy về ngang nhau. Lợi thế | ||
| của LMDB đến từ: viết 1 transaction duy nhất cho toàn bộ commit (không | ||
| WAL/journal riêng, không parser SQL mỗi op), và mapping file theo trang B+tree | ||
| kiểu B-tree copy-on-write. | ||
|
|
||
| ### Dung lượng trên đĩa (corpus `crates/`) | ||
|
|
||
| | Backend | kích thước | ghi chú | | ||
| |---------|-----------|---------| | ||
| | `sqlite` | ~590–690 KB | file db.sqlite | | ||
| | `lmdb` | ~270 KB | thư mục chứa data.mdb | | ||
|
|
||
| **Nhận xét**: LMDB chiếm **ít hơn ~2.2×** so với SQLite trên cùng dữ liệu — bản | ||
| thân LMDB chứa trang metadata + dữ liệu compact; SQLite lưu cả schema, WAL | ||
| overhead và trang trống. | ||
|
|
||
| ### query (index in-memory, backend không ảnh hưởng) | ||
|
|
||
| | Nhóm | median | | ||
| |-------|--------| | ||
| | `sample` (search_symbol + callees + flow × 200 tên) | ~84–90 ns / op | | ||
|
|
||
| Query không bị ảnh hưởng bởi backend vì sau `ingest` engine đọc từ graph | ||
| in-memory. | ||
|
|
||
| ## Khuyến nghị | ||
|
|
||
| - **LMDB đáng dùng khi cần persist nhanh hơn + nhỏ hơn** (cùng mức API | ||
| `GraphIndex::open(dsn)`), đặc biệt cho index lớn: chi phí open+ingest thấp hơn | ||
| và footprint ~2.2× nhỏ hơn SQLite. | ||
| - **SQLite vẫn là lựa chọn an toàn** nếu cần tooling/quen thuộc với file `.db` | ||
| đơn, hoặc dùng query ad-hoc bên ngoài. Độ lệch hiệu năng giữa 2 backend nằm | ||
| trong tầm 1.4–2.1× tuỳ tải máy. | ||
| - `in_memory` là baseline nhanh nhất (không I/O), dùng cho trường hợp không cần | ||
| persist (CLI một lần). | ||
| - Redis giữ vai trò dành cho triển khai cần chia sẻ index giữa nhiều process. | ||
|
|
||
| Chọn backend bằng DSN scheme: | ||
|
|
||
| ```rust | ||
| GraphIndex::open("sqlite:///tmp/db.sqlite").await?; // sqlite | ||
| GraphIndex::open("lmdb:///tmp/db").await?; // lmdb | ||
| GraphIndex::in_memory(); // RAM | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,155 @@ | ||
| //! Benchmark **storage backend** qua đúng pipeline luồng thật (extract → index → | ||
| //! query) như `codspeed.rs`, nhưng mỗi backend một group và mỗi iteration dựng | ||
| //! storage **mới** (file mới) để đo chi phí open+ingest không bị tích luỹ. | ||
| //! | ||
| //! Backend được chọn bằng DSN scheme (đúng cơ chế `GraphIndex::open(dsn)`): | ||
| //! - `in_memory` — `GraphIndex::in_memory()` (baseline RAM, không persist) | ||
| //! - `sqlite` — `sqlite://<dir>/db.sqlite` (persist) | ||
| //! - `lmdb` — `lmdb://<dir>/db` (persist) | ||
| //! | ||
| //! Chạy (repo list giống codspeed: `CODEGRAPH_BENCH_REPOS_LIST` hoặc fallback | ||
| //! `crates`): | ||
| //! ```bash | ||
| //! CODEGRAPH_BENCH_REPOS_LIST=repos.txt cargo bench -p codegraph-bench --bench storage | ||
| //! ``` | ||
|
|
||
| use codegraph_bench::{ | ||
| BenchOptions, Repo, extract, index_at, orchestrator, run_queries, sample_query_names, | ||
| }; | ||
| use criterion::{Criterion, black_box, criterion_group, criterion_main}; | ||
|
|
||
| fn load_repos() -> Vec<Repo> { | ||
| let mut out = Vec::new(); | ||
| if let Ok(list_file) = std::env::var("CODEGRAPH_BENCH_REPOS_LIST") { | ||
| if let Ok(body) = std::fs::read_to_string(&list_file) { | ||
| for line in body.lines() { | ||
| let line = line.trim(); | ||
| if line.is_empty() || line.starts_with('#') { | ||
| continue; | ||
| } | ||
| let name = std::path::Path::new(line) | ||
| .file_name() | ||
| .and_then(|s| s.to_str()) | ||
| .map(String::from) | ||
| .unwrap_or_else(|| line.to_string()); | ||
| out.push(Repo { | ||
| name, | ||
| root: line.into(), | ||
| }); | ||
| } | ||
| } | ||
| return out; | ||
| } | ||
| out.push(Repo { | ||
| name: "crates".into(), | ||
| root: "crates".into(), | ||
| }); | ||
| out | ||
| } | ||
|
|
||
| /// Dung lượng trên đĩa của một thư mục (đệ quy), dùng để so sánh footprint | ||
| /// của sqlite vs lmdb trên cùng một corpus. | ||
| fn dir_size(path: &std::path::Path) -> u64 { | ||
| let mut total = 0u64; | ||
| if let Ok(rd) = std::fs::read_dir(path) { | ||
| for ent in rd.flatten() { | ||
| let p = ent.path(); | ||
| if p.is_dir() { | ||
| total += dir_size(&p); | ||
| } else if let Ok(md) = std::fs::metadata(&p) { | ||
| total += md.len(); | ||
| } | ||
| } | ||
| } | ||
| total | ||
| } | ||
|
|
||
| /// Đo một lần dung lượng file thật trên đĩa cho sqlite vs lmdb (không chạy | ||
| /// trong benchmark lặp) để báo cáo footprint. Mỗi backend một tempdir riêng. | ||
| fn measure_on_disk(parsed: &[codegraph_graph::ParseResult]) { | ||
| let sqlite_dir = tempfile::tempdir().unwrap().keep(); | ||
| let sqlite = format!("sqlite://{}/db.sqlite", sqlite_dir.to_string_lossy()); | ||
| if let Ok(_idx) = index_at(parsed, Some(&sqlite)) {} | ||
| let sqlite_bytes = dir_size(&sqlite_dir); | ||
|
|
||
| let lmdb_dir = tempfile::tempdir().unwrap().keep(); | ||
| let lmdb = format!("lmdb://{}", lmdb_dir.to_string_lossy()); | ||
| if let Ok(_idx) = index_at(parsed, Some(&lmdb)) {} | ||
| let lmdb_bytes = dir_size(&lmdb_dir); | ||
|
Comment on lines
+78
to
+86
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Fail when persistent backend setup fails. The empty 🤖 Prompt for AI Agents🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf 'Repository files matching storage.rs:\n'
fd -a 'storage\.rs$' . | sed 's#^\./##'
printf '\nTarget file outline:\n'
ast-grep outline crates/codegraph-bench/benches/storage.rs || true
printf '\nRelevant sections:\n'
sed -n '1,180p' crates/codegraph-bench/benches/storage.rs
printf '\nDisk-related benches/search:\n'
rg -n "tempfile::tempdir|TempDir::keep|keep\(|index_at|dir_size|GraphIndex|bench|Criterion|storage" crates/codegraph-bench/benches/storage.rsRepository: hungpham10/codegraph-rs Length of output: 8097 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Check tempfile crate docs/source availability via Cargo.lock/Cargo.toml and installed docs if present.
printf 'tempfile declarations:\n'
rg -n '^\[.*tempfile|^\[package\]|name = "tempfile"|version = ' Cargo.lock crates/codegraph-bench/Cargo.toml Cargo.toml || true
printf '\nCargo docs availability:\n'
if [ -d "$CARGO_HOME/doc/tempfile" ]; then
find "$CARGO_HOME/doc/tempfile" -maxdepth 2 -type f | sed -n '1,40p'
else
echo "cargo doc for tempfile not found"
fiRepository: hungpham10/codegraph-rs Length of output: 14298 🌐 Web query:
💡 Result: In the Rust tempfile crate (version 3.27.0), the TempDir::keep method is used to persist a temporary directory on the disk [1]. Calling TempDir::keep consumes the TempDir instance and prevents the directory from being automatically deleted when it goes out of scope (i.e., when the object is dropped) [1]. It returns a PathBuf representing the path to the directory, allowing you to manage it manually [1]. Key points regarding keeping or persisting temporary directories: - TempDir::keep: This is the preferred method for turning a temporary directory into a permanent one [1]. It consumes the TempDir object [1]. - disable_cleanup: If you want to prevent automatic deletion but keep the TempDir object available for further use (rather than consuming it), you can call disable_cleanup(true) [1]. This is typically recommended for testing or debugging scenarios rather than simply persisting a directory [1]. - Deprecation Note: The method into_path has been deprecated in favor of TempDir::keep [1]. Citations: Clean up benchmark temporary storage after measurement. All these sites call 📍 Affects 1 file
🤖 Prompt for AI Agents |
||
|
|
||
| eprintln!( | ||
| "on-disk: sqlite={} bytes | lmdb={} bytes", | ||
| sqlite_bytes, lmdb_bytes | ||
| ); | ||
| } | ||
|
|
||
| fn main_benchmark(c: &mut Criterion) { | ||
| let opts = BenchOptions { | ||
| langs: None, | ||
| queries: 200, | ||
| with_flow: false, | ||
| }; | ||
| let repos = load_repos(); | ||
| for repo in &repos { | ||
| let name = repo.name.clone(); | ||
| // Parse một lần (extract), dùng chung cho mọi backend. | ||
| let parsed = match extract(&orchestrator(&opts), &repo.root) { | ||
| Ok((p, _)) => p, | ||
| Err(e) => { | ||
| eprintln!("[{name}] extract failed: {e}; skip"); | ||
| continue; | ||
| } | ||
| }; | ||
| let names = sample_query_names(&parsed, opts.queries); | ||
| measure_on_disk(&parsed); | ||
|
|
||
| // ── index: mỗi backend một group, storage MỚI mỗi iteration ── | ||
| // Mỗi backend là một closure `mk_dsn()` trả DSN cho một storage trống | ||
| // (tempdir mới). Với in-memory, dsn = None. | ||
| let mk_backends: Vec<(&str, Box<dyn Fn() -> Option<String>>)> = vec![ | ||
|
coderabbitai[bot] marked this conversation as resolved.
Outdated
|
||
| ("in_memory", Box::new(|| None)), | ||
| ( | ||
| "sqlite", | ||
| Box::new(|| { | ||
| let dir = tempfile::tempdir().unwrap().keep(); | ||
| Some(format!("sqlite://{}/db.sqlite", dir.to_string_lossy())) | ||
| }), | ||
| ), | ||
| ( | ||
| "lmdb", | ||
| Box::new(|| { | ||
| let dir = tempfile::tempdir().unwrap().keep(); | ||
| Some(format!("lmdb://{}", dir.to_string_lossy())) | ||
| }), | ||
| ), | ||
| ]; | ||
|
|
||
| for (bname, mk_dsn) in mk_backends { | ||
| let parsed = &parsed; | ||
| let mut g = c.benchmark_group(format!("{name}/{bname}/index")); | ||
| g.bench_function("open+ingest", |b| { | ||
| b.iter(|| { | ||
| let dsn = mk_dsn(); | ||
| let _ = black_box(index_at(parsed, dsn.as_deref())); | ||
| }); | ||
| }); | ||
| g.finish(); | ||
| } | ||
|
|
||
| // ── query trên index in-memory (backend không ảnh hưởng query — engine | ||
| // in-memory sau ingest) — giữ để pipeline giống codspeed. ── | ||
| if let Ok(idx) = index_at(&parsed, None) { | ||
| let mut g = c.benchmark_group(format!("{name}/query")); | ||
| let names = &names; | ||
| g.bench_function("sample", |b| { | ||
| b.iter(|| { | ||
| let _ = black_box(run_queries(&idx, names, false)); | ||
| }); | ||
| }); | ||
| g.finish(); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| criterion_group!(benches, main_benchmark); | ||
| criterion_main!(benches); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Do not silently skip all benchmarks after a list-file read error.
If
CODEGRAPH_BENCH_REPOS_LISTexists butread_to_stringfails, this function returns an empty list. Criterion can then finish successfully without benchmark results. Return an error, or log the error and use thecratesfallback.🤖 Prompt for AI Agents