[perf] Route FilterSet network rules incrementally during load

[Tilzen]

Fixes #541

Why

Loading brave-main-list.txt during Shields startup kept network rules twice in memory: a monolithic Vec<NetworkFilter> inside FilterSetCompilation, then a full rebuild into NetworkRulesBuilder via from_rules at Engine::from_filter_set. Maintainers reported ~67 MB peak and high CPU (brave-browser#50361).

What Changed

• FilterSetCompilation (debug=false): routes network rules incrementally into NetworkRulesBuilder during add_filter_list / add_filters instead of a monolithic vec.
• make_flatbuffer_from_compilation: serializes the incremental builder directly (no second from_rules pass).
• add_filter_list: pre-scans $badfilter ids, uses read_list_metadata for header comments, stream-parses lines.
• RuleTypes::NetworkOnly: skips cosmetic parsing and cosmetic reserve() on the Shields network-only path.
• Engine helpers: from_filter_list, from_filter_list_network, from_rules_network.
• Tests: 12 new tests covering incremental vs vec parity (including full brave-list hash), badfilter prescan, clone semantics, list routing, and network-only paths.
• Benchmarks: bench_memory network-only max; bench_rules filter-set path.

Serialized engine hash for brave-list is unchanged (7678096075090154250).

Benchmarks

Environment: Linux 6.1, release builds, data/brave/brave-main-list.txt (~204k lines).

Peak memory

Metric	master	this PR (All)	this PR (NetworkOnly)
Peak RSS (/proc/self/statm, 3 runs)	64.8 MB	49.6 MB (−23%)	37.7 MB (−42% vs master All)
bench_memory max (memory-usage-max/brave-list-initial/max)	60.6 MB	48.1 MB (−21%)	—
bench_memory max (memory-usage-max/brave-list-network-only/max)	—	—	37.9 MB

Startup CPU (profile_brave_list_cpu, release, no dhat)

Phase	master	this PR
add_filter_list	~69 ms	~75 ms
from_filter_set	~70 ms	~66 ms
serialize	~1 ms	~5 ms
Total	~141 ms	~151 ms (+7%)

NetworkOnly total: ~113 ms (~20% faster than master All).

Criterion (bencher output, --sample-size 10)

test blocker_new/brave-list ... bench: 110,528,193 ns/iter (+/- 1,784,746)
test blocker_new/brave-list-filter-set ... bench: 227,793,721 ns/iter (+/- 3,971,192)
test memory-usage-max/brave-list-initial/max ... bench: 50,420,937 B/iter (+/- 0)
test memory-usage-max/brave-list-network-only/max ... bench: 39,753,188 B/iter (+/- 0)
test memory-usage-alloc-count/brave-list-initial/alloc-count ... bench: 2,261,056 allocs/iter (+/- 0)

Trade-off: allocation count rises (~853k → ~2.26M) because rules live in eight routed vecs during load; peak RSS still drops because the duplicate monolithic vec + from_rules split at compile time is eliminated.

Profiling (flamegraphs)

Images captured from data/brave/brave-main-list.txt load path. Heap profiles via dhat + dhat-to-flamegraph (-m heap-max). CPU profiles via cargo flamegraph on GitHub Actions ubuntu (examples/profile_brave_list_cpu).

Heap at peak

Before (master)

After (this PR)

CPU flamegraph (Shields path: add_filter_list → Engine::from_filter_set)

Before (master)

After (this PR)

On master, from_filter_set still pays for NetworkRulesBuilder::from_rules (re-split of the monolithic vec). On this PR that work moves into incremental routing during parse; from_filter_set is dominated by flatbuffer serialize/optimize instead.

Startup CPU phases (ms)

Test Plan

• cargo test --all-features --tests (300 passed)
• RUSTFLAGS='--deny warnings' cargo test --all-features --tests
• cargo fmt --check
• deserialization_brave_list hash unchanged
• brave_streaming_matches_vec_path — incremental path matches vec/from_rules
• CI (ubuntu/macos/windows + perf-ci benchmark comment)

References

• FilterSet::add_filters uses a lot of resources #541
• brave-browser#50361

Notes

• Streaming directly into flatbuffer during parse was explored but deferred: optimize is only known at engine creation, FilterSet::clone (JS bindings) must preserve all rules, and retroactive $badfilter complicates already-serialized entries.
• Performance CI may comment on blocker_new/brave-list (+~5%) and alloc-count; peak memory benchmarks should improve.
• Profiling artifacts live under docs/perf/issue-541/images/ on the PR branch; CPU flamegraphs were captured via .github/workflows/capture-flamegraphs.yml on the fork (ubuntu + perf).

[atuchin-m]

I suppose this is already resolved in 0.13.x, mostly by #653