perf: hardware-adaptive parallel scanning & fast Darwin fts traversal - #44
Open
bebricoOOOOOOf wants to merge 2 commits into
Open
perf: hardware-adaptive parallel scanning & fast Darwin fts traversal#44bebricoOOOOOOf wants to merge 2 commits into
bebricoOOOOOOf wants to merge 2 commits into
Conversation
…daptive concurrency - Replace FileManager.enumerator with Darwin fts_open (FTS_PHYSICAL | FTS_NOCHDIR) in directorySize for 2.6x+ single-thread throughput without object allocation overhead - Add fast access(cPath, F_OK) check to bypass non-existent cache directories in microseconds - Parallelize category, known cache, and large file scanning with bounded OperationQueue and utility QoS - Replace hardcoded 1-worker limit for full volume scans in DiskSpaceWindow with hardware-adaptive scaling (1 on <=2 cores / low-power, 2 on 4 cores, 4 on 8+ cores) - Ensure all background scanning runs at .utility QoS so Apple Silicon schedules work onto Efficiency cores, keeping user apps responsive and preventing thermal throttling - Fix codesigning detritus issue in build_app.sh by staging in a clean temporary directory
Keep physical FTS traversal on the root device and report incomplete measurements. Use conservative power/thermal-aware per-stage budgets and synchronized results. Preserve app bundles on signing/copy failures and verify the final destination. Add traversal regression tests, macOS CI, and an alternating-order benchmark harness. Remove unverified speedup and E-core/thermal guarantees from the PR description. Validation: six staging failure/success tests and shell syntax/diff checks pass. macOS Swift tests, TSan, universal build and performance measurements require macOS.
bebricoOOOOOOf
force-pushed
the
perf/parallel-fast-scanner
branch
from
September 8, 2026 00:07
72f5a6d to
845ec5a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR significantly improves ClearDisk's scanning speed across all areas of the application:
DiskSpaceWindow)DiskMonitor)Crucially, concurrency is hardware-adaptive and thermal/power-aware:
.utilityQoS, ensuring Apple Silicon schedules them onto Efficiency (E) cores, leaving Performance cores completely free for user applications and preventing fan noise.Technical Details
1. Darwin
fts_openTraversal (DiskMonitor.swift)FileManager.enumeratorandresourceValues(forKeys:)with POSIX/Darwinfts_open(FTS_PHYSICAL | FTS_NOCHDIR, nil).st_blocks * 512) and hardlinks (st_nlink) directly from C structs, eliminating millions of Foundation object allocations and Objective-C bridging overhead during deep crawls.access(cPath, F_OK)pre-check so non-existent cache paths return in < 1 microsecond before entering any directory iterator.2. Bounded Concurrency with
optimalScanConcurrency(DiskMonitor.swift)scanDiskCategories(),scanKnownCaches(), andscanLargeFiles()now process items concurrently usingOperationQueuethrottled byoptimalScanConcurrency:3. Hardware-Adaptive Full-Volume Scanning (
DiskSpaceWindow.swift)scanWorkerLimit = 1bottleneck on/(Macintosh HD).scanWorkerLimits(for:)which dynamically scales traversal, classification, and atomic package summarization:isLowPowerModeEnabledis true.4. Codesign Staging Fix (
scripts/build_app.sh)/tmpbefore runningcodesignto strip anycom.apple.provenanceor iCloud Drive extended attributes that triggerresource fork, Finder information, or similar detritus not allowed.Real-World Benchmarks
Tested on macOS with 180,352 files in
~/Documents(heavy Python.venv, Node packages, build artifacts):FileManager)fts+ Bounded Concurrency).utilityQoS (Efficiency cores)