storage: last-ref unmap and defer munmap to the disk thread#850
Open
j-c-m wants to merge 3 commits into
Open
Conversation
Owner
|
Just like the aync fd close changes I did, this should be in a separate thread. |
Drop the sticky writable hold that parked dirty pieces on m_queue so they stayed mmap'd until a periodic sync drain. release() now decrements every handle and clear_chunk/munmap when references hit zero. MS_ASYNC on the sync path is unchanged; writeback pacing for retired maps is handled when teardown is deferred to the disk thread.
Queue MemoryChunk unmap regions onto rtorrent disk so last-ref/clear does not block the caller (usually main) on writeback kickoff or munmap. On disk, issue MS_ASYNC then munmap so dirty MAP_SHARED pages start writeback as pieces are retired, avoiding multi-GB flusher storms. Falls back to the same MS_ASYNC + munmap sequence when the disk thread is inactive (tests/shutdown). Drain the queue in call_events and cleanup_thread.
j-c-m
force-pushed
the
feature/disk-thread-munmap
branch
from
July 25, 2026 19:06
d250d8f to
efba7bc
Compare
Contributor
Author
|
Light testing on the move to std::async, will continue more. |
rakshasa
reviewed
Jul 26, 2026
| if (manager != nullptr) { | ||
| manager->chunk_manager()->queue_munmap(ptr, length); | ||
| return; | ||
| } |
Owner
There was a problem hiding this comment.
This is the wrong place for this, MemoryChunk is supposed to be a simple type.
| // Persistent std::async worker that issues MS_ASYNC + munmap off the caller | ||
| // (and off ThreadDisk). Same shape as FdCloseQueue. | ||
|
|
||
| class MunmapQueue { |
| MunmapQueue(const MunmapQueue&) = delete; | ||
| MunmapQueue& operator=(const MunmapQueue&) = delete; | ||
|
|
||
| using region_type = std::pair<void*, size_t>; |
Own MemoryUnmapQueue on ChunkManager (FdCloseQueue-style worker: MS_ASYNC + munmap), not ThreadDisk. Queue from ChunkList::clear_chunk so MemoryChunk stays a simple type.
j-c-m
force-pushed
the
feature/disk-thread-munmap
branch
from
July 26, 2026 14:48
19c9696 to
3a6d558
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unmap piece maps on last handle release and run MS_ASYNC + munmap on the
disk thread so main is not blocked on teardown.
Flush policy: dirty MAP_SHARED writeback is left mostly to the OS (kernel
page cache). Previously, last writable handles kept maps sticky on
ChunkList::m_queue until a periodic sync_chunks drain on main, which could
msync/munmap a large dirty set at once (main-thread flush storms under
fast leech). Maps are now retired as pieces complete instead of
accumulating for a bulk main-thread pass.
Last-ref unmap — drop the sticky writable hold; when refs hit zero, clear/unmap immediately (writable and RO share the same path). Previously there was a policy to have them be sticky for a bit, but this seems unnecessary now as modern OS stack here will hold these in page cache for some duration and manage the disk IO better.
Deferred teardown — MemoryChunk::unmap queues the VA to ThreadDisk, which does msync(MS_ASYNC) then munmap. This gets the blocking off the ui / peers thread, this was more of an issue on FreeBSD as msync(MS_ASYNC) eventually blocks for a long duration during a big flush storm, Linux would did see some blocking in munmap() but not as much overall. In any case, this keeps it from blocking UI and peer network transfers at all.
These changes offer a large removal opportunity as last-ref unmap means nothing new is parked on ChunkList::m_queue. Callers of the sticky sync path still exist, but for maps released this way they no longer do useful work