Skip to content

Add cross-platform sendfile uploads with fallback to normal send#794

Open
j-c-m wants to merge 1 commit into
rakshasa:masterfrom
j-c-m:feature/sendfile-zero-copy
Open

Add cross-platform sendfile uploads with fallback to normal send#794
j-c-m wants to merge 1 commit into
rakshasa:masterfrom
j-c-m:feature/sendfile-zero-copy

Conversation

@j-c-m

@j-c-m j-c-m commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

I've just begun testing this, really experimental, opening as a draft if you have any comment.


Introduce a unified sendfile_stream() abstraction for zero-copy piece uploads on Linux, FreeBSD, and macOS/Darwin, each with its own syscall signature detected at configure time.

When pieces.sendfile is enabled on plaintext peers, uploads resolve file fd and offset via FileList::at_file() without mmap, falling back to chunk mapping only for encryption, padding, or sendfile errors.

Expose pieces.sendfile, pieces.sendfile.set, and sendfile stats via ChunkManager for rtorrent configuration.

@rakshasa

Copy link
Copy Markdown
Owner

I've considered supporting sendfile way back, and dropped it.

Checked with AI and it agrees that it's not more efficient than mmap due to pieces being 16kb.

Lately I've been considering adding an BEP-10 extension to allow for larger piece messages, as that seems the best way to increase throughput. The new BT spec is too complicated for most clients to bother implementing.

@j-c-m
j-c-m force-pushed the feature/sendfile-zero-copy branch 3 times, most recently from 61cd4d4 to 2e4de39 Compare June 24, 2026 14:05
@j-c-m

j-c-m commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

I agree and don't expect to see much of a network performance/throughput difference. What I am targeting is less memory and possible cpu pressure during the pure uploading phase. Going to continue refining this and will update here.

@j-c-m
j-c-m force-pushed the feature/sendfile-zero-copy branch 3 times, most recently from d75a942 to bbe3d9f Compare June 24, 2026 15:39
@j-c-m

j-c-m commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Cleaned up, letting it soak test seeding on a large number of torrent files. Looking good so far, moderate effect as it can't be active on encrypted sends.

@j-c-m
j-c-m force-pushed the feature/sendfile-zero-copy branch from bbe3d9f to 56e472d Compare June 26, 2026 17:42
@j-c-m

j-c-m commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

This has been working OK for me. I think it is a useful option to reduce memory pressure in some scenarios, moving out of draft.

@j-c-m
j-c-m marked this pull request as ready for review June 29, 2026 15:59
@rakshasa

rakshasa commented Jul 1, 2026

Copy link
Copy Markdown
Owner

While it reduces memory pressure, does it improve throughput?

@j-c-m

j-c-m commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

While it reduces memory pressure, does it improve throughput?

Most of the time, it performs how you would expect. On filesystems where the sendfile() is a true zero-copy it is faster (~2x), less memory, less cpu (~1/2). On filesystems where it's not, it's slightly worse than mmap(piece)->send(16k) for max throughput. When network limited, which seems to be more real-world, the slower max throughput case uses slightly more cpu but still less user process memory.

Linux/EXT4 - faster
Linux/XFS - faster
Linux/ZFS - slower
FreeBSD/FFS - faster
FreeBSD/ZFS - faster

@rakshasa

rakshasa commented Jul 2, 2026

Copy link
Copy Markdown
Owner

I've been doing refactoring of various parts of libtorrent since becoming active again.

The core of peer connections and mmap handling is very stable and not something I want to touch yet without good reason.

I'd prefer to have it be tested irl for a while before merging.

@j-c-m j-c-m changed the title Add cross-platform sendfile uploads with lazy chunk mapping Add cross-platform sendfile uploads with fallback to normal send Jul 2, 2026
@j-c-m

j-c-m commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

I will continue to let it run for more testing, the changes in peer_connection_base my intent was to keep current send path logic the same unless sendfile is enabled, the diff makes it look worse than it is.

@j-c-m
j-c-m force-pushed the feature/sendfile-zero-copy branch from 56e472d to a1c1d80 Compare July 6, 2026 18:41
@j-c-m

j-c-m commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

update for 09b150f m_fileDesc -> file_descriptor()

Introduce a unified sendfile_stream() abstraction for zero-copy piece
uploads on Linux, FreeBSD, and macOS/Darwin, each with its own syscall
signature detected at configure time.

When pieces.sendfile is enabled on plaintext peers, uploads resolve file
fd and offset via FileList::at_file() without mmap, falling back to
chunk mapping only for encryption, padding, or sendfile errors.

Expose pieces.sendfile, pieces.sendfile.set, and sendfile stats via
ChunkManager for rtorrent configuration.
@j-c-m
j-c-m force-pushed the feature/sendfile-zero-copy branch from a1c1d80 to 90f5cf2 Compare July 6, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants