Torrent client

A BitTorrent client written in Go, with a web dashboard and a CLI that share one headless backend. It takes a .torrent file or a magnet link, finds peers through trackers, the DHT, PEX, and inbound connections, downloads the content from those peers in parallel with a rarest-first picker, verifies the integrity of every piece, and streams each verified piece straight to disk as it arrives. It handles multi-file torrents (a whole directory tree, not just a single file), and re-running (or resuming) it resumes an interrupted download or repairs corrupted pieces with no saved progress. It can also seed — listen for inbound peers and upload to them — not just download, and it uploads while still downloading so tit-for-tat reciprocation works in our favor too.

The web dashboard drives a headless engine (backend/engine) that manages many torrents at once; the CLI is a thin single-torrent tool over the same protocol packages. See Repository layout & architecture below for how the pieces fit together.

If you want to build your own, take a look at this excellent blog post to get started: https://blog.jse.li/posts/torrent/. This is a great project to get a feel for how HTTP, binary protocols, and channels work in Go.

Running the web dashboard

Start the local dashboard:

go run ./clients/web

then open http://127.0.0.1:8081 in a browser. From there you can add magnet links or .torrent files, watch each torrent's progress — shown in bytes ("512.3 MB / 1.4 GB") with live download/upload speeds in MB/s — and pause / resume / remove them, and copy a magnet link for any entry (built from its info hash — resolvable via DHT even without trackers). Remove keeps the downloaded files but clears the app's metadata for the torrent (its magnet metadata cache and its own .torrent copy); the Clear data button next to it (with a confirmation) additionally deletes the downloaded files themselves (DELETE /api/torrents/{id}?data=1, see docs/adr/0022). The engine manages every torrent concurrently, each in its own goroutine, and the page polls the REST API (GET /api/torrents) about once a second to stay live — which is also the sampling clock for the speed readings.

When a download finishes, the engine keeps seeding it (status seeding) — announcing to its trackers with left=0 and serving inbound leechers through the shared listener — until you pause or remove it, like qBittorrent. Pausing a seeding torrent stops it; resuming re-verifies the data on disk (a quick hash scan) and goes straight back to seeding. Set seed.afterDownload to false in config.json to restore the old behavior, where a completed torrent just goes inert (see docs/adr/0019).

The web app also runs a watch folder by default: every 10 seconds it scans your download directory (configurable via watch.dir) for new .torrent files and adds them automatically — save a .torrent there and the download just starts (see docs/adr/0021 and the watch.* config notes below). By default the consumed .torrent file is then deleted from the watch folder (the engine keeps its own copy); set watch.deleteAfterAdd to false to leave it in place. A file whose torrent you already have is left alone.

The torrent list persists across restarts: it is saved to a session.json under your OS config directory (~/Library/Application Support/torrent-client on macOS), so active torrents resume and paused ones stay paused when you start the server again. Added .torrent files are copied into the app's own directory, so a later resume survives the original being moved or deleted.

The server binds 127.0.0.1 only — it is an unauthenticated local dashboard and must never be reachable off-host. Downloads land under the same default parent directory as the CLI (~/Downloads, or downloadDir from config.json); pass -dir to override it, or -port to serve on a different local port. You can also build a standalone binary (the frontend is go:embed-ed, so it ships as one file):

go build -o torrent-web ./clients/web
./torrent-web

macOS: open magnet links & .torrent files with the dashboard

On macOS you can make this app the system default handler for magnet: links (clicked in Safari, Firefox, Chrome, anywhere) and .torrent files (see docs/adr/0027). Two pieces:

make install-app   # builds dist/TorrentWeb.app (bare swiftc) into ~/Applications
make agent         # installs the torrent-web LaunchAgent (does not start it)

Then, once:

1. Double-click ~/Applications/TorrentWeb.app — it offers to register itself as the default handler for magnet links and .torrent files (macOS shows a confirmation per claim), then quits. Firefox users: the first magnet click shows Firefox's "Choose application" dialog — pick TorrentWeb and tick "Always use this application".

2. Start the server agent (stop any manually-started ./torrent-web first — they'd fight over the port):

   launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.maaverik.torrent-web.plist

From then on the server starts at login and launchd restarts it if it ever exits (KeepAlive) — logs land in ~/Library/Logs/torrent-web.log. The agent runs the repo-root binary, so deploying a change is:

go build -o torrent-web ./clients/web
launchctl kickstart -k gui/$(id -u)/com.maaverik.torrent-web   # restart on new binary

To stop the server on purpose use launchctl bootout gui/$(id -u)/com.maaverik.torrent-web (a plain kill gets undone by KeepAlive within moments). The opener app itself is a UI-less forwarder: each opened link/file becomes one POST to the local API and the dashboard is the confirmation UI; failures (e.g. server not installed) show a dialog. If the server simply isn't running yet, the opener kickstarts the agent and retries for ~12 s before giving up.

CLI

The CLI is a thin single-torrent tool. With the project checked out locally, download from a .torrent file:

go run ./clients/cli <path/to/torrent/file> [parent/directory]

or from a magnet link with the -m / --magnet flag:

go run ./clients/cli -m "magnet:?xt=urn:btih:<info-hash>&tr=<tracker>" [parent/directory]

For example:

go run ./clients/cli debian.torrent ./downloads
go run ./clients/cli -m "magnet:?xt=urn:btih:abc...&tr=udp://tracker.example:6969" ./downloads

The final argument is an optional parent directory (default ~/Downloads), not an exact output path. The torrent's contents always nest under the torrent's own name: a single-file torrent writes <dir>/<name> (a file), and a multi-file torrent writes <dir>/<name>/<path...> (a directory tree). So go run ./clients/cli debian.torrent ./downloads produces ./downloads/debian-12.iso. In file mode the first argument is the input .torrent; in magnet mode the -m/--magnet flag carries the link. The two modes are mutually exclusive. Paths drawn from a torrent are untrusted and strictly validated, so a torrent can never write outside its own directory.

Magnet links are resolved via HTTP(S) and UDP trackers, and via the DHT (BEP 5). A trackerless magnet that relies solely on the DHT is supported, as long as dht.enabled is on (the default — see docs/adr/0014, which supersedes the earlier docs/adr/0001 exclusion).

Seeding

To seed a file, add the --seed flag — it works with both a .torrent and a magnet:

go run ./clients/cli --seed debian.torrent ./downloads
go run ./clients/cli --seed -m "magnet:?xt=urn:btih:abc...&tr=udp://tracker.example:6969" ./downloads

The seed target follows the same rule as a download: the optional parent directory (default ~/Downloads) under which the torrent's contents live at <dir>/<name>. --seed is download-if-needed: it makes sure that location is a complete, verified copy first. It scans the data on disk piece-by-piece; if every piece is present and verifies it seeds directly, and if any pieces are missing or corrupt it fetches just those before seeding. There is nothing to refuse — an incomplete or damaged directory is simply completed rather than treated as an error. A seeder then runs until you interrupt it — pressing Ctrl-C sends a clean stopped to the tracker before exiting.

Use --port to choose the TCP port the client listens on and announces (default 6882 — qBittorrent typically owns 6881):

go run ./clients/cli --seed --port 51413 debian.torrent ./downloads

You can also build a standalone binary:

go build -o torrent-client ./clients/cli
./torrent-client debian.torrent ./downloads
./torrent-client -m "magnet:?xt=urn:btih:abc...&tr=http://tracker.example/announce" ./downloads

While running, the client logs to standard error. By default it prints just a few milestone lines per torrent (resolving, metadata resolved, how many peers it found, download complete, seeding started/stopped); pass -debug for the full per-piece / per-peer / per-tracker detail (see Logs & debugging).

Pausing & resuming

Press Ctrl-C once to pause: the client stops cleanly (exit code 0) and prints "Paused — progress saved to disk." Every piece that had finished downloading was already written to disk the moment it verified, so to resume you simply re-run the exact same command (the same .torrent or magnet, to the same directory) — it scans what's on disk, logs Resuming: N of M pieces already on disk, and fetches only the pieces that are still missing. The same is true after a crash or a kill: nothing is lost but the one piece that was in flight. (A second Ctrl-C force-quits immediately if a graceful pause ever hangs.) Resume is piece-granular, and needs no state file — the on-disk data plus the piece hashes are the progress.

For a magnet, the first resolve caches the fetched metadata (the info dict) at <download-dir>/.torrent-meta/<info-hash>. A re-run reads that cache and skips the tracker/peer resolution entirely, so a magnet resumes instantly and keeps working even when its trackers are momentarily unreachable — instead of failing to re-fetch the metadata it already had. The cache is regenerable and safe to delete (it's re-verified against the info hash on load).

Beyond -m/--magnet, --seed, and --port, every tunable also has its own CLI flag (one per config field — peer pool sizes, timeouts, retry counts, the download directory, and so on), and the same settings can be put in a config.json file next to the binary so you don't have to pass them each run. A flag always overrides the file, and the file overrides the built-in default. See Configuration below for the full list and an example file.

Configuration

Every tunable the client has lives in one place and is resolved by a fixed precedence: an explicitly-set CLI flag beats a field in the config file, which beats the compiled-in default. Each config field has an equivalent CLI flag that overrides it for a single run; the file is for settings you want to keep.

The config file is config.json and it is read from next to the binary — the directory of the running executable, not your current working directory. It is created by hand (there is no command to generate or dump one) and auto-loaded when present: if it is absent the client just uses its defaults, and if it is present but contains malformed JSON or an out-of-range value the client fails fast with a clear message rather than running on a half-applied config. Durations are written as human-readable strings like "5s" or "2m".

go run caveat: go run compiles the binary into a temporary directory and runs it from there, so a config.json in your project root is not next to that throwaway binary and will not be picked up. To use a config file, build the binary (go build -o torrent-client ./clients/cli, or ./clients/web) and put config.json beside it. Both clients read the same config.

You only need to include the fields you want to change — anything you omit keeps its default. Here is a complete config.json showing every field at its default:

{
  "downloadDir": "",
  "logLevel": "info",
  "download": {
    "maxPeers": 100,
    "backlog": 64,
    "blockSize": 16384,
    "pieceTimeout": "30s",
    "reannounceInterval": "30s",
    "peerLowWater": 3,
    "maxConcurrentDials": 20
  },
  "peer": {
    "dialTimeout": "8s",
    "handshakeTimeout": "8s",
    "bitfieldTimeout": "5s",
    "encryption": "prefer"
  },
  "tracker": {
    "announceTimeout": "15s",
    "udpTimeout": "12s",
    "udpRetries": 3
  },
  "metadata": {
    "concurrency": 5,
    "dialTimeout": "5s",
    "opTimeout": "15s",
    "reannounceInterval": "20s"
  },
  "seed": {
    "port": 6882,
    "handshakeTimeout": "10s",
    "idleTimeout": "2m",
    "minReannounce": "1m",
    "stoppedTimeout": "5s",
    "defaultInterval": "30m",
    "afterDownload": true,
    "maxOutbound": 20
  },
  "dht": {
    "enabled": true,
    "port": 0,
    "queryTimeout": "2s",
    "lookupInterval": "3m",
    "bootstrapNodes": [
      "router.bittorrent.com:6881",
      "dht.transmissionbt.com:6881",
      "router.utorrent.com:6881"
    ]
  },
  "pex": {
    "enabled": true,
    "interval": "60s"
  },
  "network": {
    "interfacePattern": "utun\\d+"
  },
  "watch": {
    "enabled": true,
    "dir": "",
    "interval": "10s",
    "deleteAfterAdd": true
  },
  "autoRemoveCompleted": {
    "enabled": true,
    "after": "72h",
    "checkInterval": "1h"
  }
}

Notes on the less obvious fields:

• downloadDir — the default parent directory for downloads; "" means ~/Downloads. The positional directory argument still overrides it per run.
• download.maxPeers — how many peer connections download in parallel (dialed and inbound together). 0 means unlimited; the default of 100 matches qBittorrent's per-torrent connection default (Transmission ships 50). Raising it mostly stresses a home router's NAT table rather than adding throughput.
• download.backlog — how many block requests are kept in flight to a single peer at once (pipelining). The default 64 is ~1 MiB in flight at the default blockSize — enough to sustain ~10 MB/s from a single peer even at a 100 ms WAN round trip, in line with production clients — and is shared across all the pieces a worker has in flight: each worker holds enough assigned pieces to cover the budget (derived from the piece size, capped at 32; see docs/adr/0018), so the request pipe never drains to zero at a piece boundary even on small-piece torrents.
• download.blockSize — the per-request block size in bytes. The default 16384 (16 KiB) is the protocol norm; most peers reject requests larger than 16384, so raising it is rarely safe.
• download.reannounceInterval — the floor on how often the download re-announces to its trackers, not a fixed period: each tracker is otherwise re-announced on the interval it returns (clamped up to this floor and down to a 30-minute ceiling), so a well-behaved tracker isn't hammered faster than it asked for. This value also doubles as the starvation retry cadence — while the download has fewer than peerLowWater connected peers (or has never found any), every tracker is retried at this floor regardless of its returned interval, so neither a slow start nor a decayed swarm stalls waiting out a long interval (default 30s; see docs/adr/0011 and docs/adr/0024).
• download.peerLowWater — the connected-peer count below which the peer supply is treated as starved and trackers are re-announced at the reannounceInterval floor instead of the interval they asked for. A swarm riding one half-dead connection for a 30-minute tracker interval is starved in every way that matters; the default 3 only fires when the swarm is nearly dead. Values <= 1 mean "starved only at zero connected", the pre-0024 behavior (see docs/adr/0024).
• download.maxConcurrentDials — how many peer dial attempts (TCP connect + handshake) may be in flight at once, independent of maxPeers (which caps established sessions). Without it, a fresh announce batch turns into one 60-peer SYN burst — through a VPN tunnel that loses packets and fails peers that were individually reachable. 0 disables pacing (default 20; see docs/adr/0024).
• peer.encryption — the outbound MSE/PE (Message Stream Encryption) stance: "prefer" (default) tries an encrypted handshake and falls back to plaintext when a peer doesn't speak MSE; "require" talks only to encrypted peers; and "disabled" is the pre-encryption behavior. Encryption widens the connectable pool to include the large slice of modern swarms that drop plaintext handshakes (seen as accept-then-reset failures where a 42-seeder torrent yielded one connectable peer). It's obfuscation, not confidentiality — RC4 per the frozen spec — and near-free (a ~100µs handshake, cheaper per byte than the WireGuard tunnel it already rides). Applies to outbound connections; accepting encrypted inbound is a deferred phase 2. See docs/adr/0025.
• logLevel — the log verbosity: "info" (milestones only, the default), "debug" (the per-piece/per-peer/per-tracker firehose), or "warn". The -debug / -logLevel flags override it per run — see Logs & debugging.
• tracker.announceTimeout — the hard ceiling on a single tracker announce (default 15s). All trackers are announced to concurrently, and any one that hasn't answered within this is abandoned. (It also subsumes what used to be a separate HTTP timeout.)
• metadata.concurrency — how many peers are dialed in parallel when fetching a magnet's metadata; the first to return verified metadata wins and the rest are cancelled.
• metadata.reannounceInterval — the same floor/starvation-retry cadence as download.reannounceInterval, but for the trackers consulted while resolving a magnet (default 20s).
• seed.port — the TCP port the seeder listens on and announces; the --port flag overrides it.
• seed.afterDownload — whether the engine (web dashboard) keeps seeding a torrent after its download completes, until paused or removed (default true, see docs/adr/0019). The CLI ignores it — there, seeding stays opt-in via --seed, so a one-shot download still exits when done.
• seed.maxOutbound — how many leechers a seeding torrent dials out to concurrently (default 20, 0 disables). This is what makes seeding actually upload behind a VPN: the announced <vpn-ip>:port is unreachable from outside the tunnel, so inbound never arrives — instead the seed takes the peer lists its announces (and DHT) already return, connects to those leechers itself, and serves over the outbound connections (see docs/adr/0023). Dialed peers that turn out to be other seeds never request anything and are reaped by seed.idleTimeout.
• dht.enabled — turns on the DHT (BEP 5) as a second peer source alongside trackers, on by default. A torrent with zero (or dead) trackers is still reachable as long as the DHT can reach the swarm (see docs/adr/0014).
• dht.port — the UDP port the per-torrent DHT node's socket binds to; 0 (the default) picks an ephemeral port, which is fine since nothing needs to dial this node directly.
• dht.bootstrapNodes — the well-known routers used to seed the routing table on first use.
• pex.enabled — turns on ut_pex (BEP 11) as a third peer source: peers already connected for a download gossip other peers they know about, piggybacking on connections the swarm already has open. On by default (see docs/adr/0017).
• pex.interval — the minimum time between two ut_pex messages sent to the same peer (default 60s), per BEP 11 etiquette — only the delta is sent, not the whole swarm state each time.
• network.interfacePattern — an anchored Go regexp naming the ONLY network interface(s) torrent traffic may use. The default is utun\d+ — the VPN kill-switch ships armed, so with no WireGuard tunnel up the client moves zero bytes. Set it to "" for unbound traffic on any interface. See VPN kill-switch below.
• watch.enabled / watch.dir / watch.interval / watch.deleteAfterAdd — the web app's watch folder: every interval (default 10s) it scans ONE directory (no recursion) for new .torrent files and auto-adds them as downloads. dir empty (the default) means the effective download directory itself, so a .torrent saved where your downloads land just starts. Files already offered are remembered across restarts (in the session), so removing a torrent from the dashboard doesn't resurrect it while its file still sits in the folder; a half-copied file is retried until it parses. deleteAfterAdd (default true) removes a .torrent from the folder once it's been added as a NEW torrent — the engine keeps its own owned copy, so the original is redundant. It only ever deletes the exact regular .torrent file it just consumed (never a subdirectory or symlink), only after a successful add, and never a duplicate whose torrent you already have. Set it to false to leave dropped files in place (or -watch.deleteAfterAdd=false on the CLI). See docs/adr/0021.
• autoRemoveCompleted.enabled / .after / .checkInterval — the web app's completed-torrent sweeper: every checkInterval (default 1h) it removes any torrent whose download finished more than after ago (default 72h), whether it is still seeding or gone inert. Removal drops the torrent from the list and clears the metadata the engine stored for it (the owned .torrent copy and the magnet metadata cache) but keeps the downloaded files — the same thing the Delete button does, as opposed to Purge/Clear data, which also deletes the content. Paused and errored torrents are never touched. With AfterDownload on, this doubles as a time-based seeding limit ("seed three days, then drop"). Set enabled to false (or -autoRemoveCompleted.enabled=false on the CLI) to turn it off. See docs/adr/0026.

VPN kill-switch (interface binding)

network.interfacePattern (also settable via -network.interfacePattern) pins every socket the client opens — peer connections, tracker announces (HTTP and UDP), DHT, the inbound listener, and even DNS lookups — to one network interface, selected by an anchored Go regexp. The motivating use is a WireGuard VPN on macOS, whose tunnels are named utunN and get a new N on every reconnect — which is why utun\d+ is the compiled-in default: the kill-switch ships armed, and setting the pattern to "" disables it:

go run ./clients/web   # kill-switch already armed by default
go run ./clients/web -network.interfacePattern ''   # unbound, any interface

Among matching interfaces that are up with a usable address, the highest-numbered one is chosen (the freshest tunnel; macOS's own link-local-only utuns are skipped automatically). The binding fails closed: when nothing matches — the VPN is off — every dial, announce, lookup, and listen fails immediately, and zero torrent packets leave on any other interface. Torrents simply stall (the dashboard shows a "traffic suspended" banner, also available at GET /api/network) and self-heal the moment the VPN comes up: a watcher notices the new interface within ~2 seconds, rebinds the long-lived sockets (inbound listener, DHT), and the existing re-announce / peer-retry loops bring downloads back without a restart. Binding is enforced with IP_BOUND_IF on macOS and SO_BINDTODEVICE on Linux (an unsupported platform rejects the setting at startup); see docs/adr/0020.

Logs & debugging

Logging is structured (log/slog) and leveled, and works the same for both clients. A normal run is quiet: you get a handful of milestone lines per torrent — resolving, metadata resolved, Resuming N of M pieces, the first found-peers summary, download complete, seeding started/stopped. Each line is tagged with a short torrent id (the first 8 hex chars of the info hash, e.g. torrent=1a2b3c4d), so when several torrents run at once you can tell whose line is whose.

When you need to diagnose something, pass -debug (or -logLevel debug) to either the CLI or the web server to open the firehose — per-piece downloads, per-announce, per-peer handshakes, per-peer metadata failures, and per-tracker announce failures:

go run ./clients/cli -debug debian.torrent ./downloads
go run ./clients/web -debug

The level also comes from the top-level logLevel config field ("info" default, "debug", or "warn"); the flag overrides it for a single run.

Features

This is everything the client currently does:

• .torrent file parsing. Reads a bencoded metainfo file and extracts the announce (tracker) URL, the file name, total length, piece length, and the list of per-piece SHA-1 hashes.
• Magnet link support. Parses magnet:?xt=urn:btih:… links (info hash in hex or base32, plus tr= trackers and an optional dn= name), then fetches the metadata from peers so the download proceeds exactly as it would from a .torrent file.
• Metadata exchange. Implements the extension protocol (BEP 10) and ut_metadata (BEP 9) to download the info dictionary from a peer in 16 KiB pieces and verify it against the magnet's info hash before trusting it.
• Info hash computation. Computes the SHA-1 hash of the bencoded info dictionary, which uniquely identifies the content to the tracker and to peers. The hash is taken over the raw canonical bytes of the info subtree, so no key is ever dropped — multi-file and extension keys hash correctly (see docs/adr/0004).
• HTTP and UDP tracker communication, respecting each tracker's cadence. Announces over HTTP(S) (standard query parameters, compact=1, numwant=80) and over UDP (BEP 15 connect/announce, also numwant=80), fanning out to all of a download's trackers concurrently. Each tracker is re-announced on the interval it returns (clamped between download.reannounceInterval as a floor and a 30-minute ceiling) rather than one flat period, and a peer that was merely unreachable is re-offered with exponential backoff (1m<<attempts, capped at 15m) instead of written off forever (see docs/adr/0011).
• Compact peer list parsing, including IPv6. Decodes the compact 6-bytes-per-peer IPv4 list (peers) and, from HTTP trackers, the 18-bytes-per-peer IPv6 list (peers6, BEP 7), dialing whichever address families a dual-stack tracker returns (see docs/adr/0012).
• Parallel peer discovery across trackers, DHT, and PEX. Announces to all of a download's trackers at once, runs a per-torrent DHT (BEP 5) lookup, and receives peers gossiped by already-connected peers over PEX (BEP 11) — all three sources fan into one deduped-by-IP:port stream. For magnets, the metadata is fetched from a bounded pool of peers in parallel — the first peer to return hash-verified metadata wins and cancels the rest — so a dead tracker or peer no longer stalls the resolve (see docs/adr/0007).
• DHT (BEP 5) peer discovery. A per-torrent backend/dht node (KRPC over UDP, 160 k-buckets, lazy eviction) bootstraps from the configured routers, runs an iterative get_peers lookup toward the info hash, and announce_peers to the closest responders — re-running periodically for the life of the download. A torrent with zero (or dead) trackers is still reachable as long as the DHT can reach the swarm, including a trackerless magnet (see docs/adr/0014).
• Watch folder. The web app polls a single directory (default: the download dir, every 10s) for new .torrent files and auto-adds them, remembering offered files across restarts so removed torrents stay removed, and (by default) deleting each consumed .torrent once its download has been added (see docs/adr/0021).
• Auto-remove completed torrents. The web app drops a torrent from the list once its download has been finished past a retention window (default 3 days), clearing the metadata it stored (owned .torrent copy + magnet cache) while keeping the downloaded files — a time-based "seed then reclaim" lifecycle (see docs/adr/0026).
• Interface binding (VPN kill-switch). An anchored regexp (network.interfacePattern, e.g. utun\d+) pins every socket — peer dials, tracker announces, DHT, the inbound listener, and DNS lookups — to one interface, picked fresh by pattern so WireGuard's new-utunN-per-reconnect just works. Fails closed when nothing matches (zero packets off-VPN) and self-heals when the interface returns (see docs/adr/0020).
• PEX (BEP 11) peer exchange. Peers already connected for a download gossip other peers they know about over the extension protocol — a free third peer source that costs nothing beyond bytes on a connection already open. Sends are delta-only (BEP 11 etiquette), capped at 50 entries, on a per-peer interval (see docs/adr/0017).
• Tolerant peer handshakes and post-handshake setup. Performs the BitTorrent handshake ("BitTorrent protocol" pstr, 8 reserved bytes, info hash, peer ID) over TCP, then loops reading whatever a real peer sends next instead of requiring a leading bitfield: an extended handshake, a keepalive, or a Have are all tolerated, and a peer that sends no bitfield at all is adopted with an empty one that later Haves still update correctly (see docs/adr/0010).
• Peer wire protocol messages. Serializes and parses the standard length-prefixed messages: choke, unchoke, interested, not interested, have, bitfield, request, piece, cancel, and the BEP 10 extended message, plus zero-length keep-alive messages.
• Bitfield tracking. Reads each peer's bitfield (which pieces they have) and supports querying and updating individual piece bits.
• Rarest-first, cross-piece-pipelined downloading with endgame. Spawns one goroutine worker per peer. A mutex-guarded picker hands out the least available pending piece a peer's bitfield covers (ties broken randomly), each worker keeps a lookahead queue of pieces (deep enough to cover the whole backlog budget, whatever the piece size — docs/adr/0018) with block requests in flight so the pipe never drains at a piece boundary, and once nothing is left pending but some pieces are still incomplete, endgame duplicates the tail across idle workers and Cancels whichever copy loses the race (see docs/adr/0013).
• Per-piece integrity verification. Every downloaded piece is checked against its SHA-1 hash from the metainfo. Pieces that fail are requeued to be retried from another peer.
• Automatic retries and resilience. A piece a peer doesn't have, or that fails its integrity check, is returned to the work queue. Unresponsive peers are bounded by read/write deadlines so they can't stall the download.
• Multi-file torrents. Handles torrents whose info lists many files, not just one. The pieces still cover the single contiguous concatenation of all files in order; the file list says where to slice that stream onto disk, and the contents are written as a directory tree under the torrent's name. Untrusted file paths are strictly validated so a torrent can't escape its own directory.
• Streaming to disk. Each verified piece is written straight to disk as it arrives through a storage layer that maps the torrent's byte stream onto its one-or-more files (handling pieces that straddle a file boundary). The whole file is never held in RAM, so a multi-gigabyte torrent downloads in bounded memory.
• Resume & repair. Re-running the client on the same destination resumes an interrupted download or repairs corrupted pieces, with no saved progress file: it scans the bytes on disk, hashes each piece, and fetches only the pieces that are missing or fail their hash (see docs/adr/0005).
• File writing. Streams each verified piece into its place across the backing file(s) (handling the smaller final piece correctly), rather than assembling the whole download in one in-RAM buffer before writing.
• Seeding. Listens for inbound peer connections on the chosen --port, and on each one validates the info hash, sends its bitfield and an unchoke, and answers every request by reading the block from disk via ReadAt (so the whole file is never held in RAM). It unchokes all peers rather than running the choking algorithm (see docs/adr/0003), verifies the file piece-by-piece before seeding, and drives the full tracker lifecycle — started (with left=0 when seeding), periodic re-announces, completed when a download finishes, and stopped on graceful shutdown. In the web dashboard's engine, seeding starts automatically when a download completes and runs until the torrent is paused or removed (seed.afterDownload, default on — see docs/adr/0019); in the CLI it remains opt-in via --seed.
• Upload while downloading (tit-for-tat reciprocation). A download connection is no longer fetch-only: it sends our bitfield on connect, broadcasts a Have to every connected peer as each piece verifies, and answers peers' own Request messages from already-verified pieces — the same reciprocation real swarms condition their unchoke decisions on (see docs/adr/0015).
• Inbound peer connections while downloading. One process-wide seed.Router binds the announced port and routes an inbound handshake to the right torrent by info hash: a torrent still mid-download adopts the connection as a full worker (capped at download.maxPeers), a completed one seeds it, and a paused one deregisters and stops accepting new inbound dials — so a torrent is reachable the whole time it's active, not just once finished. A bind failure just falls back to outbound-only rather than failing the run (see docs/adr/0016).
• macOS default handler + always-on server. A tiny UI-less .app (make install-app) registers as the system default for magnet: links and .torrent files and forwards each open straight to the dashboard's API, while a launchd LaunchAgent (make agent, RunAtLoad + KeepAlive) starts the server at login and restarts it if it dies — click a magnet anywhere and it's downloading (see docs/adr/0027).
• Configurable. Every tunable — peer pool sizes, timeouts, retry counts, the download directory, the seed port — lives in one Config and can be set from a hand-written config.json next to the binary or overridden per-run by a CLI flag (flag beats file beats default). See the Configuration section for the full schema and docs/adr/0006 for the rationale.

How it works

Both sources converge on a single fully-populated TorrentFile, after which the download is identical.

1. Build a TorrentFile.
   • From a .torrent: parse the bencoded metainfo directly (torrent.Deserialize, bencodeUtils).
   • From a magnet: parse the link (magnet), announce to find peers, fetch and SHA-1-verify the metadata from several peers in parallel — the first peer to return verified metadata wins and the rest are cancelled (metadata) — and parse it into the same struct (torrent.DeserializeFromMagnet).
2. Discover peers from three sources fanned into one deduped stream (torrent, tracker, dht, pex, peers). Trackers are announced to concurrently, each re-announced on the interval it returns (clamped to a configurable floor and a 30-minute ceiling — not one flat period) and a dead-on-first-dial peer is re-offered with exponential backoff instead of written off forever; a per-torrent DHT node runs an iterative get_peers lookup toward the info hash and re-runs it periodically; and PEX gossip from already-connected peers streams in as a third, free source. A download self-heals rather than stalling — it can't run dry as long as any of the three sources eventually yields peers (see docs/adr/0009, docs/adr/0011, docs/adr/0014, docs/adr/0017).
3. Connect to peers over TCP, handshake, and run the tolerant post-handshake setup: a bitfield adopts it, an extended handshake or keepalive is consumed and the loop continues, and a peer that never sends a bitfield is adopted with an empty one instead of being dropped. Our own bitfield is sent right after the handshake, so a peer can start requesting from us immediately (worker, handshake, docs/adr/0010, docs/adr/0015).
4. Open the layout and scan. Open the file(s) under the destination non-destructively through the storage layer (create what's missing, size each file, keep existing bytes), then hash every piece on disk to find the missing or corrupt ones — the work set (storage). On a fresh download the work set is every piece; on a resume or repair it is only what's left.
5. Download the work set in parallel, and serve requests back. A mutex-guarded rarest-first picker hands out the least-available pending piece each peer's bitfield covers; every worker keeps a lookahead queue of pieces pipelined against a shared backlog budget so the pipe never drains at a piece boundary; and once nothing is left pending, endgame duplicates the tail across idle workers and cancels the loser. Every verified piece is streamed straight to disk through the storage io.WriterAt, broadcast as a Have to every connected peer, and made servable — the same connection answers a peer's own Requests for pieces we've already verified while the rest of the download continues (swarm, message, bitfield, docs/adr/0013, docs/adr/0015). Meanwhile, a shared seed.Router (if one was built) routes any inbound dial for this info hash in as an adopted worker, so the torrent is reachable by peers dialing in even before it completes (seed, docs/adr/0016). When the work set is empty the data is complete.

When --seed is given the client takes a slightly different path (seed), but it shares the same open→scan→fetch core:

1. Open and scan the destination through storage and compute the work set, exactly as a download does. If the work set is non-empty (missing or corrupt pieces) those are fetched first; once it is empty the data is complete and ready to serve.
2. Listen for inbound peer connections on --port.
3. Announce started to the tracker(s), with left=0 since the data is complete, and re-announce at the tracker's interval.
4. Serve each inbound peer: read and validate the handshake, send the bitfield and an unchoke, then answer every request with the block read from disk via the storage io.ReaderAt.
5. Stop on Ctrl-C, sending stopped to the tracker(s) before exiting.

Repository layout & architecture

The project is one Go module (github.com/maaverik/torrent-client) split into a headless backend and thin clients:

backend/    protocol & logic packages + the engine (no UI, no transport)
clients/
  cli/      single-torrent command-line tool
  web/      local web dashboard (HTTP server + embedded frontend)
docs/adr/   architecture decision records

The guiding principle is a headless engine. backend/engine is a transport-agnostic Go core that manages many torrents at once behind a plain Go API (AddMagnet, AddTorrentFile, List, Pause, Resume, Remove, Close). All multi-torrent lifecycle, state, and persistence live in the engine; a client is just a thin adapter that translates its transport into engine calls and holds no torrent state of its own. The web client is an HTTP adapter today; a native Mac app is a planned future client over the very same engine, which is what the split buys us. The engine depends on a small Downloader interface rather than the torrent package directly, so its lifecycle logic is testable without the real protocol stack or a network. See docs/adr/0008 for the full rationale, including the JSON session.json persistence choice.

The backend's protocol/logic packages:

Package	Responsibility
backend/engine	Headless multi-torrent core: registry, per-torrent lifecycle/status, progress, and session.json persistence, behind a transport-agnostic Go API.
backend/bencodeUtils	Parsing bencoded torrent/tracker/info data; info-hash and piece-hash handling.
backend/magnet	Parsing magnet links (info hash, trackers, display name).
backend/torrent	Top-level TorrentFile model and the file/magnet constructors + download flow.
backend/tracker	Announcing to HTTP(S) and UDP trackers behind one scheme-dispatching call.
backend/metadata	Fetching and verifying the info dictionary from peers (BEP 10 + ut_metadata).
backend/peers	Compact peer-list deserialization and peer address formatting.
backend/handshake	BitTorrent handshake construction, parsing, and the extension-protocol bit.
backend/message	Peer wire protocol message IDs, serialization, parsing, and helpers.
backend/bitfield	Bitfield type for tracking which pieces a peer has.
backend/worker	Per-peer TCP connection: handshake, tolerant post-handshake setup (bitfield/extended-handshake/keepalive), sending our own bitfield, and sending messages.
backend/swarm	Download orchestration: the rarest-first picker, cross-piece pipelined block downloads, integrity checks, streaming each verified piece to an io.WriterAt, broadcasting Have, serving peers' Requests while downloading, and sending/receiving PEX gossip.
backend/storage	Maps the torrent's byte stream onto one or more files on disk (multi-file io.ReaderAt/io.WriterAt) and scans pieces for resume/repair.
backend/dht	BEP 5 Mainline DHT: KRPC over UDP, a k-bucket routing table, bootstrap, and the iterative get_peers/announce_peer lookup used as a second peer source.
backend/pex	BEP 11 ut_pex codec and per-peer delta bookkeeping (Tracker), used as a third peer source riding on connections the swarm already has open.
backend/netbind	Interface binding (VPN kill-switch): pattern-selected interface resolution, fail-closed bound dialers/listeners/resolver, and the change watcher that rebinds long-lived sockets.
backend/seed	Inbound peer serving: the per-torrent Seeder accept loop, plus the process-wide Router that binds one socket and routes inbound handshakes to the right torrent by info hash, and the seeding tracker lifecycle.
backend/config	The one Config struct of tunables, threaded into each consuming package.
clients/cli	Command-line entry point; parses -m/--magnet, --seed, and --port.
clients/web	HTTP adapter over the engine: REST API under /api/* plus a go:embed-ed vanilla frontend, bound to 127.0.0.1.

Testing

The project has an exhaustive unit and integration test suite covering every package. The tests use only the Go standard library (net.Pipe and loopback TCP sockets stand in for real peers, an in-process UDP socket and httptest stand in for trackers), so no network access or external dependencies are needed to run them.

Run the whole suite (race detector on, and bounded: -timeout 120s per package, because Go's default is 10 minutes and a single deadlocked test should fail fast with a stack dump instead of masquerading as a stalled run):

make test     # go test -race -count=1 -timeout 120s ./...

or run every gate at once — build, gofmt, vet, tests:

make check

Notable coverage:

• Unit tests for bencode parsing, info/piece hashing, magnet parsing (hex and base32 info hashes), bitfield operations, handshake serialization/parsing and the extension bit, every wire-protocol message, peer-list decoding, tracker URL construction, and piece-bounds math.
• Tracker tests for both schemes: HTTP announce via httptest and UDP announce (connect/announce, transaction-id validation, bounded-retry timeout) against an in-process UDP server.
• Protocol-level tests that drive a worker against a simulated peer, and metadata tests that fetch the info dictionary over the extension protocol (success, multi-piece pipelining, hash-mismatch rejection with fallback, and a peer that doesn't support the extension).
• storage tests: cross-file-boundary ReadAt/WriteAt, the single-file degenerate case, zero-length files, non-destructive open, and path-traversal rejection (.., absolute, separators, NUL), plus the piece scan that finds missing/corrupt pieces.
• Hashing tests: a hand-built multi-file info dict with a known hash, that the .torrent and magnet paths produce the same info hash, and that raw-byte hashing preserves an unknown info key (the bug the old struct re-marshal had).
• End-to-end tests: a swarm test that downloads a multi-piece file from an in-process seeder, and magnet tests that run DeserializeFromMagnet → Download against an in-process tracker (HTTP and UDP) plus a peer serving both metadata and pieces, verifying the bytes match the source.
• Seeding tests: a crown jewel runs a real seed.Serve and a real swarm download against each other in-process and asserts the bytes round-trip, plus a tracker-lifecycle test (started/left=0, periodic re-announce, stopped on shutdown) and info-hash rejection.
• Multi-file crown jewels: a multi-file torrent downloads end-to-end into the right paths and then seeds back across file boundaries; and a resume/repair test pre-seeds a file with some correct, some corrupt, and some missing pieces, then asserts the repaired file is correct and that only the missing/corrupt pieces were requested from the peer.
• dht crown jewel: TestFindPeersIterativeLookupEmitsPeersAndAnnounces builds a small fake DHT network out of scripted in-process UDP responders and asserts the iterative lookup both emits peers and announce_peers to the closest nodes, with backend/torrent's TestDownloadZeroTrackersUsesDHT proving a trackerless download completes purely via DHT.
• PEX crown jewel: TestDownloadCompletesViaPexGossipedPeer completes a download whose tracker knows only one peer, because that peer gossips the peer holding the rest of the data over ut_pex; TestDownloadDoesNotUsePexWhenDisabled is the negative control.
• Inbound-routing crown jewel: TestDownloadCompletesViaInboundPeerWithZeroTrackerPeers completes a download whose tracker returns zero peers, purely from a peer dialing in through the shared seed.Router.
• Reciprocation crown jewel: TestTitForTatChokedUntilServedThenCompletesDownload drives a fake peer that withholds UNCHOKE until we've served its Request, proving upload-while-downloading actually earns reciprocation.
• Endgame crown jewel: TestEndgameDuplicateAssignmentFinishesStalledLastPiece pits a stalling peer against a fast one on the last piece and asserts the download finishes without waiting out PieceTimeout, the stalling peer gets a Cancel, and the piece is counted/written exactly once despite the race.
• Rarest-first ordering: TestPickerTryPickChoosesRarestAmongCandidates and TestPickerTryPickBreaksTiesRandomly prove the picker prefers the least-available piece a peer's bitfield covers and fans out randomly across ties, plus TestCrossPiecePipeliningRequestsNextPieceBeforeCurrentFinishes proving the request pipe never drains at a piece boundary.

TODO

• Interactive pause/resume in the CLI without restarting (the web dashboard already pauses/resumes live via the engine; in the CLI, Ctrl-C pauses and re-running the same command resumes from disk)
• Native Mac app client over the same headless engine (the web dashboard is the first client; the engine split is what makes the Mac app just another adapter — see docs/adr/0008). The always-on half is done — the launchd LaunchAgent and the magnet/.torrent default-handler app shipped in docs/adr/0027 — so what remains is a native UI, if the browser dashboard ever stops being enough.
• Fill out the web dashboard toward qbittorrent-like features (it currently does add / progress / pause / resume / remove)
• store recent logs to debug any issues
• Inbound MSE/PE (Phase 2, deferred): outbound encryption ships (see peer.encryption and docs/adr/0025); accepting encrypted inbound connections is deferred until a VPN port forward makes inbound viable
• Reap unproductive peers early: when a peer is unchoked and has requests outstanding but sends no block for ~8–10s, drop it and free the slot instead of waiting on the coarse 30s read deadline. Behind a portless VPN the reachable-peer pool is only ~5–6, so a single zombie slot (observed: an MSE peer held a slot 30s delivering 0 bytes while 1362 pieces were pending) is a large fraction of capacity. Needs a conservative threshold so merely-slow seeds aren't dropped
• uTP transport (BEP 29) (deferred — not yet implemented)
• Continue to work even if the laptop goes to sleep.
• Add more UI features like multi-select, having all configs exposed as a settings page.