Zebra-only custom Testnet lab

This lab implements the configuration, execution, and monitoring requested in Zebra issue #9159. It starts three Zebra nodes on an isolated custom Testnet, a crawler-backed DNS seeder, and Prometheus. It is a reproducible local demonstration; it does not provide permanent public hosting.

What it proves

• Every production lab node uses the same custom network name, four-byte network magic, activation schedule, difficulty limit, and disabled proof of work.
• Two seed nodes use reciprocal fixed bootstrap addresses and are crawled using the custom magic and P2P port by zcash-seeder.
• The observer starts only after the seed nodes are healthy and discovers its initial peer through seed.zebra.test, without a fixed peer address.
• Blocks generated on zebra-a propagate to the other two Zebra nodes.
• RPC, health, metrics, DNS, and Prometheus are exposed only on host loopback.
• Persistent volumes retain chain state across container restarts.

flowchart LR
  S[zcash-seeder<br/>seed.zebra.test] -->|crawl custom P2P| A[Zebra A]
  S -->|crawl custom P2P| B[Zebra B]
  O[Zebra observer] -->|DNS bootstrap| S
  O <-->|custom P2P| A
  O <-->|custom P2P| B
  P[Prometheus] -->|scrape| A
  P -->|scrape| B
  P -->|scrape| O

Loading

Requirements

• Linux or WSL2 with Docker Engine and the Compose plugin
• bash, curl, and Python 3 on the host
• About 1.5 GB of image storage; runtime state is small for this short chain

The runtime base images are pinned by digest. The seeder source is pinned to commit 11dac3e8465033f89a2f96e1dc3cb856f3590668 and compiled in a multi-stage image. Debian packages still come from the base image's configured repositories, so a later rebuild can pick up newer package revisions.

Run it

From the repository root:

custom-testnet/scripts/lab.sh up

The first clean start commonly takes 30–90 seconds while the seeder verifies its seed nodes. The bootstrap script submits the included standard Testnet genesis block to each empty Zebra state, advances a chain below height three to that target one block at a time, and verifies propagation after each new block.

Inspect the lab:

custom-testnet/scripts/lab.sh status
custom-testnet/scripts/dns-query.py
custom-testnet/scripts/rpc.sh zebra-observer getblockchaininfo
custom-testnet/scripts/rpc.sh zebra-observer getpeerinfo

Run the complete repeatable check, optionally deleting existing lab state first:

custom-testnet/scripts/verify.sh
custom-testnet/scripts/verify.sh --clean

The check validates Compose and Prometheus configuration, DNS bootstrap evidence, authenticated RPC, Zebra-only peers, a shared chain tip, health endpoints, an actual Prometheus alert transition, rejection of a node with different network magic, and persistence across an observer stop/start cycle. It stops all lab containers when it finishes and retains their volumes unless lab.sh reset is used.

Prometheus is available at http://127.0.0.1:19090. Its rules report an unreachable node, a node without peers, and repeated failed outbound handshakes.

Stop containers while retaining state, or remove the lab and its volumes:

custom-testnet/scripts/lab.sh down
custom-testnet/scripts/lab.sh reset

Local ports

Service	Host address	Purpose
DNS seeder	127.0.0.1:1053/udp	seed.zebra.test A records
Zebra A RPC / health / metrics	127.0.0.1:18241 / 18101 / 19101	Node control and observation
Zebra B RPC / health / metrics	127.0.0.1:18242 / 18102 / 19102	Node control and observation
Observer RPC / health / metrics	127.0.0.1:18243 / 18103 / 19103	DNS-bootstrapped node
Prometheus	127.0.0.1:19090	Targets, queries, and alerts

Zebra RPC uses a random cookie held inside each node's private volume. The rpc.sh helper reads it through docker compose exec; the cookie is never written into the repository.

Configuration choices

The private network is named ZebraOnlyLab and uses magic 5a435348. NU7 activates at height 1, proof of work is disabled, slow start and funding streams are empty, and the mempool debug activation is height 0. These settings make local block generation possible while still exercising Zebra's configured-Testnet code path.

The bridge uses 198.18.0.0/24, an address range reserved for benchmarking. zcash-seeder deliberately rejects private RFC1918 addresses as unroutable, so common Docker bridge ranges cannot be used for this contained crawl.

The lab uses zcash/zcash-seeder because its current command line accepts a custom network magic, P2P port, and explicit seed set. Zcash Foundation's newer Zeeder currently selects the built-in Mainnet or Testnet parameters and cannot crawl this configured Testnet without a code change.

Containers use read-only root filesystems, dropped Linux capabilities, process and memory limits, and local log rotation. Zebra retains only the four capabilities required by its upstream entrypoint to prepare its state volume and change to the unprivileged zebra user.

Services retry a failed process up to three times. They do not use an always-on restart policy, and the verification workflow shuts the stack down after collecting evidence.

Limits

• The DNS implementation in zcash-seeder serves UDP. The Compose file therefore exposes only UDP port 1053.
• The included genesis bytes are the standard Zcash Testnet genesis block because configured Zebra Testnets derive that genesis hash. Zebra auto-commits genesis only for Regtest, so bootstrap submits it over loopback RPC to each empty node.
• The short default chain exercises non-finalized state. It does not claim coverage of long-running finalization, network partitions, peer scoring, or denial-of-service behavior.
• The Docker bridge permits outbound traffic, while the custom consensus magic separates this lab from other Zcash networks. The fixed 198.18.0.0/24 subnet can conflict with software that also uses the benchmarking range and may be changed consistently in the Compose and node files when necessary.
• This is a compact development network with two crawler seeds. A production seeder needs independently operated public nodes, authoritative DNS delegation, abuse controls, backups, and an operator responsible for availability.