Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

4 Commits

Folders and files

Repository files navigation

Agent Team Test

High-performance UDP peer-to-peer communication application with real-time threading, lock-free data structures, and event-driven architecture.

Table of Contents


Overview

Agent Team Test is a UDP peer-to-peer communication test application designed for low-latency, high-throughput packet exchange between two nodes. Each node transmits periodic packets containing a lifesign counter and monitors the remote peer for communication loss or unstable timing.

Key characteristics:

  • Real-time threading with SCHED_FIFO and CPU affinity
  • Lock-free SPSC ring buffers for inter-thread communication
  • epoll-based event loop for timer-driven operations
  • CRC32-protected application packet format
  • Graceful shutdown via POSIX signal handling (SIGINT / SIGTERM)

Application Architecture

System Context

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”         UDP          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     β”‚ ◄──────────────────► β”‚                     β”‚
β”‚     Node A          β”‚    Peer-to-Peer      β”‚     Node B          β”‚
β”‚  (agent_team_test)  β”‚    Communication     β”‚  (agent_team_test)  β”‚
β”‚                     β”‚                      β”‚                     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
     src:port A                                   src:port B
     dst:port B                                   dst:port A

Each node is an independent instance of the same binary. Node A's destination is Node B's source, and vice versa.

Module Overview

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                        main (Entry Point)                          β”‚
β”‚  - CLI argument parsing                                            β”‚
β”‚  - Object wiring & initialization                                  β”‚
β”‚  - Event loop lifecycle                                            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚     app        β”‚    event     β”‚   socket     β”‚   thread            β”‚
β”‚                β”‚              β”‚              β”‚                     β”‚
β”‚ AppPacket      β”‚ EventLoop    β”‚ UdpNode      β”‚ UdpThreadManager    β”‚
β”‚ ArgParser      β”‚              β”‚              β”‚ LockFreeRingBuffer  β”‚
β”‚ SignalHandler  β”‚              β”‚              β”‚                     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                         stats                                      β”‚
β”‚                LatencyStats    TerminalUI                          β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                              timer                                 β”‚
β”‚                           TimerHandle                              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Thread Model

The application runs three execution contexts:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                         Process                                      β”‚
β”‚                                                                      β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚  Main Thread                                                   β”‚  β”‚
β”‚  β”‚  - Event loop (epoll)                                          β”‚  β”‚
β”‚  β”‚  - TX timer callback  ──► encodes packet ──► TX Ring Buffer    β”‚  β”‚
β”‚  β”‚  - Comm monitor timer callback                                 β”‚  β”‚
β”‚  β”‚  - Signal handler check                                        β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β”‚                                                                      β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”‚
β”‚  β”‚  RX Thread               β”‚    β”‚  TX Thread                   β”‚    β”‚
β”‚  β”‚  CPU Core: 2             β”‚    β”‚  CPU Core: 3                 β”‚    β”‚
β”‚  β”‚  SCHED_FIFO, Prio: 80    β”‚    β”‚  SCHED_FIFO, Prio: 70        β”‚    β”‚
β”‚  β”‚  SIGINT/SIGTERM blocked  β”‚    β”‚  SIGINT/SIGTERM blocked      β”‚    β”‚
β”‚  β”‚                          β”‚    β”‚                              β”‚    β”‚
β”‚  β”‚  do {                    β”‚    β”‚  do {                        β”‚    β”‚
β”‚  β”‚    recvfrom(socket)      β”‚    β”‚    pop(TX Ring Buffer)       β”‚    β”‚
β”‚  β”‚    push(RX Ring Buffer)  β”‚    β”‚    sendto(socket)            β”‚    β”‚
β”‚  β”‚    invoke RX callback    β”‚    β”‚  } while (running)           β”‚    β”‚
β”‚  β”‚  } while (running)       β”‚    β”‚                              β”‚    β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β”‚
β”‚                                                                      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
Thread Role Scheduling Priority CPU Core
Main Event loop, timers SCHED_OTHER default any
RX Thread Socket receive SCHED_FIFO 80 2
TX Thread Socket transmit SCHED_FIFO 70 3

Data Flow

                    Transmit Path
                    ─────────────
  TX Timer ──► AppPacket::encode() ──► TX Ring Buffer ──► TX Thread ──► sendto()
  (100 ms)         (main thread)        (lock-free)      (SCHED_FIFO)   (kernel)


                    Receive Path
                    ────────────
  recvfrom() ──► RX Ring Buffer ──► RX Callback ──► AppPacket::decode()
  (kernel)       (RX Thread)        (RX Thread)      (RX Thread)
                  (lock-free)                         β”œβ”€ lifesign update
                                                      β”œβ”€ interval measurement
                                                      └─ stability check

Packet Format

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
β”œβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”€
β”‚                         unique_id (32-bit)                        β”‚ Header
β”œβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”€
β”‚         lifesign (16-bit)         β”‚       data_length (16-bit)    β”‚
β”œβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”€
β”‚                                                                   β”‚
β”‚                       data (0..256 bytes)                         β”‚ Payload
β”‚                                                                   β”‚
β”œβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”Όβ”€β”€
β”‚                          crc32 (32-bit)                           β”‚ Footer
β””β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”΄β”€β”˜
  • unique_id: Identifies the sending node
  • lifesign: Auto-incremented counter per transmission
  • data_length: Payload length in bytes (max 256)
  • data: Application payload
  • crc32: CRC32 over header + data

Project Structure

agent_team_test/
β”œβ”€β”€ CMakeLists.txt              # Build configuration (C++26, CMake 3.20+)
β”œβ”€β”€ Makefile                    # Convenience targets: build, clean, run, test_node_1, test_node_2
β”œβ”€β”€ LICENSE                     # MIT License
β”œβ”€β”€ README.md                   # This file
β”‚
β”œβ”€β”€ include/                    # Public headers
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ AppPacket.hpp       # Packet encode/decode, comm monitoring
β”‚   β”‚   β”œβ”€β”€ ArgParser.hpp       # CLI argument parsing
β”‚   β”‚   └── SignalHandler.hpp   # POSIX signal handler (singleton)
β”‚   β”œβ”€β”€ event/
β”‚   β”‚   └── EventLoop.hpp       # epoll-based event loop
β”‚   β”œβ”€β”€ socket/
β”‚   β”‚   └── UdpNode.hpp         # UDP socket wrapper
β”‚   β”œβ”€β”€ thread/
β”‚   β”‚   β”œβ”€β”€ LockFreeRingBuffer.hpp  # SPSC lock-free ring buffer (template)
β”‚   β”‚   └── UdpThreadManager.hpp    # RX/TX thread lifecycle management
β”‚   └── timer/
β”‚       └── timer.hpp           # timerfd wrapper
β”‚
β”œβ”€β”€ include/                    # Public headers (continued)
β”‚   └── stats/
β”‚       β”œβ”€β”€ LatencyStats.hpp    # Percentile statistics (header-only template)
β”‚       └── TerminalUI.hpp      # Split-screen ANSI terminal dashboard
β”‚
β”œβ”€β”€ src/                        # Implementation
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ main.cpp            # Entry point, object wiring
β”‚   β”‚   β”œβ”€β”€ AppPacket.cpp       # Packet codec, CRC32, stability logic
β”‚   β”‚   β”œβ”€β”€ ArgParser.cpp       # --src / --dst argument parsing
β”‚   β”‚   └── SignalHandler.cpp   # sigaction setup, callback dispatch
β”‚   β”œβ”€β”€ event/
β”‚   β”‚   └── EventLoop.cpp       # epoll_wait loop, fd registration
β”‚   β”œβ”€β”€ socket/
β”‚   β”‚   └── UdpNode.cpp         # socket/bind/connect/send/recv
β”‚   β”œβ”€β”€ thread/
β”‚   β”‚   └── UdpThreadManager.cpp    # pthread create, affinity, SCHED_FIFO
β”‚   └── timer/
β”‚       └── Timer.cpp           # timerfd_create, timerfd_settime
β”‚
β”œβ”€β”€ config/                     # Runtime configuration (reserved)
└── script/                     # Utility scripts (reserved)

Module Details

app - Application Layer

Class / Function Responsibility
main() Wires all objects, runs event loop until shutdown
AppPacket Encode/decode packets with CRC32 integrity
Track lifesign, measure interval, detect comm loss
ArgParser Parse --src <addr>:<port> --dst <addr>:<port> from CLI
SignalHandler Singleton; installs SIGINT/SIGTERM via sigaction()
Thread-safe shutdown flag with std::atomic
Registers callback to stop EventLoop on signal
Supports registering additional shutdown callbacks

event - Event Loop

Class Responsibility
EventLoop Wraps Linux epoll for multiplexed I/O
Registers file descriptors with callbacks
Drives timer expiration and (optionally) socket I/O

socket - Network I/O

Class Responsibility
UdpNode Creates UDP datagram socket (SOCK_DGRAM)
Sets SO_REUSEADDR for quick restart
Binds to source address/port
Connects to destination (enables send/recv)
Exposes file descriptor for epoll or thread I/O

timer - Timer Management

Class Responsibility
TimerHandle Wraps Linux timerfd for high-resolution timers
Supports one-shot and periodic modes
Accepts std::function<void()> callbacks
Exposes fd for epoll registration

thread - Threading Infrastructure

Class Responsibility
UdpThreadManager Creates and manages dedicated RX and TX pthreads
Blocks SIGINT/SIGTERM in worker threads (pthread_sigmask)
Configures CPU affinity (pthread_setaffinity_np)
Configures SCHED_FIFO real-time scheduling
Tunes SO_RCVBUF / SO_SNDBUF socket buffer sizes
Sets SO_RCVTIMEO for clean RX thread shutdown
Handles ECONNREFUSED as transient (peer not ready)
Provides packet counters and drop statistics
LockFreeRingBuffer SPSC ring buffer (template, header-only)
Cache-line aligned (alignas(64)) to prevent false sharing
Acquire/release memory ordering for thread safety
Default: 1024 slots x 2048 bytes per slot

stats - Latency Benchmarking

Class / Template Responsibility
LatencyStats<> Collects nanosecond latency samples in a lock-free circular buffer
Computes p50, p95, p99, p99.9, p99.99 percentiles on demand
O(1) recording, O(N log N) computation via snapshot + sort
Cache-line aligned atomics, header-only template
RAII ScopedMeasurement for automatic timing
TerminalUI Split-screen ANSI terminal with pinned dashboard (upper 7 lines)
Scrolling packet log in lower region
Mutex-protected output for thread safety (RX thread + main thread)
Dashboard shows count, min, p50, p95, p99, p99.9, max per metric

Three latency metrics are measured in UdpThreadManager:

  • TX Send Latency β€” sendto() syscall duration
  • RX Processing Latency β€” recvfrom() return through queue push + callback
  • RX Inter-Packet Interval β€” time between consecutive received packets (jitter)

See README_LATENCY_BENCHMARKING.md for full architecture, percentile methodology, and performance analysis guide.


Build

Prerequisites

  • Linux (kernel 2.6.27+ for timerfd, epoll)
  • GCC 14+ or Clang 18+ (C++26 support)
  • CMake 3.20+

Commands

# Build
make build

# Clean
make clean

# Run (requires root for SCHED_FIFO)
make run

Or manually:

mkdir -p build && cd build
cmake ..
make

Usage

# Node A
sudo ./build/agent_team_test --src 127.0.0.1:5000 --dst 127.0.0.1:6000

# Node B (separate terminal)
sudo ./build/agent_team_test --src 127.0.0.1:6000 --dst 127.0.0.1:5000

Or use the convenience make targets (can start in either order):

# Terminal 1
make test_node_1

# Terminal 2
make test_node_2

sudo is required for real-time scheduling (SCHED_FIFO). Alternative:

sudo setcap cap_sys_nice+ep ./build/agent_team_test

Runtime Output

=== High-Performance UDP Configuration ===
Source:      0x7F000001:5000
Destination: 0x7F000001:6000
RX Thread:   CPU core 2, priority 80 (SCHED_FIFO)
TX Thread:   CPU core 3, priority 70 (SCHED_FIFO)
SO_RCVBUF:   2097152 bytes
SO_SNDBUF:   1048576 bytes
==========================================

[TX] Lifesign: 1, Queued: 32 bytes (TX queue: 0)
[RX] UniqueId: 0x12345678, Lifesign: 1, DataLen: 15, Interval: 100023 us

Press Ctrl+C for graceful shutdown.


Coding Conventions

This project follows safety-critical coding practices:

  • MISRA-like boolean checks: if (condition == true) / if (condition == false)
  • Always use braces: All if, else, while, for statements use {}
  • Single point return: Each function has exactly one return statement at the end
  • Prefer do-while: Loop bodies execute at least once (except event loops)
  • No continue: Loop flow is controlled via conditions, not continue statements
  • Explicit comparisons: Pointers compared with nullptr, booleans with true/false

License

MIT License - Copyright (c) 2026 Tawan Thintawornkul

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages