Skip to content

README.md

Latest commit

 

History

History
343 lines (236 loc) · 6.26 KB

README.md

File metadata and controls

343 lines (236 loc) · 6.26 KB
Raw

🛡️ SentinelVision

A high-performance real-time person and object tracking system built in Rust, using:

  • OpenCV (video processing & rendering)
  • ONNX Runtime (YOLO inference)
  • SORT algorithm (multi-object tracking)

SentinelVision detects objects in video streams, assigns persistent tracking IDs, and renders real-time annotated output with RED bounding boxes.

� Example Results

Here are some example detections from the system:

Raw Input Frame

Raw Input Frame

Sample Detection (Expected)

Sample Detection

Detection Only

Detection Overlay

Detection with Letterbox Preprocessing

Detection with Letterbox

�🚀 Features

  • 🎥 Video input support (file or webcam)
  • 🧍 Real-time person/object detection using YOLO (ONNX)
  • 🧠 HOG fallback detector (OpenCV-based)
  • 🎯 Multi-object tracking using SORT
  • 🆔 Persistent tracking IDs across frames
  • 🔴 RED bounding box visualization (strict requirement)
  • 📊 Real-time labels (ID | class | confidence)
  • 📼 Optional video output recording
  • ⚡ Optimized for real-time performance (20–30 FPS target)

📍 Status

🚧 Active development — core pipeline (OpenCV + ONNX Runtime + YOLO + SORT tracking) is being actively implemented and stabilized

🧠 System Architecture

Video Input (Webcam / File)
↓
Frame Capture (OpenCV)
↓
Preprocessing (Resize / Normalize / Letterbox)
↓
ONNX Runtime Inference (YOLO)
↓
Detection Post-processing (Confidence Filtering + NMS)
↓
SORT Tracking (Kalman Filter + IoU + Hungarian Matching)
↓
Track Management (ID lifecycle handling)
↓
Rendering (RED bounding boxes + labels)
↓
Display / Optional Video Output

🛠️ Tech Stack

Core

  • Rust 🦀
  • OpenCV (Rust bindings)
  • ONNX Runtime (ORT)

Computer Vision

  • YOLOv8 / YOLOv5 (ONNX models)
  • HOG-based fallback detector (OpenCV)

Tracking

  • SORT algorithm:

    • Kalman Filter (motion prediction)
    • IoU matching (data association)
    • Hungarian assignment

📁 Project Structure

sentinelvision/
│
├── src/
│   ├── main.rs          # Pipeline orchestration
│   ├── config.rs        # Configuration & constants
│   ├── video.rs         # Video capture & frame handling
│   ├── detector.rs      # YOLO ONNX inference + HOG fallback
│   ├── tracker.rs       # Track lifecycle manager
│   ├── sort.rs          # SORT algorithm implementation
│   ├── bbox.rs          # Geometry + IoU utilities
│   ├── renderer.rs      # Visualization (RED bounding boxes)
│   ├── utils.rs         # Helper utilities
│
├── models/
│   └── yolov8.onnx
│
├── data/
│   └── input.mp4
│
├── output/
│   └── output.mp4
│
├── Cargo.toml
└── README.md

🧠 YOLO Model Setup (YOLOv8 → ONNX)

SentinelVision uses YOLO in ONNX format via ONNX Runtime.

📥 Step 1 — Install Python + Ultralytics

Create virtual environment:

python -m venv venv
source venv/bin/activate   # Linux / macOS
# OR
venv\Scripts\activate      # Windows

Install dependencies:

pip install ultralytics

📦 Step 2 — Download YOLOv8 Model

from ultralytics import YOLO

model = YOLO("yolov8n.pt")

This automatically downloads pretrained weights.

Recommended models:

  • yolov8n.pt → fastest (recommended)
  • yolov8s.pt → balanced
  • yolov8m.pt → more accurate

🔁 Step 3 — Export to ONNX

from ultralytics import YOLO

model = YOLO("yolov8n.pt")

model.export(
    format="onnx",
    opset=12,
    simplify=True,
    dynamic=False,
    imgsz=640
)

📁 Step 4 — Move Model into Project

mv yolov8n.onnx models/yolov8.onnx

Final structure:

models/
└── yolov8.onnx

⚙️ Step 5 — Verify Input Shape

Expected ONNX input:

[1, 3, 640, 640]

If different, update preprocessing in:

src/detector.rs

▶️ Usage

Run with video file

cargo run --release -- data/input.mp4

Run with webcam

cargo run --release -- 0

🧍 Detection Format

Each detection:

[x, y, width, height, confidence, class_id]

🆔 Tracking Output Format

Each tracked object:

ID: <id> | <class> | <confidence>

🔴 Visualization Rules (STRICT)

All objects MUST be drawn using:

  • Color: RED (BGR = 0, 0, 255)
  • Thickness: 2–3 px
  • Label placed above bounding box
  • Must include tracking ID

🧪 Debug Mode (Recommended)

Enable debug outputs:

  • Detection-only view
  • Tracking-only view
  • Frame-level logging

Optional debug artifacts:

  • Saved frames with overlays
  • Detection counts per frame
  • Track counts per frame

📈 Performance Goals

  • 20–30 FPS real-time processing
  • Minimal memory allocations
  • Lightweight SORT tracking
  • Frame skipping allowed if needed

⚠️ Known Limitations

  • Occlusion may cause ID switches
  • CPU-only inference may reduce FPS
  • YOLO model quality affects accuracy
  • Lighting conditions impact detection reliability

🔮 Future Improvements

  • DeepSORT (appearance embeddings)
  • Multi-camera tracking
  • GPU acceleration (CUDA/OpenCL)
  • REST API for analytics
  • Event detection (loitering, intrusion)
  • Face recognition module

🧱 Design Philosophy

SentinelVision is built with:

  • Minimal dependencies
  • Explicit control over each pipeline stage
  • Performance-first architecture
  • No unnecessary abstractions
  • Production-oriented Rust design

📦 Dependency Policy

  • Prefer std first
  • Avoid unnecessary crates
  • No Python runtime dependency
  • No heavy ML frameworks beyond ONNX Runtime
  • Keep architecture minimal and explicit

🤝 Contributing

Rules:

  • Do NOT change architecture without discussion
  • Do NOT add unnecessary dependencies
  • Keep modules minimal and focused
  • Maintain Rust idioms and performance focus
  • Ensure changes do not break pipeline flow