Multi-drone data center digital twin — autonomous thermal anomaly detection with an OODA decision loop, built on NVIDIA Isaac Sim 5.1 + Pegasus + a Pandora-hybrid React dashboard.
Foundation codebase for HelioViews.ai. Author: Mel Torres, CMU 12-831.
Replace this image with your operator-recorded class screen capture (Chrome at :5173 showing the live OODA cycle).
Project Starlink is a high-fidelity simulated digital twin of a Stargate-class data center, with an autonomous inspection drone that detects and responds to thermal anomalies. The simulation runs on NVIDIA Isaac Sim 5.1 with a Pegasus-flown Iris quadcopter, full PhysX collision, an aisle-aware path planner, and a stochastic 12-rack thermal field. Operator visibility is a live React dashboard fed by a FastAPI SSE bridge over a wire format pre-committed to NVIDIA's Omniverse Kit App Streaming schema.
Twelve sprint days, seven of them shipping a complete simulation + dashboard stack: Isaac Sim 5.1 in Docker on WSL2, ROS 2 Jazzy bridge with OmniGraph sensor publishers, a Pegasus-subclassed OODABackend running PID + SO(3) attitude control at 50 Hz with a PATROL → INVESTIGATE → RETURN state machine, twelve PhysX-collision-boxed server racks built from measured USD bounding-box geometry, an aisle-aware WaypointRouter with fault-tolerant emergency-fallback re-planning (validated in vivo), an MPC thermal optimizer driving cooling actions during INVESTIGATE phases, and a six-panel React dashboard with rotation-aware SSE tail handling WSL2 v9fs filesystem inode reuse. Ninety unit + integration tests, deterministic replay via seeded wind RNG and world.stop() lifecycle hardening, dual-runtime validation (Docker headless CI + Windows GUI demo).
Three things distinguish Starlink from a tutorial-grade Isaac Sim project:
- Wire-format pre-commitment. Every event the simulation emits —
thermal_alert,phase_change,telemetry,agent_diagnostic— uses the{event_type, payload}shape from NVIDIA'somni.kit.livestream.messagingextension verbatim. The dashboard's transport (today: FastAPI SSE; tomorrow: WebRTC via NVIDIA'sOVAppStreamer) is decoupled from the schema. The Day 8/Stargate Kit App Streaming migration is a transport swap, not a schema change. - Deterministic replay. A single
--seed 42flag gives byte-identicalevents.jsonlacross runs (verified). The Day 6 hardening commit isolated this with a seeded global numpy RNG andworld.stop()beforeworld.reset()to tear down stale physics-step subscriptions. - Empirically validated fault-tolerance. During Windows GUI dual-runtime testing, the drone hit a transit-zone safety assertion mid-cross-aisle under non-deterministic GPU-contended timing — and the emergency-fallback re-planner recovered autonomously to a safe patrol node. This is in-vivo evidence of constraint-satisfaction recovery under realistic rendering load, not a contrived benchmark.
# WSL2 with Docker Engine + nvidia-container-toolkit
docker compose build
docker compose run starlink
# Inside the container:
bash scripts/run_tests.sh tests/ # 90 tests pass
/isaac-sim/python.sh scripts/build_datacenter_scene.py # one-time scene build
/isaac-sim/python.sh scripts/starlink_standalone.py --headless --num-steps 1000
/isaac-sim/python.sh scripts/validate_hover.py --log logs/ooda_flight_log.npzSee docs/setup-guide.md for the full Day 1 environment bring-up.
Port 8100 is not mapped in docker-compose.yml, so the FastAPI bridge runs on the WSL host (not inside the container). Chrome on Windows reaches both :5173 and :8100 via WSL localhost forwarding.
One-time setup:
python3 -m pip install --user -r palantir_integration/requirements.txt # FastAPI, uvicorn, aiofiles
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm install --lts # WSL-native Node, not Windows interopRun:
# Terminal 1 — WSL — FastAPI SSE bridge on :8100
cd ~/project/starlink
STARLINK_EVENTS_LOG=/mnt/c/starlink_win/results/events.jsonl \
python3 -m uvicorn palantir_integration.event_service:app \
--host 0.0.0.0 --port 8100
# Terminal 2 — WSL — React dashboard on :5173
cd ~/project/starlink/webapp
export NVM_DIR="$HOME/.nvm" && . "$NVM_DIR/nvm.sh"
npm install # first run only
npm run dev# Terminal 3 — Windows PowerShell — Isaac Sim launcher (deterministic replay)
cd C:\isaacsim
.\python.bat C:\starlink_win\scripts\starlink_windows.py --num-steps 5000 --seed 42Open Chrome to http://localhost:5173. Start order does not matter — the SSE bridge is rotation-aware (inode + size-shrink detection works on both ext4 and WSL /mnt/c/ v9fs).
Two-layer architecture with a deliberately frozen wire-format contract between them. See docs/architecture.md for the full component map (Layer 1 = today's Pandora-hybrid; Layer 2 = Day 8 Kit App Streaming target). OODA decision logic operates at three timescales:
- PID Flight Controller (50 Hz) — position tracking + wind-disturbance rejection
- MPC Thermal Optimizer (on-event) — per-rack cooling optimization during INVESTIGATE
- OODA State Machine (1 Hz) — PATROL → INVESTIGATE → RETURN with fault-tolerant fallback
| Day | Milestone | Status |
|---|---|---|
| 0 | Project scaffolding | ✅ |
| 1 | Isaac Sim 5.1 + Pegasus v5.1 + ROS 2 Jazzy environment validated | ✅ |
| 2 | ROS 2 Jazzy bridge + OmniGraph sensor pipeline verified | ✅ |
| 3 | OODABackend — drone flies under PID + SO(3) attitude control | ✅ |
| 4 | Data center scene + AC wind disturbance + thermal visualization + PUE | ✅ |
| 5 | Full stochastic thermal model + MPC optimizer + dual-runtime validation | ✅ |
| 6 | Visual validation + multi-aisle PhysX (bbox-measured geometry, aisle-aware routing, fault-tolerant fallback) | ✅ |
| 7 | Pandora-hybrid React dashboard + FastAPI SSE bridge + wire-format pre-commit | ✅ |
| 8–12 | Kit App Streaming migration + paper production + final demo | ⏭️ Deferred to Project Stargate |
The Day 8–12 work — Kit App Streaming with OVAppStreamer, bidirectional WebRTC data channel, optional LLM chat panel, ACM sigconf paper, final demo recording — is captured for a follow-on sibling project. See docs/stargate-handoff.md for the coupling contract and kickoff prompt. Starlink itself ships at Day 7.
starlink/
├── CLAUDE.md # Living governing doc
├── PROJECT_STARLINK_COMPLETE_V5_1.md # Authoritative master plan
├── README.md # You are here
├── Dockerfile, docker-compose.yml # Isaac Sim 5.1 NGC container + GPU passthrough
├── thermalsight_ooda/ # OODA stack — backend, PID, MPC, thermal field, router, event emitter
├── thermalsight_agent/ # Agent scaffold (Future Work — moved to Stargate)
├── palantir_integration/ # FastAPI SSE bridge + (Day 12 stretch) Foundry client
├── webapp/ # React 19 + TS + Vite Pandora-hybrid dashboard
│ └── src/{App.tsx, hooks/, panels/, theme/, types/, util/}
├── scripts/ # Standalone + Windows launchers, scene build, validation
├── tests/ # 90 tests — pure-Python OODA logic, no Isaac Sim required
├── scenes/, config/, assets/, deps/ # USD scenes, layout/scenarios, NVIDIA asset pack, Pegasus submodule
└── docs/ # Architecture, day specs, retrospectives, Stargate handoff
Mel Torres — CMU 12-831 graduate, founder of HelioViews.ai.
Built on the shoulders of:
- NVIDIA Isaac Sim 5.1 + PhysX 5.4 + Omniverse Kit 107.3 — physics, rendering, sensor sim.
- Pegasus Simulator v5.1.0 — open-source quadcopter dynamics + flight backend ABC, used as the OODABackend parent class.
- NVIDIA Data Center Asset Pack — 33 USD models for realistic rack geometry.
- NVIDIA GTC 2026 lab session "Develop AI-Powered Operational Dashboards for Digital Twin Applications" (Ashley Goldstein + Justine Lin) — reference architecture for the Pandora-hybrid wire format and the Day 8 Kit App Streaming migration target.
- NVIDIA NIM API — Nemotron Nano 30B-A3B endpoint, scoped for Stargate (Future Work).
- CMU 12-831 Decision-Making Algorithms for Autonomous Systems — coursework framework that motivated the four-algorithm OODA stack (MPC, aisle-aware path planning, rule-based state machine, fault-tolerant re-planner).