Single-pass UAV video to georeferenced 3D reconstruction
Demo Video · 3D GIS Mapping · DRIFTX Platform · Unified DRIFTX · Mounted Sensor · Ground Sensor · 3D Evidence · Three-video Results
One flight + limited views → bounded processing → inspectable 3D geometry.
DRIFTX converts a continuous UAV video into a temporally ordered, quality-filtered frame stream; estimates depth and camera geometry with a frozen Depth Anything 3 (DA3) backbone; aligns overlapping local reconstructions with confidence-weighted Sim(3) transforms; and exports a global scene for inspection.
The project targets the Smart India Hackathon 2026 Problem Statement 26158: “Single-Pass Drone Video to Accurate 3D Model Generation System” under Robotics & Drones for NTRO.
Accuracy note: the repository contains measured runtime, memory, artifact, and reconstruction diagnostics. The proposal target of ≤1 m spatial accuracy, <15 minutes for a 10-minute video, and full visible-scene coverage requires independent surveyed checkpoints/reference geometry; it is not a universal guarantee.
- Problem and solution
- System pipeline
- Repository structure
- Offline/local setup
- Run a reconstruction
- Run the local viewer and API
- Benchmark profiles and metrics
- Output contract
- Technical implementation
- Multimodal vibration subsystem
- Applications and limitations
- Team
Conventional aerial mapping often depends on repeated passes, high image overlap, large image collections, and long post-processing cycles. That is a poor fit for disaster zones, unsafe infrastructure, inaccessible terrain, and time-critical inspections where only one flight may be available.
DRIFTX addresses that constraint by:
- accepting one continuous moving-UAV video as the primary observation stream;
- preserving source-frame identity and timestamps through sampling and export;
- filtering blur, near-duplicates, low-confidence regions, and invalid geometry;
- processing bounded overlapping frame windows instead of loading the entire video into GPU memory;
- aligning adjacent local reconstructions using shared frames and confidence-weighted 3D correspondences;
- exporting GLB, PLY, NPZ, camera metadata, depth/confidence visualizations, metrics, and a machine-readable run report;
- optionally correlating spatial reconstruction with synchronized vibration-condition data.
UAV video + optional GNSS/RTK/PPK, IMU, EXIF/XMP, calibration
│
▼
CPU decode → deterministic sampling → blur/dedup/quality checks
│
▼
Depth Anything 3 → per-frame depth, rays, confidence, camera geometry
│
▼
Bounded overlapping windows → shared-frame correspondences
│
▼
Confidence-weighted Sim(3) alignment → global scene merge
│
▼
PLY / GLB / NPZ / depth views / metadata / metrics / run report
│
▼
Local API + WebGL inspection viewer
- Mode A — bounded streaming baseline: deterministic sampling, bounded DA3 windows, overlap alignment, CPU staging, and structured export. This is the reproducible local smoke path.
- Mode B — survey workflow: extends the baseline with telemetry, camera calibration, control points, checkpoints, scale recovery, and georeferencing validation.
- Optional Gaussian branch: available only with a DA3 checkpoint that exposes a compatible Gaussian head and a separately compiled
gsplatinstallation. The default DA3 Large 1.1 baseline is depth/pose-capable, not Gaussian-capable.
| Path | Responsibility |
|---|---|
driftx/ |
Product CLI, orchestration, streaming benchmark, reconstruction, validation, export, API, and doctor commands |
frozen_ml/3d/ |
Frozen model/runtime implementation retained for compatibility |
src/depth_anything_3/ |
Install-time compatibility import path for the DA3 package |
da3_streaming/ |
Sliding-window video inference utilities |
frontend/ |
Vite + Three.js WebGL inspection interface |
scripts/ |
Benchmark and figure-generation helpers |
input/ |
Tracked sample images and videos |
outputs/ |
Tracked example GLB scenes and depth visualizations |
output_figures/ |
Benchmark dashboards, frame pipeline, confidence, temporal, trajectory, and scaling figures |
docs/ |
CLI, API, benchmark, streaming design, deployment, and evidence documentation |
tests/ |
CPU-safe unit and integration-contract tests |
cloud/ |
Optional notebook workflows for larger/Gaussian experiments |
The following procedure runs the repository locally. The only step that needs network access is the initial installation and model-weight download. After the dependencies and checkpoint are present, inference, export, API serving, and frontend inspection run on the local machine.
- Python 3.9–3.13; Python 3.11 is recommended.
- Git.
- Node.js and npm for the WebGL frontend.
- NVIDIA GPU with a compatible CUDA-enabled PyTorch wheel for practical DA3 inference. CPU is useful for tests, CLI inspection, and API/viewer work but is not a practical path for full reconstruction.
- Sufficient disk space for PyTorch, DA3 weights, extracted frames, and generated GLB/PLY artifacts.
git clone https://github.com/hitakshijoshi20072911/drift-3d.git
cd drift-3d
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pipgit clone https://github.com/hitakshijoshi20072911/drift-3d.git
cd drift-3d
py -3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pipInstall a CUDA-enabled PyTorch wheel appropriate for the installed NVIDIA driver. The package name is torch, not pytorch.
# Example CUDA wheel; use the wheel matching the target machine.
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
python -m pip install -e ".[figures]"For a CPU-only environment used for tests and API development:
python -m pip install -e ".[figures,test]"Verify the install:
python -c "import torch; print('Torch:', torch.__version__); print('CUDA available:', torch.cuda.is_available()); print('GPU:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'none')"
python -m driftx --help
python -m driftx doctor --jsonKeep model weights outside Git. Download into a regular local directory so the same path can be reused offline:
python -c "from huggingface_hub import snapshot_download; snapshot_download('depth-anything/DA3-LARGE-1.1', local_dir='models/DA3_LARGE_1.1_SAFE')"Set the model path for the current shell:
export MODEL_DIR="$PWD/models/DA3_LARGE_1.1_SAFE"PowerShell:
$env:MODEL_DIR = "$PWD\models\DA3_LARGE_1.1_SAFE"python -m unittest discover -s tests -v
python -m compileall -q driftx src frozen_ml app_live.py da3_streaming
python -m driftx benchmark --helpThese checks cover profile defaults, window coverage, overlap alignment, OOM fallback sizing, artifact contracts, mocked benchmark paths, and API contracts. They do not validate GPU quality or geographic accuracy.
mkdir -p workspace/gallery
python -m depth_anything_3.cli auto input/SOH \
--model-dir "$MODEL_DIR" \
--export-format glb \
--export-dir workspace/gallery/DRIFTX_SOHThe compatibility console command is also available after installation:
da3 auto input/SOH --model-dir "$MODEL_DIR" --export-format glb --export-dir workspace/gallery/DRIFTX_SOHThe smoke profile is the reproducible baseline:
- sample rate: 1 FPS;
- total sampled-frame cap: 16;
- processing resolution: 504;
- window size: 8 frames;
- overlap: 2 frames;
- three inference windows for the 16-frame workload;
- automatic conservative VRAM sizing and CUDA OOM retries.
python -m driftx benchmark \
--video input/test3.mp4 \
--output outputs/benchmarks/test3_smoke \
--model "$MODEL_DIR" \
--device cuda \
--profile smokeUse --device auto to let the runtime choose an available device. Use --device cpu only for a functional smoke test; full DA3 reconstruction is GPU-oriented.
python -m driftx benchmark \
--video input/test3.mp4 \
--output outputs/benchmarks/test3_quality \
--model "$MODEL_DIR" \
--device cuda \
--profile quality \
--max-frames 0 \
--precision autoAvailable profiles:
| Profile | Sampling | Frame cap | Resolution | Window / overlap | Use |
|---|---|---|---|---|---|
smoke |
1 FPS | 16 | 504 | 8 / 2 | Reproducible first run |
balanced |
2 FPS | All sampled | 630 | 16 / 4 | More coverage with bounded memory |
quality |
2 FPS | All sampled | 756 | 12 / 4 | Higher-detail reconstruction |
For low-memory GPUs, start with --chunk-size 8 --chunk-overlap 2, reduce --process-res, or lower --max-frames. Run jobs sequentially so GPU memory is not shared between processes.
python -m depth_anything_3.cli video input/test.mp4 \
--fps 15 \
--model-dir "$MODEL_DIR" \
--export-dir workspace/gallery/DRIFTX_VIDEO \
--export-format glbThe repository includes a local FastAPI backend and a Vite/Three.js inspection interface. Precomputed demo scenes can be viewed without running new inference.
python -m driftx doctor --json
python -m driftx server --host 127.0.0.1 --port 8123Health check:
curl http://127.0.0.1:8123/api/health
curl http://127.0.0.1:8123/api/demoscd frontend
npm install
npm run devOpen the Vite URL printed in the terminal. The viewer supports:
- precomputed demo selection;
- GLB scene loading;
- RGB and depth evidence inspection;
- point, distance, height, area, volume, profile, sight-line, and note tools;
- screenshot and report export;
- raw video upload and asynchronous processing when the backend has model access.
GET /api/healthGET /api/demosGET /api/demos/{demo_id}GET /api/demos/{demo_id}/artifacts/{filename}POST /api/runs/uploadPOST /api/runs/{run_id}/processGET /api/runs/{run_id}/statusGET /api/runs/{run_id}/manifestGET /api/runs/{run_id}/artifacts/{filename}
Useful environment variables:
DRIFTX_MODEL— default checkpoint for uploaded processing;DRIFTX_RUN_ROOT— persistent run/artifact directory;DRIFTX_API_URL— frontend API base URL when the backend is separate;DRIFTX_ALLOWED_ORIGINS— comma-separated CORS origins;DRIFTX_HOST,DRIFTX_PORT— backend bind settings.
python app_live.py outputs/DA3_LARGE_1.1_SAFE/scene.glbThe command reports the loaded path, geometry-object count, bounding-box dimensions, and diagonal before opening the local trimesh viewer.
The repository’s measured CUDA/BF16 smoke runs use DA3 Large 1.1, 16 processed frames per video, chunk size 8, overlap 2, and three inference windows. Source videos are approximately 30 FPS and contain 991–1,706 frames.
| Run | Video duration | Source frames | Wall time | Inference | Export | Throughput | Peak GPU |
|---|---|---|---|---|---|---|---|
test3 |
56.87 s | 1,706 | 43.45 s | 6.02 s | 2.43 s | 0.368 frames/s | 3,879.18 MB |
test6 |
44.44 s | 1,332 | 39.04 s | 4.89 s | 2.55 s | 0.410 frames/s | 3,879.18 MB |
test7 |
33.07 s | 991 | 35.38 s | 4.63 s | 2.46 s | 0.452 frames/s | 3,879.18 MB |
| Mean | 44.79 s | — | 39.29 s | 5.18 s | 2.48 s | 0.410 frames/s | 3,879.18 MB |
Additional measured facts:
- 48/48 selected frames processed successfully;
- 0 CUDA OOM retries in the reported smoke runs;
- 100% finite-depth tensor coverage in the processed tensors;
- output sizes were approximately 16.0 MB GLB and 16.0 MB PLY per run;
- finite-depth coverage is a tensor-validity measure, not geographic scene completeness;
- GPU memory, accuracy, and runtime vary with checkpoint, resolution, hardware, and input motion.
The supplied project report also records a separate three-video evaluation over 537 total frames:
| Metric | Mean | Range |
|---|---|---|
| Reprojection error | 0.543 px | 0.532–0.554 px |
| DSM/surface-model coverage | 85.8% | 80.6–91.9% |
| GPS alignment RMSE | 1.92 m | 1.59–2.25 m |
| Frames registered | 90.4% | 71.2–100% |
These figures belong to the supplied evaluation record and should be reproduced from the corresponding run reports before being treated as a new benchmark result.
The following require independent ground truth and are not established by a visually coherent GLB alone:
- absolute spatial accuracy and the ≤1 m target;
- surveyed georeferencing error;
- camera ATE/RPE against a reference trajectory;
- depth AbsRel/RMSE/δ1 against ground-truth depth;
- point-cloud/mesh Chamfer, RMSE, and completeness;
- full visible-scene coverage;
- dynamic-mask precision/recall/F1/IoU.
A successful benchmark writes a self-contained run directory similar to:
outputs/benchmarks/<run-name>/
├── input_images/ # sampled frames
├── scene.glb # inspection-ready glTF scene
├── scene.ply # point-cloud export
├── results.npz # root geometry arrays
├── exports/mini_npz/results.npz # compact array export
├── depth_vis/ # depth/confidence visualizations
├── frames.json # source-frame and timestamp metadata
├── camera_poses.json # estimated camera metadata
├── metrics.json # measured run metrics
└── run_report.json # status, timings, memory, errors, artifacts
The runner does not silently publish an incomplete scene. If inference or export fails, inspect run_report.json, especially status, error, and error_traceback. Preserve the report and artifacts for reproducibility.
- CPU-side video decoding keeps GPU memory focused on geometry inference.
- Sampling retains source-frame number and timestamp.
- Blur rejection can use Laplacian variance.
- Near-duplicate filtering can use perceptual hash/SSIM-style comparisons.
- Optional dynamic masking and confidence thresholds remove unreliable pixels.
- Camera intrinsics, distortion, GNSS/RTK/PPK, IMU, EXIF/XMP, and telemetry are supported as the survey workflow evolves.
- The default baseline uses Depth Anything 3 Large 1.1.
- The backbone is treated as frozen for the stable inference path; the project documentation describes selected-layer aerial adaptation using IGDTUW UAV frames.
- DA3 supplies depth and per-pixel ray geometry suitable for direct 3D back-projection:
P = o + D · d
where o is ray origin, D is predicted depth, and d is ray direction.
- selected frames are split into bounded overlapping windows;
- each window is reconstructed in its own local coordinate frame;
- shared frames generate confidence-weighted 3D correspondences;
- robust residual trimming rejects high-error matches;
- a similarity transform
Sim(3)maps the next local chunk into the global frame; - chunks merge in source-frame order;
- insufficient or degenerate overlap causes an explicit failure rather than an incoherent export;
- CUDA OOM recovery retries with smaller windows down to a safe floor.
- deterministic point-cap logic limits export size;
- CPU/memory-mapped staging avoids retaining all live inference tensors on the GPU;
- artifact existence and report paths are checked after the run;
- metrics are recorded as measured values or
"not measured"; no GPU result or quality score is fabricated.
The companion condition-monitoring prototype is separate from the core visual reconstruction path. It uses an ESP32 and MPU-class three-axis IMU to measure vibration severity, compare it with a calibrated baseline, and emit persistent events.
- ESP32 Dev Module: sampling, calibration, event logic, communication;
- GY-521 / MPU-class IMU: three-axis acceleration;
- Prototype wiring:
VCC → 3V3,GND → GND,SDA → GPIO21,SCL → GPIO22; - Mechanical mount: rigid, repeatable coupling to the monitored asset;
- Receiver/dashboard: serial records and timestamped monitoring storage.
| Parameter | Prototype value |
|---|---|
| Sampling rate | 200 Hz |
| Analysis window | 200 samples / 1 second |
| Static calibration | 600 samples |
| Baseline RMS calibration | 1,000 samples |
| Event trigger | RMS ≥ 0.100 g for 3 consecutive windows |
| Event clear | RMS < 0.090 g for 3 consecutive windows |
Prototype bench observations:
- quiet baseline: ≈0.0047–0.0050 g RMS;
- light disturbance: ≈0.0384 g RMS, peak ≈0.2972 g;
- strong disturbance: ≈0.3558 g RMS, peak ≈0.6166 g;
- sustained high vibration: ≈0.18–0.23 g RMS, peak ≈1.06–1.45 g;
- later low-vibration plateau: ≈0.0205–0.0211 g RMS.
These are prototype observations, not field-calibrated railway safety limits. The intended evidence chain is:
spatial observation → vibration measurement → timestamp correlation → operator review → validation
- disaster assessment: landslides, floods, debris, and post-earthquake inspection;
- infrastructure inspection: roads, bridges, buildings, and asset-condition monitoring;
- urban planning and digital-twin inputs;
- terrain and archaeological documentation;
- aerial inspection where a single safe flight is preferable to repeated passes;
- future multimodal threat/context analysis using thermal, night-vision, LiDAR, GNSS, IMU, and vibration data.
- absolute georeferencing requires synchronized telemetry, calibration, scale recovery, and independent checkpoints;
- blur, low texture, occlusion, lighting changes, dynamic objects, and weak motion can reduce reconstruction quality;
- the default tracked demos are precomputed local reconstructions, not live inference results;
- sensor fusion and domain fine-tuning are project capabilities/roadmap items unless the corresponding local implementation and run artifacts are present;
- Gaussian export requires a compatible checkpoint and optional native
gsplatbuild; - reported runtime is hardware-specific and should not be generalized to a 10-minute video without a new run.
ZeroError — Smart India Hackathon 2026
- Hitakshi Joshi
- Ridhima Kulashri
- Mannu
- Renaissance Das
- Saranya Jogi
- Risha Rastogi
Institution: Indira Gandhi Delhi Technical University for Women (IGDTUW)
The repository is released under the Apache-2.0 license. See LICENSE and THIRD_PARTY_LICENSES.md for the local license files shipped with the project.

