Skip to content
GITLUVVPublic

About

No description, website, or topics provided.

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

SANKET (संकेत) — AI-Assisted Indian Sign Language Accessibility System

License Phase Python CI Security: Protected

SANKET (संकेत) is an open-source, low-latency, privacy-preserving AI accessibility system designed to recognize Indian Sign Language (ISL) from camera video input and translate recognized signs into accessible text and speech on commodity edge hardware.


Current Project Status

  • Active Phase: Phase 2 — Real-Time Sanket MVP
  • Active Milestone: Milestone 2 — Real-Time Sanket MVP (Issues #51–#60)
  • Current Objective: Convert the validated Phase 1 offline recognition pipeline into a usable, low-latency real-time camera prototype operating at $\ge 20$ FPS with end-to-end latency $< 100\text{ ms}$.
  • Phase 0 Status: Completed and Verified (Foundational scaffolding, security policies, dataset licensing audits, initial 16-sign vocabulary, and signer-independent protocols).
  • Phase 1 Status: Completed and Certified (14 of 14 tasks merged):
    • Dataset ingestion with SHA-256 integrity verification (sanket.data.ingestion).
    • Dataset inspection and label balance profiling (sanket.data.inspection).
    • Signer-independent train/validation/test splitting (sanket.data.splits).
    • Uniform temporal subsampling and letterbox video preprocessing (sanket.vision.preprocessing).
    • MediaPipe 3D hand landmark extraction (sanket.vision.landmarks).
    • Upper-body pose reference anchor investigation (sanket.vision.pose).
    • Wrist-centered and palm-scale coordinate normalization (sanket.vision.normalization).
    • 645-dimensional geometric and temporal feature vector extraction (sanket.features).
    • Random Forest baseline implementation (93.3% Top-1 Accuracy, 0.73 ms p95 latency).
    • SVM baseline with RBF kernel and standard scaling (sanket.models.svm).
    • MLP neural network baseline with Adam optimization (sanket.models.mlp).
    • Unified evaluation pipeline and JSON manifest generation (sanket.evaluation.evaluator).
    • Confusion matrix and phonological error diagnostics (sanket.evaluation.diagnostics).
    • Cross-baseline comparative study and model selection (ADR-0003, EXP-P1-001).

System Architecture

SANKET follows a modular, feed-forward pipeline separating vision acquisition from feature extraction, inference, and accessible presentation:

flowchart LR
    A["Camera Input\n(OpenCV VideoCapture)"] --> B["Preprocessing\n(Letterbox & Mirror)"]
    B --> C["Landmark Extraction\n(MediaPipe Hands & Pose)"]
    C --> D["Coordinate Normalization\n(Wrist Origin & Palm Scale)"]
    D --> E["Feature Extraction\n(645-dim Geometry & Moments)"]
    E --> F["ML Inference\n(Random Forest Classifier)"]
    F --> G["Temporal Smoothing\n(EMA & Hysteresis Gating)"]
    G --> H["Accessible UI\n(HUD & Text Subtitles)"]
Loading

Authoritative Documentation Hierarchy

Core Documentation

Technical Documentation (docs/technical/)

Operational Documentation (docs/operational/)

Research & Architecture Decisions (docs/)


Quickstart

# 1. Clone repository
git clone https://github.com/GITLUVV/Sanket.git
cd Sanket

# 2. Set up virtual environment
python -m venv venv
# Windows: .\venv\Scripts\Activate.ps1 | Linux: source venv/bin/activate

# 3. Install in editable mode with development tooling
pip install --upgrade pip
pip install -e ".[dev]"

# 4. Verify environment setup and test suite
python scripts/verify_setup.py
pytest -q
ruff check .

Research Ethics & Privacy

Important

SANKET strictly enforces a Zero-Biometric-Retention Mandate. Live camera streams reside exclusively in volatile memory for landmark localization and are evicted immediately without touching persistent storage. No raw facial or bodily recordings of identifiable individuals are ever committed to this repository.


License

This project is licensed under the Apache License 2.0. See the LICENSE file for details. Third-party datasets retain their respective original licenses as recorded in the Dataset Register.

About

No description, website, or topics provided.

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages