Drone Hand Control

Control a real DJI Tello drone using hand gestures, computer vision, and machine learning.

Demo

YouTube video

Installation

1. Clone the repository

git clone https://github.com/nojusbarth/drone-hand-control.git
cd drone-hand-control

2. Create a virtual environment

Windows

python -m venv .venv

Linux / macOS

python3 -m venv .venv

3. Activate the virtual environment

Windows PowerShell

.\.venv\Scripts\Activate.ps1

Linux / macOS

source .venv/bin/activate

4. Install dependencies

pip install -r requirements.txt

5. Start the application

python src/__init__.py

---

Running Options

Simulation Mode

Starts the hand tracking pipeline and the simulation environment.

python src/__init__.py

DJI Tello Mode

Starts the hand tracking pipeline and connects to a DJI Tello drone.

python src/__init__.py --mode tello

The application will automatically try to connect to the drone. If no connection can be established after three attempts, the program exits.

---

Tello Controls

Key	Action
S	Take off
E	Land
Q	Exit program and land

Important: Make sure the gesture detection window has focus when pressing keys.

---

Gesture Controls

Gesture	Action
👍 Thumb Up	Move Up
👎 Thumb Down	Move Down
👈 Thumb Left	Move Left
👉 Thumb Right	Move Right
✊ Fist	Stop
✋ Palm Forward	Move Backward
🤚 Palm Back	Move Forward

---

Custom Gestures

You can train your own gesture classifier:

models/CGCNN_script.py

To add new gestures, you will need your own labeled dataset. Datasets are not included in this repository.

---

Architecture

Video Capture
    ↓
MediaPipe Hands
    ↓
Landmark Extraction
    ↓
Gesture Classifier
    ↓
Temporal Filter
    ↓
Drone Commands
    ↓
DJI Tello

---

Troubleshooting

Unable to connect to the DJI Tello

1. Turn on the drone.
2. Connect your computer to the Tello Wi-Fi network.
3. Verify that the connection is active.
4. Start the application again.

If the connection still fails, restart both the drone and the application.