Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 6 additions & 25 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,38 +1,19 @@
name: CI

# Temporarily disabled: repo structure is mid-consolidation (task registration,
# vis.py rewrite, test-data relocation) and CI would fail on pre-existing
# issues unrelated to any given PR. Re-enable by restoring the push/pull_request
# triggers below. Still runnable manually via workflow_dispatch in the meantime.
# Lint (ruff/black) isn't gated here yet - the repo carries a lot of
# pre-existing style debt outside what this consolidation pass touched, and
# fixing it repo-wide is a separate task from getting tests running again.
# pytest-only for now.
on:
push:
pull_request:
workflow_dispatch:

# CI installs only the classical dependency group. Torch/YOLO/XFeat testing
# will land as a separate, non-blocking job once dependency weight and model
# availability are settled.

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install dependencies
run: |
pip install -r requirements-classical.txt
pip install ruff black

- name: Lint (ruff)
run: ruff check .

- name: Format check (black)
run: black --check .

test-classical:
runs-on: ubuntu-latest
steps:
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
*.MP4
*.mp4
*.avi
*.png
*.jpg

# Byte-compiled / optimized / DLL files
__pycache__/
Expand All @@ -23,3 +25,6 @@ build/
.vscode/

data/

# uv-managed virtual environment
.venv/
98 changes: 52 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,38 +4,42 @@ Code Quality [![CodeFactor](https://www.codefactor.io/repository/github/berkeley

## Installation

We will use Conda for managing environments. We recommend installing Miniconda for Python 3.11 [here](https://docs.conda.io/en/latest/miniconda.html).
Python 3.11 is required to support YOLO models.
Then create an environment with
We use [uv](https://docs.astral.sh/uv/) for managing both the Python version and the virtual environment. Install it once per machine:

conda create -n urobotics python=3.11
curl -LsSf https://astral.sh/uv/install.sh | sh

activate it with
(or `brew install uv` on macOS). Full install docs [here](https://docs.astral.sh/uv/getting-started/installation/).

conda activate urobotics

Then clone the repo in a directory of your choice
Clone the repo in a directory of your choice

git clone https://github.com/berkeleyauv/perception.git
cd perception

Create the virtual environment — this downloads and pins Python 3.11 (required to support YOLO models) if you don't already have it, and creates `.venv/` in the repo:

uv venv --python 3.11

Activate it

Change into the cloned repo directory and install it
source .venv/bin/activate

pip3 install -e ./
Install the package in editable mode

uv pip install -e .

Install dependencies with

pip3 install -r requirements-classical.txt
uv pip install -r requirements-classical.txt

If you also need YOLO/torch-based models, install the torch group instead (it includes everything in `requirements-classical.txt`, plus `torch`, `torchvision`, and `ultralytics`)

pip3 install -r requirements-torch.txt
uv pip install -r requirements-torch.txt


Also, our training data is stored here https://www.dropbox.com/sh/rrbfqfutrmifrxs/AAAfXxlcCtWZmUELp4wXyTIxa?dl=0 so download it and unzip it in the same folder as `perception`.

### Cython
To compile cythonized code, run the following commands after `cd`ing into the folder with Cython `setup.py`
To compile cythonized code, run the following commands (with the venv activated) after `cd`ing into the folder with Cython `setup.py`

python setup.py build_ext --inplace
cythonize file_to_cythonize.pyx
Expand All @@ -45,46 +49,48 @@ To compile cythonized code, run the following commands after `cd`ing into the fo
Misc code, camera calibration etc.

## tasks:
Code for specific tasks like
Code for specific competition tasks, one folder per task:

1. cross: cross detection
2. segmentation:
3. path_marker: path_marker detection
4. spinny_wheel_detection

etc
1. `gate`: qualification gate detection (`classical/`) and orientation estimation (`orientation/`)
2. `slalom`: slalom pipe-set detection (`classical/`) and pipe-set sequence tracking (`slalom_sequence_tracker.py`)
3. `path_marker`: path marker detection
4. `buoy`, `torpedo`, `octagon`: scaffolded, no detection logic yet
5. `_archive`: retired tasks (cross, dice, roulette, slots) kept for reference, excluded from registry discovery

In order to create your own algorithm to test:

1. Create <your_algo>.py and put it in one of the specific task folders in perception/tasks.
1. Create `<your_algo>.py` and put it in the relevant task folder under `perception/tasks/` (e.g. a new classical approach goes in that task's `classical/`).

2. Create a class which extends `TaskPerceiver` (see `perception/tasks/TaskPerceiver.py` for the template with documentation) and decorate it with `@register_perceiver(task=..., algo=...)` from `perception/tasks/registry.py`. This is what makes it discoverable by `vis.py` — see the **vis** section below.

2. Create a class which extends the TaskPerceiver class. perception/tasks/TaskPerceiver.py includes a template with documentation for how to do this.
3. Pass `default=True` to `@register_perceiver` if this should be the algo `vis.py` runs for its task when `--algo` is omitted. A task with only one registered algo defaults to it automatically; a task with several needs one of them explicitly marked (currently `segmentation_a` for `gate`, and the sole algo for `slalom`/`path_marker`). Registering a second `default=True` algo for the same task raises an error.

## vis:
Visualization tools
Code for testing tasks (Ideally this should be placed a separate folder called `tests`).

After writing the code for your specific task algorithm, you can do one of two things:

1. Add this to the end of <your algorithm>.py file:

if __name__ == '__main__':
from perception.vis.vis import run
run(<list of file/directory names>, <new instance of your class>, <save your video?>)
and then run

python <your algorithm>.py
2. Add this to the perception/__init__.py file:

import <path to your module>

ALGOS = {
'custom_name': <your module>.<your class reference>
}
and then run

python perception/vis/vis.py --algorithm custom_name [--data <path to file/directory>] [--profile <function name>] [--save_video]
The **algorithm** parameter is required. If **data** isn't specified, it'll default to your webcam. If **profile** isn't specified, it will be off by default. Add the **save_video** tag if you want to save your vis test as an mp4 file.
Visualization tools for interactively running and debugging task algorithms.

Every algorithm is a `TaskPerceiver` subclass (see `perception/tasks/TaskPerceiver.py`) decorated with `@register_perceiver(task=..., algo=...)`. Decorating a class is all that's needed to make it discoverable — there's no shared file to hand-edit:

from perception.tasks.registry import register_perceiver
from perception.tasks.TaskPerceiver import TaskPerceiver

@register_perceiver(task="gate", algo="my_algo")
class MyAlgo(TaskPerceiver):
...

Then run it with:

python -m perception.vis.vis --task gate [--algo my_algo] [--data <path to file/directory>] [--profile <function name>] [--save_video] [--resize <scale>] [--compare <algo>] [--hide_labels]

- `--task` is required and selects which task's perceivers to run.
- `--algo` is optional — omit it to use the task's default algo (see point 3 above); `vis.py` prints which algo it picked. If the task has no default set, it errors and asks you to pass `--algo` explicitly.
- `--data` defaults to your webcam; point it at an image, video, or a directory of either.
- `--profile` is off by default; pass a `cProfile` stats key (or omit for `'all'`) to profile the run.
- `--save_video` writes the debug-frame grid to `vis_rec.mp4`.
- `--resize` scales every frame before display (default `1.0`, no resize).
- `--compare <algo>` runs a second algo for the same `--task` on the same frames and stacks it below the primary algo's grid in one "Debug Frames" window, each half labeled with its algo name in the top-left corner — useful for A/B'ing two algorithms (e.g. `center` vs. `segmentation_a` for `gate`) against the same footage. Stacking below (rather than beside) keeps each pane's width unchanged, so sub-frame resolution and label/slider text stay legible regardless of how many debug frames either algo returns. Sliders for both algos appear in the same window, prefixed with their algo name (e.g. `center: canny_low`) to keep them distinguishable. `--save_video` saves the combined, labeled view.
- `--hide_labels` drops the corner labels in `--compare` mode (shown by default).

While a window is focused: `q`/`Esc` quits, `p` pauses, `i`/`o` slow down/speed up frame playback.

## wiki:
Flowchart on TaskPerceiver, TaskReceiver, AlgorithmRunner.
5 changes: 2 additions & 3 deletions perception/tasks/gate/classical/GateSegmentationAlgoA.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,14 @@
from perception.tasks.TaskPerceiver import TaskPerceiver


@register_perceiver(task="gate", algo="segmentation_a")
@register_perceiver(task="gate", algo="segmentation_a", default=True)
class GateSegmentationAlgoA(TaskPerceiver):
center_x_locs, center_y_locs = [], []

def __init__(self):
super().__init__()
self.combined_filter = init_combined_filter()

# TODO: fix return typing

def analyze(self, frame: np.ndarray, debug: bool, slider_vals=None) -> Tuple[float, float]:
"""Takes in the background removed image and returns the center between
the two gate posts.
Expand Down
33 changes: 31 additions & 2 deletions perception/tasks/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,15 @@
from perception.tasks.TaskPerceiver import TaskPerceiver

_REGISTRY: dict[tuple[str, str], type[TaskPerceiver]] = {}
_DEFAULTS: dict[str, str] = {}


def register_perceiver(task: str, algo: str):
"""Class decorator: register a TaskPerceiver subclass under (task, algo)."""
def register_perceiver(task: str, algo: str, default: bool = False):
"""Class decorator: register a TaskPerceiver subclass under (task, algo).

Pass default=True to make this the algo get_default_algo() returns for
`task` when a caller (e.g. the vis CLI) doesn't specify one explicitly.
"""

def decorator(cls: type[TaskPerceiver]) -> type[TaskPerceiver]:
key = (task, algo)
Expand All @@ -37,6 +42,14 @@ def decorator(cls: type[TaskPerceiver]) -> type[TaskPerceiver]:
f"register {cls.__module__}.{cls.__qualname__}"
)
_REGISTRY[key] = cls
if default:
existing_default = _DEFAULTS.get(task)
if existing_default is not None and existing_default != algo:
raise ValueError(
f"task={task!r} already has default algo {existing_default!r}, "
f"cannot also mark {algo!r} as default"
)
_DEFAULTS[task] = algo
return cls

return decorator
Expand All @@ -62,6 +75,22 @@ def list_algos(task: str) -> list[str]:
return sorted(algo for t, algo in _REGISTRY if t == task)


def get_default_algo(task: str) -> str:
"""The default algo for a task: whichever was marked default=True, or the
sole registered algo if the task only has one. Raises KeyError if neither
applies, i.e. the caller must specify an algo explicitly.
"""
explicit = _DEFAULTS.get(task)
if explicit is not None:
return explicit
algos = list_algos(task)
if len(algos) == 1:
return algos[0]
raise KeyError(
f"no default algo for task={task!r} (algos: {', '.join(algos) or 'none registered'})"
)


_EXCLUDED_PACKAGES = ("perception.tasks._archive",)


Expand Down
19 changes: 14 additions & 5 deletions perception/vis/Visualizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,23 @@ def nothing(x):
pass

class Visualizer:
def __init__(self, kwargs: Dict[str, Tuple[Tuple[int, int], int]]):
def __init__(self, kwargs: Dict[str, Tuple[Tuple[int, int], int]], window_name: str = 'Debug Frames', label: str = None):
"""
label: when set, prefixes this instance's trackbar names with it (e.g.
"center: canny_low"). Needed when two Visualizers share a window_name
(compare mode) so trackbars for the same variable name in two
different algos don't collide.
"""
self.variables = kwargs.keys()
cv.namedWindow('Debug Frames')
self.window_name = window_name
self._trackbar_names = {name: (f'{label}: {name}' if label else name) for name in self.variables}
cv.namedWindow(self.window_name)
for name, info in kwargs.items():
slider_range, default_val = info
low_range, high_range = slider_range
cv.createTrackbar(name, 'Debug Frames', low_range, high_range, nothing)
cv.setTrackbarPos(name, 'Debug Frames', default_val)
trackbar_name = self._trackbar_names[name]
cv.createTrackbar(trackbar_name, self.window_name, low_range, high_range, nothing)
cv.setTrackbarPos(trackbar_name, self.window_name, default_val)

def three_stack(self, frames: List[np.ndarray]) -> List[np.ndarray]:
newLst = []
Expand Down Expand Up @@ -75,5 +84,5 @@ def display(self, frames: List[np.ndarray]) -> np.ndarray:
def update_vars(self) -> Dict[str, int]:
variable_values = {}
for var in self.variables:
variable_values[var] = cv.getTrackbarPos(var, 'Debug Frames')
variable_values[var] = cv.getTrackbarPos(self._trackbar_names[var], self.window_name)
return variable_values
Loading
Loading