Skip to content

Feat/roboflow workflow dsp - #860

Open
klemen1999 wants to merge 4 commits into
mainfrom
feat/roboflow_workflow_dsp
Open

Feat/roboflow workflow dsp#860
klemen1999 wants to merge 4 commits into
mainfrom
feat/roboflow_workflow_dsp

Conversation

@klemen1999

Copy link
Copy Markdown
Collaborator

Purpose

Summary

  • Migrate the Roboflow Workflow standalone app to the released OAK4 QNN runtime stack:

    • Use luxonis/oakapp-base:1.4.0-onnxruntime.
    • Pin depthai==3.9.0, depthai-nodes==0.6.1, and inference==1.3.7.
    • Run the app on the base image’s Python 3.12 runtime.
    • Pass through the Hexagon DSP/HTP devices and NPU runtime mount.
  • Route Roboflow-created ONNX Runtime sessions through depthai_nodes.runtime.onnx_qnn_session:

    • Fix dynamic batch dimensions to static batch size 1 before QNN compilation.
    • Run QNN on HTP with fp16 and reuse cached compiled graphs.
    • Fall back to CPU by default; expose EP=cpu and STRICT=1 for debugging.
  • Replace the custom OpenCV generator-capture/probe path with Roboflow’s VideoFrameProducer integration backed by a DepthAI output queue.

  • Discover workflow outputs and parameter definitions from the Roboflow API rather than performing dummy inference.

  • Support runtime workflow controls:

    • Switch API key, workspace, workflow ID, and workflow parameters.
    • Rebuild DepthAI visualizer topics when output schemas change.
    • Refresh the active workflow definition from Roboflow without redeploying.
    • Surface inference pipeline health/errors in the frontend.
  • Prevent workflow switching from aborting the app:

    • Quiesce AnnotationNode before DepthAI pipeline teardown.
    • Handle closed output queues inside the host-node callback.
  • Update the custom frontend:

    • Render controls from the discovered workflow interface.
    • Add error/retry and refresh UI states.
    • Align Luxonis frontend dependencies and enforce one React/ReactDOM version.
    • Limit Rollup parallel file operations and raise the frontend build file-descriptor limit to avoid EMFILE.
  • Simplify and update documentation:

    • Update README/agent guidance for released QNN support, runtime workflow controls, and visualization rules.

Specification

None / not applicable

Dependencies & Potential Impact

None / not applicable

Deployment Plan

None / not applicable

Testing & Validation

None / not applicable

AI Usage

Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

Submitted code was reviewed by a human: YES/NO

The author is taking the responsibility for the contribution: YES/NO

@klemen1999
klemen1999 requested a review from dtronmans August 28, 2026 13:51
@klemen1999
klemen1999 marked this pull request as ready for review August 28, 2026 13:51
@klemen1999 klemen1999 added the testable PR is ready to be tested label Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testable PR is ready to be tested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant