Repository navigation
feat(backend): run the Python backend as a subprocess (drop embedded libpython) - #27
Conversation
…libpython) Embedding libpython in the plugin clashes with OBS' own obs-scripting Python symbols on macOS and breaks numpy/torch in-process. Instead of linking libpython, launch the user's Python as a separate process and talk to it over shared memory. - CMakeLists: drop find_package(Python3) and the include/link step; the plugin no longer embeds an interpreter. - DrawDock: spawn the backend with QProcess, stream its stdout into a log panel in the dock, manage deck lists from a decklists folder in the OBS plugin config dir, and handle the start/stop lifecycle. - shared_memory_wrapper: shared-memory transport between plugin and backend (shm_open/mmap on Unix, boost windows_shared_memory on Windows). - SettingsDialog: Python prefix selection and deck-list pickers. - README + all translations (es/fr/jp/pt): document building from source (now a universal build) and the macOS Python-backend setup, described for this subprocess model. Single-player; multi-detector (per-player) support is a separate change.
507463e to
a10a0b1
Compare
|
Sorry for taking so long to review this PR, but this change is important since it essentially sets the direction for others so it needs to be tested carefully; it also involves (from my understanding) an update to the Python backend to support the addition of the new Unsurprisingly, it works well on Linux; we just need to run additional tests on Windows to make sure it works and that it’s still possible to provide the plugin with the built-in “all-in-one” Python module via the installer. Thank you so much for the huge work you provided, I'll do my best to merge this one this week-end. |
…nstants in windows
|
This PR is almost ready to be merged, we still need to update the python backend in the |
|
Python backend in the This PR is ready to be merged, I'll just conduct one last test on windows to be sure everything works correctly as expected. |
|
That works perfectly fine on windows as expected. I'll just take advantage of this PR to also update Linux installation instructions, I'll do that tomorrow and merge that right after |
|
I'm merging this PR when all the actions are complete, thank you for the work, if I made any mistake in the doc or anywhere else, we can take advantage of the next PR to correct them, I'm reviewing it right away! Edit: Also I'll publish a new release when all the PR have been merged |
Summary
Stop embedding libpython in the plugin. Instead, launch the user's Python as a
separate process and communicate with it over shared memory. This is a
cross-platform change motivated by macOS, where embedding the interpreter is
not viable.
Motivation
Linking libpython into the plugin and running the backend in-process has two
problems:
obs-scriptingPython symbols, andimporting
numpy/torchin-process crashes.Python ABI, and pins the user to that exact interpreter version.
Running the backend as a subprocess removes the symbol clash entirely and lets
the user point the plugin at any compatible Python that has the
drawpackageinstalled.
Changes
CMakeLists.txt— dropfind_package(Python3)and the include/linkstep. The plugin no longer embeds an interpreter.
src/DrawDock.{cpp,hpp}— launch the backend withQProcess; stream itsstdout into a log panel in the dock; manage deck lists from a
decklistsfolder in the OBS plugin config dir; start/stop lifecycle.src/shared_memory_wrapper.cpp— shared-memory transport between theplugin and the backend. Cross-platform:
shm_open/mmapon Unix/macOS,boost::interprocess::windows_shared_memoryon Windows (matching Python'smultiprocessing.shared_memory).src/SettingsDialog.{cpp,hpp}— select the Python prefix and thedeck-list files used at launch.
data/locale/en-US.ini— strings for the new dock controls.Cross-platform
windows_shared_memory) and aUnix path (
shm_open/mmap);QProcessis cross-platform.Behaviour change (worth a maintainer's attention)
The plugin no longer bundles a Python interpreter on any platform. Users now
select a Python install that has the
drawbackend package. This PR updates theREADME (and all translations —
es/fr/jp/pt-br) accordingly: building fromsource is now a universal build, and the macOS Python-backend setup is documented
for this subprocess model.
How it was tested
Environment: macOS 26 (Apple Silicon), OBS Studio 32.1.1, a Python prefix with
the
drawbackend installed.This PR's scope is the process model, not the detection pipeline (the
capture + model code in
src/draw.cis untouched here), so the test confirmsthe backend now runs out-of-process and the plugin talks to it:
Build —
cmake --preset macos+cmake --build build_macos→ universalbuild succeeds (no libpython link; the previous arm64-only constraint from
embedding Homebrew's Python is gone).
Backend runs as a separate process — Start DRAW spawns the Python backend;
its stdout streams into the dock's log panel (impossible with an in-process
interpreter), the plugin and backend connect over shared memory
(
Shared memory found), and OBS does not crash — the libpython/obs-scriptingsymbol clash on macOS is gone.