Skip to content

README_WASM.md

Latest commit

 

History

History
216 lines (149 loc) · 11.7 KB

README_WASM.md

File metadata and controls

216 lines (149 loc) · 11.7 KB
Raw

SuperCollider on WebAssembly

WebAssembly (wasm) is used by browsers (and beyond) to provide native-like performance across a variety of platforms and processor architectures using the same binary. Since almost all browsers ship with a WebAssembly runtime, it is possible to run SuperCollider in a browser without any kind of installation.

scsynth and sclang have been ported to WebAssembly, exposing a JavaScript API to interact with them. They can be used independently on their own or together.

scsynth

Any interaction with scsynth needs to happen through OSC, so it is necessary to have a client which creates the appropriate OSC messages, which are passed as binary Uint8Array to the wasm version of scsynth. Therefore, instead of recreating a network interface in JavaScript, OSC messages are passed to scsynth via JavaScript functions.

In sclang, one can use .asRawOSC on an array to obtain the bytes of an OSC message.

Alternatively, use a JS client for scsynth or wrap the OSC communication in a WebSocket through a script.

A basic JS binding for building and parsing OSC messages is exposed by the function parseOscMessage and the object OscMessage().

Example

The example should be available via https://supercollider.dennis-scheiba.com/

An example on how to use the wasm version of scsynth in a website is given in platform/wasm/index.html and platform/wasm/init.js, which require the wasm build artifact scsynth.js and scsynth.wasm and should be served using the provided dev_server.py. Check the CI of SuperCollider to obtain the wasm build artifact.

Limitations

Source code

The basic idea of the wasm implementation is to remove the network stack since it requires socket access which is not available in a browser environment. Instead of "faking" a network stack, SC_ComPort.cpp is removed from compilation and the ReplyAddress struct also removes its boost::asio::ip::address mAddress member. Any OSC I/O is instead provided by JavaScript functions which write the messages into the scsynth World directly.

Beyond that, the main additions to the scsynth source code are

  • SC_WebAudioDriver, located in server/scsynth/SC_WebAudio.cpp, which implements a WebAudioWorklet as a "fake" audio driver which receives a callback from the real time environment of the browser to generate the next n samples. The emscripten documentation provides a good introduction on WebAudioWorklets, see https://emscripten.org/docs/api_reference/wasm_audio_worklets.html. It also contains all missing fake implementations necessary due to the removal of SC_ComPort.cpp and contains the JS<->CPP glue code using emscripten.
  • platform/wasm/SC_WebOsc.cpp which implements a basic OSC builder and parser for JavaScript using the already included oscpack
  • platform/wasm/scsynth.pre.js which provides some glue code and scaffolding such that a nice user facing JS API is available.

Additionally, all plugins are statically linked.

In server/scsynth/CMakeList.txt, the symbol to look out for is EMSCRIPTEN and within the source code any modifications are guarded by the __EMSCRIPTEN__ symbol.

sclang

Interaction with sclang happens via OSC and text I/O, where both kinds are accessible via a JavaScript API. Like for scsynth, all kind of networking is stripped away since wasm does not provide socket access. Instead, all incoming OSC messages should be passed as binary Uint8Array to sendOsc and all outgoing messages can be received as Uint8 array via onOsc. Take a look at the web editor how to interact with the API.

The current network mock is rather limited and aimed at communicating with the scsynth.wasm server. To keep the implementation easier, the concept of IP address and port has been removed, which limits the usage of the wasm build to single client/server setups. This may change in the future though.

Limitations

  • No file access yet. Will be available once the best approach for a filesystem in conjunction with scsynth and the browser has been determined.
  • DOM interaction is untested, but JS.runCode can run JS code in the main thread of the browser, which allows to interact with the DOM.
  • Any kind of OSCFunc/OSCdef that goes beyond the server is not implemented.
  • Clock-re-synchronization with the server is currently not in-place.

Source code

Any kind of patching in the source code is guarded by the __EMSCRIPTEN__ definition. The most crucial patching has been performed in

  • PyrObject.cpp: Compilation of the class library is not happening in a boost thread pool but using std::async - like the Windows implementation. Using the thread pool created termination problem of the thread pool.
  • OSCData.cpp: WASM does not allow to create sockets, so it is assumed that the network socket is located at 127.0.0.1:57120. Operations such as NetAddr.broadcastFlag are no-oped.
  • SC_ComPort.h: Any reference to a socket is removed

The terminal client SC_TerminalClient, which is derived from LanguageClient, is replaced with a SC_WasmClient, which exposes text and OSC I/O via a JavaScript API using embind.

Additionally, SC_WebOsc.cpp is also added to the build, which allows to construct and parse OSC messages.

Editor

The wasm exclusive target webeditor is a mockup of the SC IDE in the browser using CodeMirror 5, bundling sclang and scsynth. The source code is located in platform/wasm/editor. It acts as an example how to connect sclang and scsynth in a wasm environment.

Deployment

As mentioned, it is necessary to set HTTP headers in the webserver in order to make the necessary SharedArrayBuffers available. The provided dev_server.py allows to spin up a local development server with the necessary headers. This script should not be used to expose the files to the internet, instead use a webserver like nginx with a configuration along the following lines

server {
    server_name supercollider.dennis-scheiba.com;

    # Path to the root directory for serving static files
    root /home/scheiba/supercollider;

    # URL prefix where the static files are located
    location / {
        # Set the default index file
        index index.html;

        # Set additional HTTP headers
        add_header Cross-Origin-Embedder-Policy "require-corp";
        add_header Cross-Origin-Opener-Policy "same-origin";

        types {
          application/wasm wasm;
          text/html html;
          application/javascript js;
          text/css css;
          default text/plain;
        }
    }

    listen 80;
}

Use e.g. certbot to obtain a certificate for the domain, since a https context is also necessary for SharedArrayBuffer and AudioContext access.

Building

Building the wasm version is only necessary for development purposes. It is advised to use the wasm build from the CI artifacts for plain usage of SuperCollider in the web.

Install emscripten by following its download instructions.

Assuming the emsdk repository has been cloned, the setup looks in general something like

./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh

Refer to the CI file /.github/workflows/build_wasm.yml for the currently supported emscripten version.

This should make the emcc compiler available in this shell, which can be verified by running emcc -v.

Create a clean(!) build directory, e.g. build-emscripten and execute the following command in it to configure the build

emcmake cmake \
  -DSC_EL=no \
  -DSUPERNOVA=no \
  -DSC_HIDAPI=no \
  -DNO_LIBSNDFILE=yes \
  -DSTATIC_PLUGINS=yes \
  -DSC_QT=no \
  -DNO_AVAHI=yes \
  -DSC_ABLETON_LINK=no \
  -DCMAKE_BUILD_TYPE="Release" \
  -Wno-dev \
  -DNO_X11=yes \
  -DAUDIOAPI=webaudio \
  ..

Now build webeditor for wasm using emscripten by running

emmake cmake --build . --target webeditor

The build artifacts with examples will be written to <your_build_directry>/wasm-dist.

CLion setup

The wasm build has been developed using CLion, which is free to use for Open Source projects like SuperCollider and allows to easily switch between a "normal" and "emscripten" development pipeline.

The general idea is to create a new EMSDK toolchain within the CLion settings, using e.g. /Users/scheiba/github/emsdk/upstream/emscripten/em++ for its C++ compiler, bundled CMake and bundled LLDB as debugger.

After the toolchain has been added, a new CMake Profile, named e.g. Emscripten, can be added which uses the EMSDK toolchain. This needs to use its own build directory in order to avoid clashes with the local build!

Additionally, the following environment variables needs to be specified which should look something like

EMSDK=/Users/scheiba/github/emsdk;EMSDK_NODE=/Users/scheiba/github/emsdk/node/22.16.0_64bit/bin/node;EMSDK_PYTHON=/Users/scheiba/github/emsdk/python/3.13.3_64bit/bin/python3;SSL_CERT_FILE=/Users/scheiba/github/emsdk/python/3.13.3_64bit/lib/python3.13/site-packages/certifi/cacert.pem

and the following CMake options should be set

-DCMAKE_TOOLCHAIN_FILE=/Users/scheiba/github/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DSC_EL=no  -DSUPERNOVA=no  -DSC_HIDAPI=no  -DNO_LIBSNDFILE=yes  -DSC_QT=no  -DNO_AVAHI=yes  -DSC_ABLETON_LINK=no  -DCMAKE_BUILD_TYPE="Release" -Wno-dev -DSSE=no -DSSE2=no -DNO_X11=yes -DAUDIOAPI=webaudio

Debugging

Use the flag -DWASM_DEBUG=on to create a debug wasm build.

See https://developer.chrome.com/blog/wasm-debugging-2020 how to use chrome to debug the wasm built.

Credits

The first implementation of scsynth in wasm has been written by Hanns Holger Rutz (Sciss). The current implementation, which is enhanced by using AudioWorklets, does not retain any of the implementation from Hanns Holger Rutz except for a few lines of CMake and the demo SynthDef gcd is written by Dennis Scheiba.

sclang was ported by Dennis Scheiba and benefited heavily from a review by Chirstof Ressi.

License

scsynth and sclang are licensed via GPL-3.0. The wasm bindings of sclang and scsynth are licensed via AGPL-3.0.