diff --git a/README.md b/README.md index 2095a82889..1b53f450b3 100644 --- a/README.md +++ b/README.md @@ -62,6 +62,49 @@ e.g. To point to the default dev port: VITE_JS_PLUGINS_DEV_PORT=4100 ``` +### Plugin module formats (ES module and CommonJS) + +Remotely-loaded plugins may be authored as either modern **ES modules** or +legacy **CommonJS** bundles. The host auto-detects the format of each plugin +entry at load time (using [es-module-lexer](https://github.com/guybedford/es-module-lexer)), +so no manifest changes are required and both formats are supported +indefinitely. + +ES module plugins can **code-split / lazy-load** parts of their UI via dynamic +`import()`, so only the code needed for a given view is fetched. They resolve +the same host singletons (`react`, `redux`, `@deephaven/*`, ...) provided by +[remote-component.config.ts](packages/app-utils/src/plugins/remote-component.config.ts) +through an import map the host registers at runtime with +[es-module-shims](https://github.com/guybedford/es-module-shims). The host runs +es-module-shims in **shim mode**, so every plugin module is loaded through +`importShim()` on all browsers (including Chromium and Safari), and the host's +own module scripts are never processed by the shim. + +To author an ES module plugin: + +- Build with Vite lib mode `formats: ['es']` and `build.cssCodeSplit: true`. +- Mark host singletons as `external` so they are emitted as bare `import` + statements rather than bundled (one React, one redux store, one design + system, ...). +- Use `vite-plugin-css-injected-by-js` with `relativeCSSInjection: true` so a + lazy chunk's compiled CSS/SCSS is injected when the chunk loads. +- Lazy-load heavier views with `React.lazy(() => import('./View'))` (and import + any `.scss`/`.css` inside the lazy module so its styles travel with the + chunk). + +See [@deephaven/plugin-example](packages/plugin-example) for a complete, +runnable example plus a tiny dev server. To try it locally: + +``` +npm run plugin-example # builds + serves on :4100 +# then set VITE_JS_PLUGINS_DEV_PORT=4100 in packages/code-studio/.env.development.local +npm start +``` + +Migrating existing `deephaven-plugins` from CommonJS to ES modules is optional +and can be done incrementally — see +[the migration guide](packages/app-utils/docs/migrating-plugins-to-esm.md). + ## Local Vite Config If you'd like to override the vite config for local dev, you can define a `packages/code-studio/vite.config.local.ts` file that extends from `vite.config.ts`. This file is excluded via `.gitignore` which makes it easy to keep local overrides in tact. diff --git a/package-lock.json b/package-lock.json index 2965bff0cb..6ffdad5a5c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -2307,6 +2307,10 @@ "resolved": "packages/plugin", "link": true }, + "node_modules/@deephaven/plugin-example": { + "resolved": "packages/plugin-example", + "link": true + }, "node_modules/@deephaven/pouch-storage": { "resolved": "packages/pouch-storage", "link": true @@ -5521,6 +5525,326 @@ "resolved": "https://registry.npmjs.org/@paciolan/remote-module-loader/-/remote-module-loader-3.0.3.tgz", "integrity": "sha512-gwdJcP5QQbO7OUf00FWh+A5DkF3TnIv06JB3aMpm9pbbHcdouFrjo4nEW7HtbWeIs7z3gwGEVd82clAmzVzh3Q==" }, + "node_modules/@parcel/watcher": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher/-/watcher-2.6.0.tgz", + "integrity": "sha512-7FNeNl8NCE7aINx7WXiKQrPYZWC/hvrTsmk6zmxbI7LTXE7hVek/n8AfVgpe2y82zl3w0HvCHN0bVKMBoJcC0w==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "dependencies": { + "detect-libc": "^2.0.3", + "is-glob": "^4.0.3", + "node-addon-api": "^7.0.0", + "picomatch": "^4.0.4" + }, + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + }, + "optionalDependencies": { + "@parcel/watcher-android-arm64": "2.6.0", + "@parcel/watcher-darwin-arm64": "2.6.0", + "@parcel/watcher-darwin-x64": "2.6.0", + "@parcel/watcher-freebsd-x64": "2.6.0", + "@parcel/watcher-linux-arm-glibc": "2.6.0", + "@parcel/watcher-linux-arm-musl": "2.6.0", + "@parcel/watcher-linux-arm64-glibc": "2.6.0", + "@parcel/watcher-linux-arm64-musl": "2.6.0", + "@parcel/watcher-linux-x64-glibc": "2.6.0", + "@parcel/watcher-linux-x64-musl": "2.6.0", + "@parcel/watcher-win32-arm64": "2.6.0", + "@parcel/watcher-win32-x64": "2.6.0" + } + }, + "node_modules/@parcel/watcher-android-arm64": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-android-arm64/-/watcher-android-arm64-2.6.0.tgz", + "integrity": "sha512-trgpLSCKRC/huFjXX/Smh+0sWe4+YtKfktIToiMl59ghz7z+qkH6kMvNnUbLyRs9N11t8l4svSCs1+5B3rOAhA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-darwin-arm64": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-darwin-arm64/-/watcher-darwin-arm64-2.6.0.tgz", + "integrity": "sha512-Y3QV0gl7Q1zbfueunkWIERICbEojQFCgpyG7YqOGNFLsckXyI1xu9mAIUpKY9QBYzBtSkN8dBPwd3yiAO9ovMw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-darwin-x64": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-darwin-x64/-/watcher-darwin-x64-2.6.0.tgz", + "integrity": "sha512-Ohv6OpzhUfKYD7Beb8kDvG0jbIxORCYY1JRdZnaBtnjjkJxgD7ZVL0nw2sCYd0yTMKTvz3nnTnOF3cDifK+kvw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-freebsd-x64": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-freebsd-x64/-/watcher-freebsd-x64-2.6.0.tgz", + "integrity": "sha512-5HmXvDgs8VK+74jF9y9/2FE3/OnlcKmc56tjmSrEuZjpSZOGL+fvAu+HKJBdPs9uwoP2hE6TlSUpXZ/C5jUFmQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-linux-arm-glibc": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm-glibc/-/watcher-linux-arm-glibc-2.6.0.tgz", + "integrity": "sha512-Ps/hui3A+vMbjdqlqAowK2ZL8+BO8dBjxeWXj6npTBs3jx4wWmbPpaLuqwrQrSqIVMCnpWo238bJ1U37GhQOYg==", + "cpu": [ + "arm" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-linux-arm-musl": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm-musl/-/watcher-linux-arm-musl-2.6.0.tgz", + "integrity": "sha512-9c6AUHgHoG+IY88MRIHupztQiQnrbqHYQjkM2btA+Bf/wQnQMuiD0Wfk1EVv3TlNT3x41uU71rn6E4xh/+zvkw==", + "cpu": [ + "arm" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-linux-arm64-glibc": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm64-glibc/-/watcher-linux-arm64-glibc-2.6.0.tgz", + "integrity": "sha512-yHRqS2owEXe6Hic9z6Mh1ECsCd+ODVOGvZDyciqRd21+v+o+DnXMOrw50DSpIG2sb8GPEaPPmfeCAWKPJdq46g==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-linux-arm64-musl": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-arm64-musl/-/watcher-linux-arm64-musl-2.6.0.tgz", + "integrity": "sha512-WhB2e/V7rqdHHWZusBSPuy5Ei8S6lSz6FE5TKKQz5h3a0O+C+mhY7vxU9b/stqvMb8beLnPY82ZrFTLKs+SrKA==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-linux-x64-glibc": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-x64-glibc/-/watcher-linux-x64-glibc-2.6.0.tgz", + "integrity": "sha512-ulGE6x6Oz6iAwg75T8YQSoguBWasniIbX+QWpaYPcCnDOpdWX3k+4xbEYPZVLxOuoJI+svJJPD3sEj8G7lrQ3A==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-linux-x64-musl": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-linux-x64-musl/-/watcher-linux-x64-musl-2.6.0.tgz", + "integrity": "sha512-tkBYKt7YQrjIJWYDnto2YgO8MRkjlMTSNoRHzsXinBqbLdeOM3L32wPZJvIZxqaLMfSlS/4sUjH/6STVP/XDLw==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-win32-arm64": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-arm64/-/watcher-win32-arm64-2.6.0.tgz", + "integrity": "sha512-gIZAP23jaHjGWasY/TY6yL7NHFClf0Ga7FN+iINvk+KN94rhm94lYZhFsbYFNcA04/onvGD9kKmiJLJB2HbNwQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher-win32-x64": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/@parcel/watcher-win32-x64/-/watcher-win32-x64-2.6.0.tgz", + "integrity": "sha512-cA+/pXV2YkfxlIcXOQ5fSWqAzzPyD78/x5qbK/I0vUkrlYHA8TIz+MXjAbGouguKVSI4bOmkTSJ1/poVSsgt+A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/@parcel/watcher/node_modules/picomatch": { + "version": "4.0.7", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz", + "integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==", + "dev": true, + "license": "MIT", + "optional": true, + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, "node_modules/@pkgr/core": { "version": "0.2.9", "resolved": "https://registry.npmjs.org/@pkgr/core/-/core-0.2.9.tgz", @@ -11181,6 +11505,17 @@ "resolved": "https://registry.npmjs.org/detect-kerning/-/detect-kerning-2.1.2.tgz", "integrity": "sha512-I3JIbrnKPAntNLl1I6TpSQQdQ4AutYzv/sKMFKbepawV/hlH0GmYKhUoOEMd4xqaUHT+Bm0f4127lh5qs1m1tw==" }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "dev": true, + "license": "Apache-2.0", + "optional": true, + "engines": { + "node": ">=8" + } + }, "node_modules/detect-newline": { "version": "3.1.0", "resolved": "https://registry.npmjs.org/detect-newline/-/detect-newline-3.1.0.tgz", @@ -11724,6 +12059,18 @@ "node": ">= 0.4" } }, + "node_modules/es-module-lexer": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.7.0.tgz", + "integrity": "sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==", + "license": "MIT" + }, + "node_modules/es-module-shims": { + "version": "2.8.4", + "resolved": "https://registry.npmjs.org/es-module-shims/-/es-module-shims-2.8.4.tgz", + "integrity": "sha512-ea5srn5L89PWVad6Qle6r2kg+HvvLiL/GHqgNx06eFrkERHkrTFWaqo1w8Sd+XI3Xr0iAG5x3ZQ7mQ/hnQzNpA==", + "license": "MIT" + }, "node_modules/es-object-atoms": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz", @@ -19064,6 +19411,14 @@ "integrity": "sha512-1nh45deeb5olNY7eX82BkPO7SSxR5SSYJiPTrTdFUVYwAl8CKMA5N9PjTYkHiRjisVcxcQ1HXdLhx2qxxJzLNQ==", "dev": true }, + "node_modules/node-addon-api": { + "version": "7.1.1", + "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-7.1.1.tgz", + "integrity": "sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ==", + "dev": true, + "license": "MIT", + "optional": true + }, "node_modules/node-fetch": { "version": "2.6.7", "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.7.tgz", @@ -25192,6 +25547,16 @@ } } }, + "node_modules/vite-plugin-css-injected-by-js": { + "version": "3.5.2", + "resolved": "https://registry.npmjs.org/vite-plugin-css-injected-by-js/-/vite-plugin-css-injected-by-js-3.5.2.tgz", + "integrity": "sha512-2MpU/Y+SCZyWUB6ua3HbJCrgnF0KACAsmzOQt1UvRVJCGF6S8xdA3ZUhWcWdM9ivG4I5az8PnQmwwrkC2CAQrQ==", + "dev": true, + "license": "MIT", + "peerDependencies": { + "vite": ">2.0.0-0" + } + }, "node_modules/vt-pbf": { "version": "3.1.3", "resolved": "https://registry.npmjs.org/vt-pbf/-/vt-pbf-3.1.3.tgz", @@ -25802,6 +26167,8 @@ "@paciolan/remote-component": "2.13.0", "@paciolan/remote-module-loader": "^3.0.2", "classnames": "^2.5.1", + "es-module-lexer": "^1.5.4", + "es-module-shims": "^2.0.0", "lodash.debounce": "^4.0.8", "lodash.merge": "^4.6.2", "lodash.throttle": "^4.1.1" @@ -26484,6 +26851,7 @@ "node": ">=16" } }, + "packages/jsapi-types": {}, "packages/jsapi-utils": { "name": "@deephaven/jsapi-utils", "version": "1.29.0", @@ -26577,6 +26945,132 @@ } } }, + "packages/plugin-example": { + "name": "@deephaven/plugin-example", + "version": "1.22.0", + "license": "Apache-2.0", + "dependencies": { + "@deephaven/components": "file:../components", + "@deephaven/icons": "file:../icons", + "@deephaven/jsapi-types": "file:../jsapi-types", + "@deephaven/log": "file:../log", + "@deephaven/plugin": "file:../plugin" + }, + "devDependencies": { + "@vitejs/plugin-react-swc": "^3.7.0", + "react": "^17.0.2", + "react-dom": "^17.0.2", + "sass": "^1.62.1", + "vite": "^5.4.2", + "vite-plugin-css-injected-by-js": "^3.5.2" + }, + "peerDependencies": { + "react": "^17.0.2", + "react-dom": "^17.0.2" + } + }, + "packages/plugin-example/node_modules/@deephaven/jsapi-types": { + "resolved": "packages/jsapi-types", + "link": true + }, + "packages/plugin-example/node_modules/chokidar": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-5.0.0.tgz", + "integrity": "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==", + "dev": true, + "license": "MIT", + "dependencies": { + "readdirp": "^5.0.0" + }, + "engines": { + "node": ">= 20.19.0" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "packages/plugin-example/node_modules/immutable": { + "version": "5.1.9", + "resolved": "https://registry.npmjs.org/immutable/-/immutable-5.1.9.tgz", + "integrity": "sha512-m8nVez3rwrgmWxtLMt1ZYXB2Lv7OKYn/disyxAlSDYAlKSlFoPPfIAmAM/M5xqL4m4C/wAPw7S2/CNaUii1Hxg==", + "dev": true, + "license": "MIT" + }, + "packages/plugin-example/node_modules/react": { + "version": "17.0.2", + "resolved": "https://registry.npmjs.org/react/-/react-17.0.2.tgz", + "integrity": "sha512-gnhPt75i/dq/z3/6q/0asP78D0u592D5L1pd7M8P+dck6Fu/jJeL6iVVK23fptSUZj8Vjf++7wXA8UNclGQcbA==", + "dev": true, + "license": "MIT", + "dependencies": { + "loose-envify": "^1.1.0", + "object-assign": "^4.1.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "packages/plugin-example/node_modules/react-dom": { + "version": "17.0.2", + "resolved": "https://registry.npmjs.org/react-dom/-/react-dom-17.0.2.tgz", + "integrity": "sha512-s4h96KtLDUQlsENhMn1ar8t2bEa+q/YAtj8pPPdIjPDGBDIVNsrD9aXNWqspUe6AzKCIG0C1HZZLqLV7qpOBGA==", + "dev": true, + "license": "MIT", + "dependencies": { + "loose-envify": "^1.1.0", + "object-assign": "^4.1.1", + "scheduler": "^0.20.2" + }, + "peerDependencies": { + "react": "17.0.2" + } + }, + "packages/plugin-example/node_modules/readdirp": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-5.1.1.tgz", + "integrity": "sha512-Kko+Y5XQ6fM+Ce3dq3m9YGxnacYZYl9cA1wZjaF3Vbry2L3i1qVg8+CAgNPsXRArPMUMCaOR7oa9Nqntc43JKA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 20.19.0" + }, + "funding": { + "type": "individual", + "url": "https://paulmillr.com/funding/" + } + }, + "packages/plugin-example/node_modules/sass": { + "version": "1.104.1", + "resolved": "https://registry.npmjs.org/sass/-/sass-1.104.1.tgz", + "integrity": "sha512-yDA+1aIG3EHgN4V/BvuhCvu61FF6hEd4e+9DxikUm9U0CAGuvdIZ/UYy7qbOxjhbbWQraoyLlMuzdRGOSV5Bmw==", + "dev": true, + "license": "MIT", + "dependencies": { + "chokidar": "^5.0.0", + "immutable": "^5.1.5", + "source-map-js": ">=0.6.2 <2.0.0" + }, + "bin": { + "sass": "sass.js" + }, + "engines": { + "node": ">=20.19.0" + }, + "optionalDependencies": { + "@parcel/watcher": "^2.4.1" + } + }, + "packages/plugin-example/node_modules/scheduler": { + "version": "0.20.2", + "resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.20.2.tgz", + "integrity": "sha512-2eWfGgAqqWFGqtdMmcL5zCMK1U8KlXv8SQFGglL3CEtd0aDVDWgeF/YoCmvln55m5zSk3J/20hTaSBeSObsQDQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "loose-envify": "^1.1.0", + "object-assign": "^4.1.1" + } + }, "packages/pouch-storage": { "name": "@deephaven/pouch-storage", "version": "1.30.0", diff --git a/package.json b/package.json index c31d457a83..01ab6f3da3 100644 --- a/package.json +++ b/package.json @@ -43,6 +43,7 @@ "start": "run-s build:necessary dev", "start:app": "lerna run start --scope=@deephaven/code-studio --stream", "start:embed-widget": "lerna run start --scope=@deephaven/embed-widget --stream", + "plugin-example": "lerna run start --scope=@deephaven/plugin-example --stream", "test": "jest --watch --changedSince origin/main", "test:debug": "node --inspect-brk node_modules/.bin/jest --config jest.config.unit.cjs --runInBand --watch", "test:unit": "npm run build:necessary && jest --config jest.config.unit.cjs", diff --git a/packages/app-utils/docs/migrating-plugins-to-esm.md b/packages/app-utils/docs/migrating-plugins-to-esm.md new file mode 100644 index 0000000000..1bab5b00dc --- /dev/null +++ b/packages/app-utils/docs/migrating-plugins-to-esm.md @@ -0,0 +1,156 @@ +# Migrating Deephaven plugins from CommonJS to ES modules + +The web UI's remote plugin loader supports both **CommonJS (CJS)** and modern +**ES module (ESM)** plugins. The format of each plugin is auto-detected at load +time, so migrating is **optional and incremental** — CJS plugins continue to +work indefinitely, and you can migrate one plugin at a time. + +> This guide targets the JS bundle build of plugins in the +> [deephaven-plugins](https://github.com/deephaven/deephaven-plugins) repo. The +> migration should land as its own commit, separate from any behavioral change. + +## Why migrate? + +ES module plugins can **code-split** and **lazy-load** parts of their UI with +dynamic `import()`. Instead of shipping one large bundle that must be fully +downloaded and evaluated before the plugin can be used, an ESM plugin only +fetches the code (and styles) for the view actually being shown. They still +resolve the host's shared singletons (`react`, `redux`, `@deephaven/*`, ...) +rather than bundling their own copies. + +## How loading differs + +| | CommonJS (today) | ES module (new) | +| --------------- | -------------------------------------------------------- | -------------------------------------------------- | +| Bundle format | `formats: ['cjs']` | `formats: ['es']` | +| Host singletons | Custom `require` backed by the resolve map | Bare `import` resolved via runtime import map | +| Loading | Bundle text `eval`'d by `@paciolan/remote-module-loader` | `importShim()` from `es-module-shims` | +| Lazy loading | Not supported (single bundle) | `import()` / `React.lazy` produce on-demand chunks | +| Styling | Bundled into the single entry | Per-chunk CSS injected on import | + +No manifest (`manifest.json`) changes are required. The host fetches each +plugin's `main` entry, detects ESM vs CJS from the source, and loads it +accordingly. + +## Migration steps + +### 1. Switch the Vite library format to ESM + +```ts +// vite.config.ts +build: { + cssCodeSplit: true, // emit per-chunk CSS so lazy chunks carry their styles + lib: { + entry: './src/index.ts', + formats: ['es'], // was ['cjs'] + fileName: () => 'index.js', + }, + rollupOptions: { + // Keep the SAME external list you already use for CJS. These are provided + // by the host at runtime and must NOT be bundled. + external: [ + 'react', + 'react-dom', + 'redux', + 'react-redux', + '@deephaven/components', + '@deephaven/dashboard', + '@deephaven/icons', + '@deephaven/jsapi-bootstrap', + '@deephaven/jsapi-types', + '@deephaven/log', + '@deephaven/plugin', + // ...any other @deephaven/* host packages your plugin imports + ], + }, +}, +``` + +The host resolves these `external` specifiers through the import map it injects +at runtime from +[remote-component.config.ts](../src/plugins/remote-component.config.ts), so the +plugin shares the host's exact instances. + +### 2. Keep `package.json` consistent + +Ensure the package is treated as an ES module and points at the ESM entry: + +```jsonc +{ + "type": "module", + "main": "dist/index.js" +} +``` + +The `manifest.json` entry (`name`, `version`, `main`, and the nested +`loader.package`) stays the same. + +### 3. Inject per-chunk CSS via JS + +So a lazy chunk's compiled CSS/SCSS is added to the document when the chunk is +imported (rather than relying on a separate `` the host doesn't know +about): + +```ts +import cssInjectedByJsPlugin from 'vite-plugin-css-injected-by-js'; + +plugins: [ + react(), + cssInjectedByJsPlugin({ relativeCSSInjection: true }), +], +``` + +### 4. (Optional) Add lazy loading + +This is the main benefit of migrating. Split heavier views into their own +chunks and import any styles inside the lazy module so the CSS travels with the +chunk: + +```tsx +// EntryView.tsx +const HeavyView = React.lazy(() => import('./HeavyView')); + +function EntryView(props) { + return ( + }> + + + ); +} +``` + +```tsx +// HeavyView.tsx +import './HeavyView.scss'; // compiled + injected with this chunk +export default function HeavyView() { + /* ... */ +} +``` + +Dynamic imports of other modules (including additional `.scss`/`.css`) also work +and produce additional on-demand chunks. + +## Verifying the migration + +1. Build the plugin and serve it (or use the + [example plugin's dev server](../../plugin-example/serve.js) as a template). +2. Point the host at it with `VITE_JS_PLUGINS_DEV_PORT` and run `npm start`. +3. In the browser devtools Network tab, confirm: + - The plugin entry loads as a module. + - Lazy chunks (and their CSS) are fetched only when the corresponding view is + opened — not on initial plugin load. +4. Confirm the plugin renders and is styled correctly, and that there is only + one React instance (no "invalid hook call" / duplicate-React errors), which + verifies host singletons are being shared. + +## Notes and limitations + +- CJS plugins remain fully supported; migrate incrementally. +- Cross-plugin imports by package name work ESM→ESM. An ESM plugin importing a + CJS plugin (or vice versa) by package name is not supported — migrate + interdependent plugins together if they import each other. +- The import map is registered at runtime with `es-module-shims` running in + **shim mode**, so plugin modules are loaded through `importShim()` on every + browser rather than passing through to the native loader. Shim mode only + affects modules loaded via `importShim()`, so the host application's own + module scripts are unaffected. diff --git a/packages/app-utils/package.json b/packages/app-utils/package.json index 9856b0dbde..5a36e8231e 100644 --- a/packages/app-utils/package.json +++ b/packages/app-utils/package.json @@ -49,6 +49,8 @@ "@paciolan/remote-component": "2.13.0", "@paciolan/remote-module-loader": "^3.0.2", "classnames": "^2.5.1", + "es-module-lexer": "^1.5.4", + "es-module-shims": "^2.0.0", "lodash.debounce": "^4.0.8", "lodash.merge": "^4.6.2", "lodash.throttle": "^4.1.1" diff --git a/packages/app-utils/src/declarations.d.ts b/packages/app-utils/src/declarations.d.ts new file mode 100644 index 0000000000..f38a10dc82 --- /dev/null +++ b/packages/app-utils/src/declarations.d.ts @@ -0,0 +1,3 @@ +// es-module-shims ships global type declarations only (no module exports), so +// declare the side-effect module for the dynamic import in esmPluginLoader. +declare module 'es-module-shims'; diff --git a/packages/app-utils/src/plugins/PluginUtils.test.ts b/packages/app-utils/src/plugins/PluginUtils.test.ts index a8aa57e89f..38d381e987 100644 --- a/packages/app-utils/src/plugins/PluginUtils.test.ts +++ b/packages/app-utils/src/plugins/PluginUtils.test.ts @@ -7,22 +7,42 @@ import { import { loadModulePlugins } from './PluginUtils'; import { resolve } from './remote-component.config'; -jest.mock('./loadRemoteModule', () => { - const mockLoadRemoteModule = jest.fn(); +jest.mock('./loadCommonJsModule', () => { + const mockLoadCommonJsModule = jest.fn(); return { __esModule: true, - default: mockLoadRemoteModule, - loadRemoteModule: mockLoadRemoteModule, + default: mockLoadCommonJsModule, + loadCommonJsModule: mockLoadCommonJsModule, }; }); +// ESM plugin loading touches es-module-shims / blob URLs which aren't available +// in jsdom. Mock it so loadModulePlugins exercises the CommonJS path by default; +// individual tests can override isEsModuleSource to exercise the ESM path. +jest.mock('./esmPluginLoader', () => ({ + __esModule: true, + addImportMap: jest.fn(), + buildHostImportMap: jest.fn(() => ({ imports: {} })), + buildPluginImportMap: jest.fn(() => ({ imports: {} })), + isEsModuleSource: jest.fn().mockResolvedValue(false), + loadEsModulePlugin: jest.fn(), +})); + // eslint-disable-next-line @typescript-eslint/no-var-requires, global-require -const { default: loadRemoteModule } = require('./loadRemoteModule') as { +const { default: loadCommonJsModule } = require('./loadCommonJsModule') as { default: jest.Mock; }; +const { isEsModuleSource, loadEsModulePlugin } = + // eslint-disable-next-line @typescript-eslint/no-var-requires, global-require + require('./esmPluginLoader') as { + isEsModuleSource: jest.Mock; + loadEsModulePlugin: jest.Mock; + }; + describe('loadModulePlugins', () => { const BASE_URL = 'http://localhost:4100/plugins'; + const PLUGIN_SOURCE = 'module.exports = {};'; // Snapshot the resolve map and global.fetch once so each test starts // from a known baseline regardless of what previous tests added. @@ -34,10 +54,13 @@ describe('loadModulePlugins', () => { } function mockManifest(plugins: PluginManifestPluginInfo[]) { - global.fetch = jest.fn().mockResolvedValue({ - ok: true, - json: jest.fn().mockResolvedValue({ plugins }), - }); + global.fetch = jest + .fn() + .mockImplementation(async (url: string) => + url.endsWith('/manifest.json') + ? { ok: true, json: async () => ({ plugins }) } + : { ok: true, text: async () => PLUGIN_SOURCE } + ); } beforeEach(() => { @@ -63,9 +86,9 @@ describe('loadModulePlugins', () => { { name: 'test-plugin-b', main: 'index.js', version: '2.0.0' }, ]); - loadRemoteModule - .mockResolvedValueOnce(pluginA) - .mockResolvedValueOnce(pluginB); + loadCommonJsModule + .mockReturnValueOnce(pluginA) + .mockReturnValueOnce(pluginB); const pluginMap = await loadModulePlugins(BASE_URL); @@ -93,7 +116,7 @@ describe('loadModulePlugins', () => { }, ]); - loadRemoteModule.mockResolvedValueOnce(moduleExports); + loadCommonJsModule.mockReturnValueOnce(moduleExports); await loadModulePlugins(BASE_URL); @@ -109,7 +132,7 @@ describe('loadModulePlugins', () => { { name: 'test-plugin-a', main: 'index.js', version: '1.0.0' }, ]); - loadRemoteModule.mockResolvedValueOnce(moduleExports); + loadCommonJsModule.mockReturnValueOnce(moduleExports); await loadModulePlugins(BASE_URL); @@ -144,14 +167,13 @@ describe('loadModulePlugins', () => { // Plugin B depends on A, so it loads in the next level. // When plugin B loads, verify plugin A is already in the resolve map. - loadRemoteModule.mockResolvedValueOnce(moduleA).mockImplementationOnce( - () => - new Promise(res => { - // At this point, plugin A should already be registered - expect(resolve['@deephaven/js-plugin-test-plugin-a']).toBe(moduleA); - res(pluginB); - }) - ); + loadCommonJsModule + .mockReturnValueOnce(moduleA) + .mockImplementationOnce(() => { + // At this point, plugin A should already be registered + expect(resolve['@deephaven/js-plugin-test-plugin-a']).toBe(moduleA); + return pluginB; + }); await loadModulePlugins(BASE_URL); @@ -167,9 +189,11 @@ describe('loadModulePlugins', () => { { name: 'test-plugin-b', main: 'index.js', version: '1.0.0' }, ]); - loadRemoteModule - .mockRejectedValueOnce(new Error('Network error')) - .mockResolvedValueOnce(pluginB); + loadCommonJsModule + .mockImplementationOnce(() => { + throw new Error('Evaluation error'); + }) + .mockReturnValueOnce(pluginB); const pluginMap = await loadModulePlugins(BASE_URL); @@ -191,13 +215,36 @@ describe('loadModulePlugins', () => { }, ]); - loadRemoteModule.mockRejectedValueOnce(new Error('Load failed')); + loadCommonJsModule.mockImplementationOnce(() => { + throw new Error('Load failed'); + }); await loadModulePlugins(BASE_URL); expect(resolve['@deephaven/js-plugin-test-plugin-a']).toBeUndefined(); }); + it('does not register a plugin whose entry fails to fetch', async () => { + global.fetch = jest.fn().mockImplementation(async (url: string) => + url.endsWith('/manifest.json') + ? { + ok: true, + json: async () => ({ + plugins: [ + { name: 'test-plugin-a', main: 'index.js', version: '1.0.0' }, + ], + }), + } + : { ok: false, status: 404, statusText: 'Not Found' } + ); + + const pluginMap = await loadModulePlugins(BASE_URL); + + expect(pluginMap.size).toBe(0); + expect(loadCommonJsModule).not.toHaveBeenCalled(); + expect(loadEsModulePlugin).not.toHaveBeenCalled(); + }); + it('returns empty map when manifest fetch fails', async () => { global.fetch = jest.fn().mockResolvedValue({ ok: false, @@ -234,7 +281,7 @@ describe('loadModulePlugins', () => { { name: 'test-plugin-multi', main: 'index.js', version: '1.0.0' }, ]); - loadRemoteModule.mockResolvedValueOnce(multiPlugin); + loadCommonJsModule.mockReturnValueOnce(multiPlugin); const pluginMap = await loadModulePlugins(BASE_URL); @@ -247,20 +294,48 @@ describe('loadModulePlugins', () => { expect(resolve['@deephaven/js-plugin-test-plugin-multi']).toBeUndefined(); }); - it('loads plugins from correct URLs based on manifest', async () => { + it('fetches each plugin entry exactly once from the manifest URL', async () => { const pluginA = makePlugin('test-plugin-a'); mockManifest([ { name: 'test-plugin-a', main: 'bundle.js', version: '1.0.0' }, ]); - loadRemoteModule.mockResolvedValueOnce(pluginA); + loadCommonJsModule.mockReturnValueOnce(pluginA); await loadModulePlugins(BASE_URL); - expect(loadRemoteModule).toHaveBeenCalledWith( - `${BASE_URL}/test-plugin-a/bundle.js` + const pluginUrl = `${BASE_URL}/test-plugin-a/bundle.js`; + const pluginFetches = (global.fetch as jest.Mock).mock.calls.filter( + ([url]) => url === pluginUrl ); + expect(pluginFetches.length).toBe(1); + expect(loadCommonJsModule).toHaveBeenCalledWith(PLUGIN_SOURCE); + }); + + it('loads ES module plugins with the already-fetched source', async () => { + const pluginA = makePlugin('test-plugin-a'); + + mockManifest([ + { name: 'test-plugin-a', main: 'index.js', version: '1.0.0' }, + ]); + + isEsModuleSource.mockResolvedValueOnce(true); + loadEsModulePlugin.mockResolvedValueOnce({ default: pluginA }); + + const pluginMap = await loadModulePlugins(BASE_URL); + + const pluginUrl = `${BASE_URL}/test-plugin-a/index.js`; + const pluginFetches = (global.fetch as jest.Mock).mock.calls.filter( + ([url]) => url === pluginUrl + ); + expect(pluginFetches.length).toBe(1); + expect(loadEsModulePlugin).toHaveBeenCalledWith(pluginUrl, PLUGIN_SOURCE); + expect(loadCommonJsModule).not.toHaveBeenCalled(); + expect(pluginMap.get('test-plugin-a')).toEqual({ + ...pluginA, + version: '1.0.0', + }); }); it('uses manifest package field as resolve key when provided', async () => { @@ -276,7 +351,7 @@ describe('loadModulePlugins', () => { }, ]); - loadRemoteModule.mockResolvedValueOnce(moduleExports); + loadCommonJsModule.mockReturnValueOnce(moduleExports); await loadModulePlugins(BASE_URL); diff --git a/packages/app-utils/src/plugins/PluginUtils.ts b/packages/app-utils/src/plugins/PluginUtils.ts index 2d004e8498..af85c49988 100644 --- a/packages/app-utils/src/plugins/PluginUtils.ts +++ b/packages/app-utils/src/plugins/PluginUtils.ts @@ -15,8 +15,15 @@ import { type PluginManifestPluginInfo, getPluginModuleValue, } from '@deephaven/plugin'; -import loadRemoteModule from './loadRemoteModule'; -import { resolve } from './remote-component.config'; +import loadCommonJsModule from './loadCommonJsModule'; +import { esmResolve, resolve } from './remote-component.config'; +import { + addImportMap, + buildHostImportMap, + buildPluginImportMap, + isEsModuleSource, + loadEsModulePlugin, +} from './esmPluginLoader'; const log = Log.module('@deephaven/app-utils.PluginUtils'); @@ -36,15 +43,28 @@ export type { PluginManifest }; export { getPluginModuleValue }; /** - * Imports a commonjs plugin module from the provided URL + * Imports a plugin module from the provided URL. The entry is fetched once and + * its format detected from the source: ES module plugins are handed to + * es-module-shims (so they can lazy-load chunks and resolve host singletons + * through the import map), while legacy CommonJS plugins are evaluated with a + * `require` backed by the host resolve map. * @param pluginUrl The URL of the plugin to load * @returns The loaded module */ export async function loadModulePlugin( pluginUrl: string ): Promise { - const myModule = await loadRemoteModule(pluginUrl); - return myModule; + const res = await fetch(pluginUrl); + if (!res.ok) { + throw new Error(`${res.status} ${res.statusText} (${pluginUrl})`); + } + const source = await res.text(); + if (await isEsModuleSource(source)) { + return (await loadEsModulePlugin(pluginUrl, source)) as + | LegacyPlugin + | { default: Plugin }; + } + return loadCommonJsModule(source); } /** @@ -86,6 +106,12 @@ export async function loadModulePlugins( log.debug('Plugin manifest loaded:', manifest); + // Register import maps before loading any plugin so ESM plugins can + // resolve host singletons (react, @deephaven/*, ...) and cross-plugin + // package imports. Harmless for CommonJS plugins, which never use them. + addImportMap(buildHostImportMap(esmResolve)); + addImportMap(buildPluginImportMap(manifest.plugins, modulePluginsUrl)); + const levels = groupByDependencyLevel(manifest.plugins); const pluginMap: PluginModuleMap = new Map(); diff --git a/packages/app-utils/src/plugins/esmPluginLoader.test.ts b/packages/app-utils/src/plugins/esmPluginLoader.test.ts new file mode 100644 index 0000000000..4089f05328 --- /dev/null +++ b/packages/app-utils/src/plugins/esmPluginLoader.test.ts @@ -0,0 +1,243 @@ +import { + addImportMap, + buildHostImportMap, + buildPluginImportMap, + ensureEsModuleShims, + isEsModuleSource, + loadEsModulePlugin, +} from './esmPluginLoader'; + +const SHARED_MODULES_KEY = '__DH_SHARED_PLUGIN_MODULES__'; + +// es-module-shims sets window.importShim as a side effect when imported. +const mockImportShim = Object.assign(jest.fn(), { + addImportMap: jest.fn(), +}); +jest.mock('es-module-shims', () => { + (globalThis as unknown as { importShim: unknown }).importShim = + mockImportShim; + return {}; +}); + +const blobsByUrl = new Map(); + +beforeAll(() => { + // jsdom doesn't implement createObjectURL. + let counter = 0; + URL.createObjectURL = jest.fn((blob: Blob) => { + // eslint-disable-next-line no-plusplus + const url = `blob:mock-${counter++}`; + blobsByUrl.set(url, blob); + return url; + }); +}); + +function readBlob(blob: Blob): Promise { + return new Promise((resolve, reject) => { + const reader = new FileReader(); + reader.onload = () => resolve(reader.result as string); + reader.onerror = () => reject(reader.error); + reader.readAsText(blob); + }); +} + +/** + * Evaluate a generated re-export module by rewriting its `export` statements + * into assignments, returning what an importer would see. + */ +async function evaluateReexportModule( + url: string +): Promise> { + const source = await readBlob(blobsByUrl.get(url) as Blob); + const body = source + .replace('export default ', 'exports.default = ') + .replace(/export const (\w+) =/g, 'exports.$1 ='); + const exports: Record = {}; + // eslint-disable-next-line no-new-func + new Function('window', 'exports', body)(window, exports); + return exports; +} + +beforeEach(() => { + delete (window as unknown as Record)[SHARED_MODULES_KEY]; + mockImportShim.mockReset(); + mockImportShim.addImportMap.mockReset(); +}); + +describe('isEsModuleSource', () => { + it('detects modules with export statements', async () => { + expect(await isEsModuleSource('export const foo = 1;')).toBe(true); + expect(await isEsModuleSource('export default function () {}')).toBe(true); + }); + + it('detects modules with static imports', async () => { + expect(await isEsModuleSource('import React from "react";')).toBe(true); + }); + + it('treats CommonJS source as not an ES module', async () => { + expect( + await isEsModuleSource( + 'module.exports = function () { return 1; };\nvar x = require("react");' + ) + ).toBe(false); + }); + + it('does not treat a lone dynamic import as an ES module', async () => { + expect(await isEsModuleSource('const p = import("./lazy.js");')).toBe( + false + ); + }); +}); + +describe('buildHostImportMap', () => { + it('re-exports default and named exports of host modules', async () => { + const react = { useState: jest.fn(), createElement: jest.fn() }; + const Log = { module: jest.fn() }; + const logInit = jest.fn(); + const logModule = { default: Log, Log, logInit }; + const map = buildHostImportMap({ react, '@deephaven/log': logModule }); + + expect(map.imports.react).toMatch(/^blob:mock-/); + expect( + (window as unknown as Record>)[ + SHARED_MODULES_KEY + ].react + ).toBe(react); + + // A module without an explicit default uses the value itself as default. + const reactExports = await evaluateReexportModule(map.imports.react); + expect(reactExports.default).toBe(react); + expect(reactExports.useState).toBe(react.useState); + expect(reactExports.createElement).toBe(react.createElement); + + const logExports = await evaluateReexportModule( + map.imports['@deephaven/log'] + ); + expect(logExports.default).toBe(Log); + expect(logExports.Log).toBe(Log); + expect(logExports.logInit).toBe(logInit); + }); + + it('memoizes the result across calls', () => { + const first = buildHostImportMap({ react: {} }); + const second = buildHostImportMap({ '@deephaven/components': {} }); + // Memoized: second call returns the same cached map, ignoring new input. + expect(second).toBe(first); + }); +}); + +describe('buildPluginImportMap', () => { + it('maps each plugin package name to its served entry url', () => { + const map = buildPluginImportMap( + [ + { + name: 'plugin-a', + main: 'index.js', + loader: { package: '@scope/plugin-a' }, + }, + { + name: 'plugin-b', + main: 'dist/index.js', + loader: { package: '@scope/plugin-b' }, + }, + ], + 'http://localhost/js-plugins' + ); + expect(map.imports).toEqual({ + '@scope/plugin-a': 'http://localhost/js-plugins/plugin-a/index.js', + '@scope/plugin-b': 'http://localhost/js-plugins/plugin-b/dist/index.js', + }); + }); + + it('skips plugins without a package field', () => { + const map = buildPluginImportMap( + [ + { name: 'plugin-a', main: 'index.js' }, + { name: 'plugin-b', main: 'index.js', loader: {} }, + ], + 'http://localhost/js-plugins' + ); + expect(map.imports).toEqual({}); + }); +}); + +// These tests share module state (import map entries, shim readiness) and +// intentionally run in order: entries added before es-module-shims loads must +// be applied on load, and later additions go straight to importShim. +describe('es-module-shims integration', () => { + it('defers import map entries until es-module-shims has loaded', async () => { + addImportMap({ imports: { foo: 'blob:1' } }); + expect(mockImportShim.addImportMap).not.toHaveBeenCalled(); + + await ensureEsModuleShims(); + + expect(window.esmsInitOptions).toMatchObject({ shimMode: true }); + expect(window.esmsInitOptions?.source).toEqual(expect.any(Function)); + expect(mockImportShim.addImportMap).toHaveBeenCalledTimes(1); + expect(mockImportShim.addImportMap).toHaveBeenCalledWith({ + imports: { foo: 'blob:1' }, + }); + }); + + it('adds only new specifiers once loaded', () => { + addImportMap({ imports: { foo: 'blob:2', bar: 'blob:3' } }); + expect(mockImportShim.addImportMap).toHaveBeenCalledTimes(1); + expect(mockImportShim.addImportMap).toHaveBeenCalledWith({ + imports: { bar: 'blob:3' }, + }); + }); + + it('does not call importShim when nothing new is added', () => { + addImportMap({ imports: { foo: 'blob:4', bar: 'blob:5' } }); + expect(mockImportShim.addImportMap).not.toHaveBeenCalled(); + }); + + it('imports via importShim and serves the prefetched source once', async () => { + const url = 'http://localhost/p/main.js'; + const source = 'export default { name: "plugin" };'; + const defaultSourceHook = jest + .fn() + .mockResolvedValue({ url, type: 'js', source: 'from network' }); + const sourceHook = window.esmsInitOptions?.source; + if (sourceHook == null) { + throw new Error('source hook not registered'); + } + + mockImportShim.mockResolvedValue({ default: { name: 'plugin' } }); + const result = await loadEsModulePlugin(url, source); + + expect(mockImportShim).toHaveBeenCalledWith(url); + expect(result).toEqual({ default: { name: 'plugin' } }); + + // First request for the entry is served from the prefetched source + expect(await sourceHook(url, {}, '', defaultSourceHook)).toEqual({ + type: 'js', + source, + }); + expect(defaultSourceHook).not.toHaveBeenCalled(); + + // Anything else (and re-requests) go to the default network loader + expect(await sourceHook(url, {}, '', defaultSourceHook)).toEqual({ + url, + type: 'js', + source: 'from network', + }); + expect(defaultSourceHook).toHaveBeenCalledWith(url, {}, ''); + }); + + it('keys prefetched sources by resolved URL', async () => { + const sourceHook = window.esmsInitOptions?.source; + if (sourceHook == null) { + throw new Error('source hook not registered'); + } + mockImportShim.mockResolvedValue({}); + + await loadEsModulePlugin('/plugins/p/main.js', 'export {};'); + + const resolved = new URL('/plugins/p/main.js', document.baseURI).href; + expect(await sourceHook(resolved, {}, '', jest.fn())).toEqual({ + type: 'js', + source: 'export {};', + }); + }); +}); diff --git a/packages/app-utils/src/plugins/esmPluginLoader.ts b/packages/app-utils/src/plugins/esmPluginLoader.ts new file mode 100644 index 0000000000..fa687a827b --- /dev/null +++ b/packages/app-utils/src/plugins/esmPluginLoader.ts @@ -0,0 +1,294 @@ +import Log from '@deephaven/log'; +import { init as initLexer, parse } from 'es-module-lexer'; + +const log = Log.module('@deephaven/app-utils.esmPluginLoader'); + +/** + * Global registry of host singleton modules keyed by their bare specifier (e.g. + * `react`, `@deephaven/components`). The generated blob re-export modules read + * the live host instance from here so plugin imports share the host's + * singletons rather than bundling their own copies. + */ +const SHARED_MODULES_KEY = '__DH_SHARED_PLUGIN_MODULES__'; + +declare global { + interface Window { + [SHARED_MODULES_KEY]?: Record; + } +} + +export interface PluginImportMap { + imports: Record; +} + +/** Reserved words that cannot be used as `export const ` identifiers. */ +const RESERVED_WORDS = new Set([ + 'break', + 'case', + 'catch', + 'class', + 'const', + 'continue', + 'debugger', + 'default', + 'delete', + 'do', + 'else', + 'enum', + 'export', + 'extends', + 'false', + 'finally', + 'for', + 'function', + 'if', + 'import', + 'in', + 'instanceof', + 'new', + 'null', + 'return', + 'super', + 'switch', + 'this', + 'throw', + 'true', + 'try', + 'typeof', + 'var', + 'void', + 'while', + 'with', + 'yield', + 'let', + 'static', + 'await', +]); + +const IDENTIFIER_REGEX = /^[A-Za-z_$][A-Za-z0-9_$]*$/; + +function isValidExportName(name: string): boolean { + return IDENTIFIER_REGEX.test(name) && !RESERVED_WORDS.has(name); +} + +/** + * Memoized host import map. The host singletons never change, so the blob + * re-export modules are created once and reused across calls. + */ +let hostImportMapCache: PluginImportMap | null = null; + +/** + * Plugin entry sources that have already been fetched (for format detection), + * keyed by resolved URL. The es-module-shims source hook serves these so the + * entry is not fetched a second time when it is imported. + */ +const prefetchedSources = new Map(); + +function getSharedModules(): Record { + if (window[SHARED_MODULES_KEY] == null) { + window[SHARED_MODULES_KEY] = {}; + } + return window[SHARED_MODULES_KEY] as Record; +} + +/** + * Create a blob URL for an ES module that re-exports a host singleton module. + * The generated module reads the live instance from the shared module registry + * so plugins and the host share the exact same singleton. + * @param specifier The bare specifier (e.g. `react`) used as the registry key + * @param value The host module value to re-export + * @returns A blob URL referencing the generated ES module + */ +function createHostReexportModule(specifier: string, value: unknown): string { + const namedExports = + value != null && typeof value === 'object' + ? Object.keys(value as Record).filter( + key => key !== 'default' && isValidExportName(key) + ) + : []; + + const registryAccess = `window[${JSON.stringify( + SHARED_MODULES_KEY + )}][${JSON.stringify(specifier)}]`; + + const lines = [ + `const __m = ${registryAccess};`, + // If the module has an explicit default, use it; otherwise the module value + // itself is the default (e.g. the React object for `react`). + 'export default (__m && __m.default !== undefined) ? __m.default : __m;', + ...namedExports.map( + name => `export const ${name} = __m[${JSON.stringify(name)}];` + ), + ]; + + const blob = new Blob([lines.join('\n')], { + type: 'application/javascript', + }); + return URL.createObjectURL(blob); +} + +/** + * Build (and memoize) the import map that maps host bare specifiers to blob + * re-export modules backed by the host's live singletons. + * @param resolve The resolve map of host singletons (from remote-component.config) + * @returns An import map `{ imports: { specifier: blobUrl } }` + */ +export function buildHostImportMap( + resolve: Record +): PluginImportMap { + if (hostImportMapCache != null) { + return hostImportMapCache; + } + + const sharedModules = getSharedModules(); + const imports: Record = {}; + + Object.entries(resolve).forEach(([specifier, value]) => { + sharedModules[specifier] = value; + imports[specifier] = createHostReexportModule(specifier, value); + }); + + hostImportMapCache = { imports }; + return hostImportMapCache; +} + +/** + * Build the import map that maps each plugin's package name to its served entry + * URL, enabling ESM plugins to import other plugins by package name. + * @param plugins The plugin manifest entries + * @param modulePluginsUrl The base URL the plugins are served from + * @returns An import map `{ imports: { packageName: entryUrl } }` + */ +export function buildPluginImportMap( + plugins: readonly { + name: string; + main: string; + loader?: { package?: string }; + }[], + modulePluginsUrl: string +): PluginImportMap { + const imports: Record = {}; + plugins.forEach(({ name, main, loader }) => { + const packageName = loader?.package; + if (packageName != null) { + imports[packageName] = `${modulePluginsUrl}/${name}/${main}`; + } + }); + return { imports }; +} + +/** + * es-module-shims source hook. Serves plugin entries that were already fetched + * for format detection, and defers to the default (network) loader for + * everything else (lazy chunks, cross-plugin imports not yet loaded, ...). + */ +const resolvePluginSource: NonNullable = async ( + url, + fetchOpts, + parent, + defaultSourceHook +) => { + const source = prefetchedSources.get(url); + if (source != null) { + prefetchedSources.delete(url); + return { type: 'js', source }; + } + return defaultSourceHook(url, fetchOpts, parent); +}; + +let esModuleShimsPromise: Promise | null = null; +let isEsModuleShimsReady = false; + +/** + * Import map entries registered for plugins; the first mapping for a specifier + * wins. Kept locally so entries registered before es-module-shims has loaded + * can be applied once it is ready. + */ +const pluginImports: Record = {}; + +/** + * Lazily load and initialize es-module-shims in shim mode. Shim mode only + * processes modules loaded through `importShim`, so the host's own module + * scripts are never touched, import maps can be added at runtime on every + * browser, and our source hook can feed already-fetched plugin entries. + */ +export function ensureEsModuleShims(): Promise { + if (esModuleShimsPromise == null) { + // esmsInitOptions must be set before es-module-shims evaluates + window.esmsInitOptions = { + ...window.esmsInitOptions, + shimMode: true, + source: resolvePluginSource, + }; + esModuleShimsPromise = import('es-module-shims').then(() => { + isEsModuleShimsReady = true; + window.importShim.addImportMap({ imports: pluginImports }); + log.debug('es-module-shims loaded with import map', pluginImports); + }); + } + return esModuleShimsPromise; +} + +/** + * Register an import map so ESM plugins can resolve host singletons and + * cross-plugin package specifiers. Specifiers that are already mapped are left + * untouched. Does not load es-module-shims; entries are applied when it loads. + * @param map The import map to add + */ +export function addImportMap(map: PluginImportMap): void { + const imports: Record = {}; + Object.entries(map.imports).forEach(([specifier, url]) => { + if (!(specifier in pluginImports)) { + pluginImports[specifier] = url; + imports[specifier] = url; + } + }); + if (Object.keys(imports).length === 0) { + return; + } + if (isEsModuleShimsReady) { + window.importShim.addImportMap({ imports }); + log.debug('Added plugin import map', imports); + } +} + +/** + * Load an ES module plugin whose entry source has already been fetched. The + * source is handed to es-module-shims through the source hook so it is not + * fetched again, while relative chunk imports still resolve against `pluginUrl`. + * @param pluginUrl The URL of the ESM plugin entry + * @param source The already-fetched entry source + * @returns The loaded module exports + */ +export async function loadEsModulePlugin( + pluginUrl: string, + source: string +): Promise { + await ensureEsModuleShims(); + prefetchedSources.set(new URL(pluginUrl, document.baseURI).href, source); + return window.importShim(pluginUrl); +} + +let lexerInitPromise: Promise | null = null; + +/** + * Detect whether the provided module source is an ES module by checking for + * static `import`/`export` statements using es-module-lexer. + * @param source The module source text + * @returns True if the source is an ES module + */ +export async function isEsModuleSource(source: string): Promise { + if (lexerInitPromise == null) { + lexerInitPromise = initLexer; + } + await lexerInitPromise; + try { + const [imports, exports] = parse(source); + // `d === -1` denotes a static import statement (vs. a dynamic import()). + const hasStaticImport = imports.some(({ d }) => d === -1); + return exports.length > 0 || hasStaticImport; + } catch (e) { + log.warn('Failed to parse module source for ESM detection', e); + return false; + } +} diff --git a/packages/app-utils/src/plugins/loadCommonJsModule.test.ts b/packages/app-utils/src/plugins/loadCommonJsModule.test.ts new file mode 100644 index 0000000000..d0c9c43731 --- /dev/null +++ b/packages/app-utils/src/plugins/loadCommonJsModule.test.ts @@ -0,0 +1,30 @@ +import loadCommonJsModule from './loadCommonJsModule'; +import { resolve } from './remote-component.config'; + +describe('loadCommonJsModule', () => { + it('evaluates the source and returns module.exports', () => { + expect(loadCommonJsModule('module.exports = { name: "plugin" };')).toEqual({ + name: 'plugin', + }); + }); + + it('supports assigning to exports directly', () => { + expect(loadCommonJsModule('exports.foo = 1; exports.bar = 2;')).toEqual({ + foo: 1, + bar: 2, + }); + }); + + it('resolves require calls against the host resolve map', () => { + const result = loadCommonJsModule( + 'module.exports = require("react");' + ) as unknown; + expect(result).toBe(resolve.react); + }); + + it('throws for unknown requires', () => { + expect(() => + loadCommonJsModule('module.exports = require("not-a-real-module");') + ).toThrow(/Could not require 'not-a-real-module'/); + }); +}); diff --git a/packages/app-utils/src/plugins/loadCommonJsModule.ts b/packages/app-utils/src/plugins/loadCommonJsModule.ts new file mode 100644 index 0000000000..27097357f8 --- /dev/null +++ b/packages/app-utils/src/plugins/loadCommonJsModule.ts @@ -0,0 +1,21 @@ +import { createRequires } from '@paciolan/remote-module-loader'; +import { resolve } from './remote-component.config'; + +const requires = createRequires(resolve); + +/** + * Evaluate an already-fetched CommonJS plugin bundle. `require` calls resolve + * against the host resolve map so the plugin shares the host's singletons. + * @param source The CommonJS module source + * @returns The module's exports + */ +// eslint-disable-next-line @typescript-eslint/no-explicit-any +export function loadCommonJsModule(source: string): any { + const module = { exports: {} }; + // eslint-disable-next-line @typescript-eslint/no-implied-eval, no-new-func + const evaluate = new Function('require', 'module', 'exports', source); + evaluate(requires, module, module.exports); + return module.exports; +} + +export default loadCommonJsModule; diff --git a/packages/app-utils/src/plugins/loadRemoteModule.ts b/packages/app-utils/src/plugins/loadRemoteModule.ts deleted file mode 100644 index 85744d307e..0000000000 --- a/packages/app-utils/src/plugins/loadRemoteModule.ts +++ /dev/null @@ -1,15 +0,0 @@ -import createLoadRemoteModule, { - createRequires, -} from '@paciolan/remote-module-loader'; -import { resolve } from './remote-component.config'; - -const requires = createRequires(resolve); - -export const loadRemoteModule: { - // eslint-disable-next-line @typescript-eslint/no-explicit-any - (url: string): Promise; -} = createLoadRemoteModule({ - requires, -}); - -export default loadRemoteModule; diff --git a/packages/app-utils/src/plugins/remote-component.config.test.ts b/packages/app-utils/src/plugins/remote-component.config.test.ts new file mode 100644 index 0000000000..e4e4658020 --- /dev/null +++ b/packages/app-utils/src/plugins/remote-component.config.test.ts @@ -0,0 +1,19 @@ +import Log, * as LogModule from '@deephaven/log'; +import { esmResolve, resolve } from './remote-component.config'; + +describe('remote-component.config', () => { + it('keeps the default export for CommonJS require', () => { + expect(resolve['@deephaven/log']).toBe(Log); + }); + + it('exposes the full module namespace for ES module imports', () => { + const logModule = esmResolve['@deephaven/log'] as typeof LogModule; + expect(logModule.default).toBe(Log); + expect(logModule.Log).toBe(Log); + expect(logModule.logInit).toBe(LogModule.logInit); + }); + + it('includes every CommonJS resolve entry', () => { + expect(Object.keys(esmResolve).sort()).toEqual(Object.keys(resolve).sort()); + }); +}); diff --git a/packages/app-utils/src/plugins/remote-component.config.ts b/packages/app-utils/src/plugins/remote-component.config.ts index 98805705d8..0af8f33aa5 100644 --- a/packages/app-utils/src/plugins/remote-component.config.ts +++ b/packages/app-utils/src/plugins/remote-component.config.ts @@ -20,11 +20,10 @@ import * as DeephavenJsapiBootstrap from '@deephaven/jsapi-bootstrap'; import * as DeephavenJsapiComponents from '@deephaven/jsapi-components'; import * as DeephavenJsapiUtils from '@deephaven/jsapi-utils'; import * as DeephavenConsole from '@deephaven/console'; -import DeephavenLog from '@deephaven/log'; +import DeephavenLog, * as DeephavenLogModule from '@deephaven/log'; import * as DeephavenReactHooks from '@deephaven/react-hooks'; import * as DeephavenPlugin from '@deephaven/plugin'; -// eslint-disable-next-line import/prefer-default-export export const resolve: Record = { react, 'react-dom': ReactDOM, @@ -46,3 +45,13 @@ export const resolve: Record = { '@deephaven/plugin': DeephavenPlugin, '@deephaven/react-hooks': DeephavenReactHooks, }; + +/** + * Host modules for ES module plugins, as full module namespaces so both default + * and named imports resolve. `resolve` keeps the CommonJS `require` contract, + * where some entries are the default export only (e.g. `@deephaven/log`). + */ +export const esmResolve: Record = { + ...resolve, + '@deephaven/log': DeephavenLogModule, +}; diff --git a/packages/plugin-example/README.md b/packages/plugin-example/README.md new file mode 100644 index 0000000000..2a835a84d9 --- /dev/null +++ b/packages/plugin-example/README.md @@ -0,0 +1,64 @@ +# @deephaven/plugin-example + +A minimal **ES module** Deephaven plugin used to develop and test the modern +remote-plugin loader in [@deephaven/app-utils](../app-utils). It demonstrates: + +- Building a plugin as an ES module (`vite` lib mode, `formats: ['es']`). +- Resolving host singletons (`react`, `@deephaven/*`, ...) through the import map + the host injects at runtime, so the plugin shares the host's instances instead + of bundling its own. +- **Lazy-loading** a heavier component on demand via dynamic `import()`, so it is + split into a separate chunk that loads only when needed. +- **Dynamically imported styling**: the lazy chunk imports a `.scss` file that is + compiled to CSS at build time and injected via JS when the chunk loads. + +This package is dev-only (`"private": true`) and is not published or part of the +monorepo's main build. + +## How it works + +| File | Purpose | +| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| [src/index.ts](src/index.ts) | Plugin entry; default-exports a `WidgetPlugin`. | +| [src/ExampleWidgetView.tsx](src/ExampleWidgetView.tsx) | Eagerly-loaded view; `React.lazy(() => import('./LazyExampleContent'))`. | +| [src/LazyExampleContent.tsx](src/LazyExampleContent.tsx) | Lazy chunk; imports `./LazyExampleContent.scss`. | +| [src/LazyExampleContent.scss](src/LazyExampleContent.scss) | SCSS compiled + injected with the lazy chunk. | +| [vite.config.ts](vite.config.ts) | ESM lib build; externals; `cssCodeSplit` + `vite-plugin-css-injected-by-js` (`relativeCSSInjection`). | +| [serve.js](serve.js) | Tiny static server that serves `dist/` and a generated `manifest.json` with CORS. | + +The plugin's host singletons are declared `external` in +[vite.config.ts](vite.config.ts) so they are emitted as bare `import` statements +(e.g. `import { Button } from '@deephaven/components'`). At runtime the host +resolves those bare specifiers via the import map built from +[remote-component.config.ts](../app-utils/src/plugins/remote-component.config.ts). + +## Local development + +1. Build and serve the plugin (rebuilds on change): + + ```sh + npm run start -w @deephaven/plugin-example + ``` + + This serves on `http://localhost:4100` with: + + - manifest: `http://localhost:4100/manifest.json` + - entry: `http://localhost:4100/@deephaven/plugin-example/index.js` + + You can also run the steps separately: `npm run build:plugin` then + `npm run serve` (set `PORT` to change the port). + +2. Point the host dev server at it by adding to + `packages/code-studio/.env.development.local`: + + ```sh + VITE_JS_PLUGINS_DEV_PORT=4100 + ``` + +3. Start the host app (`npm start`). The host proxies `/js-plugins` to the + plugin server, auto-detects the entry as an ES module, injects the import + map, and loads it via `es-module-shims`. + +To exercise the widget, open an object of type `Example` from a Deephaven +console (or adjust `supportedTypes` in [src/index.ts](src/index.ts) to match an +object type you can create). diff --git a/packages/plugin-example/package.json b/packages/plugin-example/package.json new file mode 100644 index 0000000000..aca5e40d9b --- /dev/null +++ b/packages/plugin-example/package.json @@ -0,0 +1,39 @@ +{ + "name": "@deephaven/plugin-example", + "version": "1.22.0", + "description": "Example ES module Deephaven plugin demonstrating lazy-loaded components and dynamically imported styling", + "private": true, + "author": "Deephaven Data Labs LLC", + "license": "Apache-2.0", + "type": "module", + "main": "dist/index.js", + "module": "dist/index.js", + "files": [ + "dist" + ], + "scripts": { + "build:plugin": "vite build", + "watch": "vite build --watch", + "serve": "node serve.js", + "start": "run-p watch serve" + }, + "dependencies": { + "@deephaven/components": "file:../components", + "@deephaven/icons": "file:../icons", + "@deephaven/jsapi-types": "file:../jsapi-types", + "@deephaven/log": "file:../log", + "@deephaven/plugin": "file:../plugin" + }, + "devDependencies": { + "@vitejs/plugin-react-swc": "^3.7.0", + "react": "^17.0.2", + "react-dom": "^17.0.2", + "sass": "^1.62.1", + "vite": "^5.4.2", + "vite-plugin-css-injected-by-js": "^3.5.2" + }, + "peerDependencies": { + "react": "^17.0.2", + "react-dom": "^17.0.2" + } +} diff --git a/packages/plugin-example/serve.js b/packages/plugin-example/serve.js new file mode 100644 index 0000000000..68e4be5030 --- /dev/null +++ b/packages/plugin-example/serve.js @@ -0,0 +1,110 @@ +// Minimal static dev server for the example ES module plugin. +// +// Serves the built `dist/` directory under `/@deephaven/plugin-example/...` +// and generates a `manifest.json` describing the plugin, mirroring how a real +// plugin server (e.g. deephaven-plugins) exposes plugins. CORS is enabled so +// the host dev server can fetch the plugin entry for ESM/CJS detection. +// +// Point the host at this server by setting in +// packages/code-studio/.env.development.local: +// VITE_JS_PLUGINS_DEV_PORT=4100 +// +// Usage: node serve.js (defaults to port 4100, override with PORT) + +import http from 'node:http'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const dirname = path.dirname(fileURLToPath(import.meta.url)); +const distDir = path.join(dirname, 'dist'); +const port = Number.parseInt(process.env.PORT ?? '4100', 10); + +const PLUGIN_NAME = '@deephaven/plugin-example'; +const PLUGIN_VERSION = '1.22.0'; + +const CONTENT_TYPES = { + '.js': 'text/javascript', + '.mjs': 'text/javascript', + '.css': 'text/css', + '.json': 'application/json', + '.map': 'application/json', + '.svg': 'image/svg+xml', +}; + +/** Build the plugin manifest served at /manifest.json. */ +function buildManifest() { + return { + plugins: [ + { + name: PLUGIN_NAME, + version: PLUGIN_VERSION, + main: 'index.js', + // `loader.package` enables other ESM plugins to import this one by + // package name. + loader: { + package: PLUGIN_NAME, + }, + }, + ], + }; +} + +const server = http.createServer((req, res) => { + res.setHeader('Access-Control-Allow-Origin', '*'); + res.setHeader('Access-Control-Allow-Methods', 'GET, OPTIONS'); + + if (req.method === 'OPTIONS') { + res.writeHead(204); + res.end(); + return; + } + + const url = new URL(req.url ?? '/', `http://localhost:${port}`); + let pathname = decodeURIComponent(url.pathname); + + if (pathname === '/manifest.json') { + const body = JSON.stringify(buildManifest()); + res.writeHead(200, { 'Content-Type': 'application/json' }); + res.end(body); + return; + } + + // Strip the plugin name prefix so `/@deephaven/plugin-example/index.js` + // resolves to `dist/index.js`. + if (pathname.startsWith(`/${PLUGIN_NAME}/`)) { + pathname = pathname.slice(`/${PLUGIN_NAME}`.length); + } + + // Prevent path traversal outside dist. Comparing the relative path rather + // than a string prefix avoids matching sibling directories like `dist-secret`. + const filePath = path.resolve(distDir, path.join('.', pathname)); + const relative = path.relative(distDir, filePath); + if (relative.startsWith('..') || path.isAbsolute(relative)) { + res.writeHead(403); + res.end('Forbidden'); + return; + } + + fs.readFile(filePath, (err, data) => { + if (err) { + res.writeHead(404); + res.end('Not found'); + return; + } + const ext = path.extname(filePath); + res.writeHead(200, { + 'Content-Type': CONTENT_TYPES[ext] ?? 'application/octet-stream', + }); + res.end(data); + }); +}); + +server.listen(port, () => { + // eslint-disable-next-line no-console + console.log( + `Example plugin server running at http://localhost:${port}\n` + + ` manifest: http://localhost:${port}/manifest.json\n` + + ` entry: http://localhost:${port}/${PLUGIN_NAME}/index.js` + ); +}); diff --git a/packages/plugin-example/src/ExampleWidgetView.tsx b/packages/plugin-example/src/ExampleWidgetView.tsx new file mode 100644 index 0000000000..21c6f88bf7 --- /dev/null +++ b/packages/plugin-example/src/ExampleWidgetView.tsx @@ -0,0 +1,48 @@ +import React, { Suspense, useState } from 'react'; +import { Button } from '@deephaven/components'; +import type { WidgetComponentProps } from '@deephaven/plugin'; +import Log from '@deephaven/log'; + +const log = Log.module('@deephaven/plugin-example.ExampleWidgetView'); + +// Lazy-load the heavier view (and its SCSS) on demand. Vite emits this as a +// separate chunk that is only fetched when the user reveals it, demonstrating +// code splitting of a remotely-loaded ES module plugin. The chunk's compiled +// CSS is injected via JS (vite-plugin-css-injected-by-js) when the chunk loads. +const LazyExampleContent = React.lazy(() => import('./LazyExampleContent')); + +/** + * Entry component for the example widget plugin. Renders immediately, then + * lazy-loads the styled content chunk when requested. + */ +function ExampleWidgetView({ + fetch, +}: WidgetComponentProps): React.ReactElement { + const [showContent, setShowContent] = useState(false); + + return ( +
+

Example ES module plugin

+

+ This plugin is a modern ES module loaded remotely. Click below to + lazy-load a styled component chunk on demand. +

+ + {showContent && ( + Loading…
}> + + + )} + + ); +} + +export default ExampleWidgetView; diff --git a/packages/plugin-example/src/LazyExampleContent.scss b/packages/plugin-example/src/LazyExampleContent.scss new file mode 100644 index 0000000000..8fd8693da1 --- /dev/null +++ b/packages/plugin-example/src/LazyExampleContent.scss @@ -0,0 +1,16 @@ +// Styles for the lazy-loaded content chunk. Compiled to CSS at build time and +// injected via JS when the chunk loads, so it ships with the lazy chunk rather +// than the plugin entry. + +.dh-example-plugin-card { + margin-top: 16px; + padding: 16px; + border: 1px solid var(--dh-color-accent); + border-radius: 8px; + background: var(--dh-color-highlight-selected); + + &__title { + margin-top: 0; + color: var(--dh-color-accent); + } +} diff --git a/packages/plugin-example/src/LazyExampleContent.tsx b/packages/plugin-example/src/LazyExampleContent.tsx new file mode 100644 index 0000000000..eb385a76d0 --- /dev/null +++ b/packages/plugin-example/src/LazyExampleContent.tsx @@ -0,0 +1,49 @@ +import React, { useEffect, useState } from 'react'; +import './LazyExampleContent.scss'; + +interface LazyExampleContentProps { + fetch: () => Promise; +} + +/** + * Heavier content for the example plugin, loaded lazily via dynamic import. + * Importing the SCSS here ensures the compiled CSS travels with this chunk and + * is injected only once the chunk is loaded. + */ +function LazyExampleContent({ + fetch, +}: LazyExampleContentProps): React.ReactElement { + const [widgetType, setWidgetType] = useState('loading…'); + + useEffect(() => { + let cancelled = false; + fetch() + .then(widget => { + if (!cancelled) { + setWidgetType( + (widget as { type?: string })?.type ?? 'unknown widget' + ); + } + }) + .catch(() => { + if (!cancelled) { + setWidgetType('failed to fetch widget'); + } + }); + return () => { + cancelled = true; + }; + }, [fetch]); + + return ( +
+

Lazy-loaded content

+

+ This component and its styling were dynamically imported. The widget + type is: {widgetType} +

+
+ ); +} + +export default LazyExampleContent; diff --git a/packages/plugin-example/src/index.ts b/packages/plugin-example/src/index.ts new file mode 100644 index 0000000000..49bbe057f0 --- /dev/null +++ b/packages/plugin-example/src/index.ts @@ -0,0 +1,23 @@ +import { type WidgetPlugin, PluginType } from '@deephaven/plugin'; +import { vsGraph } from '@deephaven/icons'; +import ExampleWidgetView from './ExampleWidgetView'; + +/** + * Example ES module widget plugin. + * + * This plugin is built as a modern ES module (`formats: ['es']`) and resolves + * its host singletons (`react`, `@deephaven/*`, ...) through the import map the + * host injects at runtime. Its view lazy-loads a heavier component (and that + * component's SCSS) on demand via dynamic `import()`, demonstrating code + * splitting of remotely loaded plugins. + */ +const ExamplePlugin: WidgetPlugin = { + name: '@deephaven/plugin-example', + title: 'Example', + type: PluginType.WIDGET_PLUGIN, + supportedTypes: 'Example', + component: ExampleWidgetView, + icon: vsGraph, +}; + +export default ExamplePlugin; diff --git a/packages/plugin-example/src/vite-env.d.ts b/packages/plugin-example/src/vite-env.d.ts new file mode 100644 index 0000000000..b739ec75ff --- /dev/null +++ b/packages/plugin-example/src/vite-env.d.ts @@ -0,0 +1,4 @@ +/// + +declare module '*.scss'; +declare module '*.css'; diff --git a/packages/plugin-example/tsconfig.json b/packages/plugin-example/tsconfig.json new file mode 100644 index 0000000000..f64964f20a --- /dev/null +++ b/packages/plugin-example/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "@deephaven/tsconfig", + "compilerOptions": { + "jsx": "react-jsx", + "module": "ESNext", + "moduleResolution": "Bundler", + "noEmit": true, + "types": ["node"] + }, + "include": ["src", "vite.config.ts"] +} diff --git a/packages/plugin-example/vite.config.ts b/packages/plugin-example/vite.config.ts new file mode 100644 index 0000000000..d2b09ede16 --- /dev/null +++ b/packages/plugin-example/vite.config.ts @@ -0,0 +1,48 @@ +/* eslint-disable import/no-extraneous-dependencies */ +import { defineConfig } from 'vite'; +import react from '@vitejs/plugin-react-swc'; +import cssInjectedByJsPlugin from 'vite-plugin-css-injected-by-js'; + +// The host application provides these singletons at runtime via the import map +// injected by @deephaven/app-utils. They must NOT be bundled into the plugin so +// that the plugin shares the host's instances (one React, one redux store, one +// design system, ...). +const external = [ + 'react', + 'react-dom', + 'redux', + 'react-redux', + '@deephaven/components', + '@deephaven/dashboard', + '@deephaven/icons', + '@deephaven/jsapi-bootstrap', + '@deephaven/jsapi-types', + '@deephaven/log', + '@deephaven/plugin', +]; + +// https://vitejs.dev/config/ +export default defineConfig({ + plugins: [ + react(), + // Inject each chunk's CSS via JS so styles for lazy-loaded chunks are added + // to the document when the chunk is dynamically imported, rather than + // requiring a separate the host would have to know about. + // `relativeCSSInjection` keeps each chunk's CSS with that chunk so the lazy + // chunk's styles only load when the chunk is imported. + cssInjectedByJsPlugin({ relativeCSSInjection: true }), + ], + build: { + // Emit a code-split ES module so dynamic import() in the plugin produces + // separate chunks that load on demand. + cssCodeSplit: true, + lib: { + entry: './src/index.ts', + formats: ['es'], + fileName: () => 'index.js', + }, + rollupOptions: { + external, + }, + }, +}); diff --git a/plans/esm-remote-plugins.md b/plans/esm-remote-plugins.md new file mode 100644 index 0000000000..d557114891 --- /dev/null +++ b/plans/esm-remote-plugins.md @@ -0,0 +1,175 @@ +# ESM remote plugins with lazy loading + +Allow remotely-loaded plugins to be modern ES-module plugins that can +code-split / lazy-load parts of their components (and their CSS/SCSS) instead of +shipping everything in one CommonJS bundle. ESM plugin components must resolve +the same host singletons (`react`, `redux`, `@deephaven/*`, ...) that the +existing CommonJS plugins resolve via the `remote-component.config.ts` `resolve` +map. The existing CommonJS plugin format keeps working — the format is +auto-detected per plugin at load time. + +## Background — how plugins load today + +1. [`PluginsBootstrap`](../packages/app-utils/src/components/PluginsBootstrap.tsx) + calls + [`loadModulePlugins(pluginsUrl)`](../packages/app-utils/src/plugins/PluginUtils.ts). +2. `loadModulePlugins` fetches `manifest.json`, groups plugins into dependency + levels via `groupByDependencyLevel`, and loads each level in parallel. +3. Each plugin is loaded by `loadModulePlugin` → + [`loadRemoteModule`](../packages/app-utils/src/plugins/loadRemoteModule.ts), + which uses `@paciolan/remote-module-loader` to **fetch a single CommonJS + bundle as text and `eval` it** with a custom `require` backed by the + [`resolve`](../packages/app-utils/src/plugins/remote-component.config.ts) map. + Because this is a JS-level shim, the browser module system is never involved, + so there are no import-map timing constraints. +4. `processLoadedModule` (in `@deephaven/plugin`) registers each plugin's raw + exports in the `resolve` map under its `package` name so later plugins can + import earlier ones, then registers the plugin in the plugin map. + +The plugin build (deephaven-plugins repo) uses Vite library mode with +`formats: ['cjs']` and `external: ['react', '@deephaven/components', ...]` so the +deps become `require(...)` calls satisfied by the `resolve` map. + +## Why ESM needs more than "just import it" + +Native `import 'react'` / `import('./Chunk.js')` are resolved by the **browser**, +which rejects bare specifiers (`react`, `@deephaven/components`) unless an +**import map** maps them. Our import map can only be built **at runtime** because: + +- the host singletons live inside the app bundle (we expose them as blob ESM + modules), and +- cross-plugin URLs come from the manifest. + +By the time we load plugins, the app's own module entry has already executed. +Per [MDN](https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script/type/importmap#merging_multiple_import_maps), +an import map must be declared before any module that depends on it loads, and +late / multiple import maps only work in Chrome 133+ and Safari 18.4+ — +**Firefox does not support multiple import maps** +([caniuse](https://caniuse.com/mdn-html_elements_script_type_importmap_multiple)). +web-client-ui supports Firefox, so a native-only approach is not reliable. + +### Resolution: es-module-shims in shim mode + +We use [`es-module-shims`](https://github.com/guybedford/es-module-shims) in +**shim mode**. Shim mode only processes modules loaded through the explicit +`importShim()` API, so the host's own module scripts are never touched, and +import maps can be added at runtime on every browser (including Firefox) via +`importShim.addImportMap()`. Every plugin module therefore goes through the +shim loader — specifiers are resolved and rewritten by es-module-shims rather +than the native loader — which is what makes late, incremental import maps +reliable across all supported browsers. + +## Plan + +### Phase 1 — Resolution infrastructure (`packages/app-utils/src/plugins`) + +- Add `es-module-shims` dependency, initialized with `shimMode: true`. +- New `esmPluginLoader.ts`: + - `buildHostImportMap(resolve)` — for each `resolve` entry, create a blob ESM + module that re-exports the live host instance (`default` plus named keys + enumerated via `Object.keys`), returning + `{ imports: { react: blobUrl, '@deephaven/components': blobUrl, ... } }`. + - `buildPluginImportMap(manifest, baseUrl)` — map each plugin's + `loader.package` to its served entry URL so cross-plugin imports resolve up + front. + - `addImportMap(map)` — register the map with `importShim.addImportMap()` + before importing the first plugin module. + - `loadEsModulePlugin(url)` — `importShim(url)`. +- Leave `loadRemoteModule.ts` (CJS path) untouched. + +### Phase 2 — Loader integration (`PluginUtils.ts`) + +- `loadModulePlugin`: fetch the entry source once and detect ESM vs CJS using + `es-module-lexer` (presence of `import`/`export` statements ⇒ ESM). Route ESM + to `loadEsModulePlugin`, CJS to the existing `loadRemoteModule`. +- `loadModulePlugins`: build the host import map (once) and the plugin-package + map, then inject both before the existing level-by-level load loop. +- `processLoadedModule` is unchanged — it operates on the resolved exports. + +### Phase 3 — Example plugin + dev server (`packages/plugin-example`, dev-only) + +- Vite library build with `formats: ['es']`, the same `external` list as today, + `cssCodeSplit`, and `vite-plugin-css-injected-by-js` so a lazy chunk's compiled + CSS injects automatically when the chunk is dynamically imported. +- A widget plugin whose component is lazy-loaded via dynamic `import()`, and the + lazy chunk imports a `.scss` file — proving both code-splitting and dynamic + SCSS→CSS styling loading with the chunk. +- A tiny static dev server (`sirv` / `http-server`) serving `dist/` plus a + generated `manifest.json` with CORS on port 4100, exposed via a + `plugin-example` script. Set `VITE_JS_PLUGINS_DEV_PORT=4100` to proxy + `/js-plugins` to it (existing `vite.config.ts` proxy). + +### Phase 4 — Tests, docs & migration + +- Unit tests in + [`PluginUtils.test.ts`](../packages/app-utils/src/plugins/PluginUtils.test.ts) + for ESM detection / loading (mock dynamic import + `es-module-lexer`) and tests + for the host import-map builder. +- Optional e2e: load the example plugin, assert the lazy JS chunk and its CSS are + fetched on demand and the widget renders styled. +- README: authoring ESM plugins (`formats: ['es']`, css-injected-by-js, lazy + `import()`) and serving locally. +- Migration doc for the `deephaven-plugins` widget template (a **separate future + commit**): switch Vite `formats` `cjs`→`es`, add css-injected-by-js, keep + externals. CJS stays supported, so teams migrate when ready. + +## Relevant files + +- `packages/app-utils/src/plugins/PluginUtils.ts` — detect + branch in + `loadModulePlugin`, inject import maps in `loadModulePlugins`. +- `packages/app-utils/src/plugins/esmPluginLoader.ts` (new) — blob host modules, + import-map builders, ESM loader. +- `packages/app-utils/src/plugins/loadRemoteModule.ts` — CJS path, unchanged. +- `packages/app-utils/src/plugins/remote-component.config.ts` — `resolve` + singletons source. +- `packages/app-utils/src/declarations.d.ts` — declares the `es-module-shims` + side-effect module for the lazy dynamic import. +- `packages/plugin-example/**` (new) — example ESM plugin + static dev server. +- `packages/app-utils/src/plugins/PluginUtils.test.ts` — new ESM cases. + +## Verification + +1. `npm run plugin-example` + `VITE_JS_PLUGINS_DEV_PORT=4100 npm start`; + confirm the example widget loads, the lazy chunk and its CSS fetch only on + demand (Network tab), styles apply, and host React is shared (no + duplicate-React errors). +2. An existing CJS plugin still loads unchanged (auto-detect path). +3. `npm run test:unit -- PluginUtils` passes the new ESM cases. +4. Cross-browser smoke — Chromium and Firefox — both load the ESM plugin and + its lazy CSS through the shim loader. + +## Implementation notes / deviations + +The following reflect how this was actually implemented (the rest of the plan +holds): + +- **es-module-shims is loaded lazily, not via `index.html`.** Rather than adding + a `