Repository navigation
feat(models): support extension-scoped shared weight groups - #348
lightningpixel merged 4 commits into
Conversation
Reserve physical roots across downloads, cancellation and removal; preserve paused targets and require confirmed runtime shutdown before deleting files. Stop install-all on interrupted work and refresh dependent readiness after partial success. Fix live model paths and explicit generator identity, reject portable node aliases, and exercise the regressions on Windows/Linux with Python 3.11 and 3.12.
|
Is this possible with different venv's though? Each venv has its own dependencies that are weight based for calculations in many cases. I would LOVE if this would work as it would reduce some memory and space on my laptop. Very curious how this plays out @DrHepa |
|
Thanks, @iammojogo-sudo! Sharing checkpoint files doesn’t mean sharing a Python environment. This PR only reuses weights within one extension, whose nodes already share a venv. I checked our extensions: SD15/SDXL have matching dependency versions, but others need different Torch/CUDA or Transformers versions, so one shared venv isn’t a safe default. I’ve opened a separate proposal to keep venvs isolated while reusing verified package files where possible. That could save disk space—not RAM/VRAM. |
|
" so one shared venv isn’t a safe default" exactly! That's why I was wondering. But it is good that "keep venvs isolated while reusing verified package files" because that right there could save a lot of re-downloading python dependencies at the least. Good stuff! If it can be made internal (in the .asar), that would be helpful for extension creators too, so that they don't have to code a shared communication across extensions. Could be a big deal! I love this. |
Resolve conflicts with the scene artifacts PR (lightningpixel#357): - generator_registry: keep both the weight-group and scene-shape manifest validation; move the shared-weight readiness check into get_ready_generator; recompute shared_model_dirs inside the lifecycle lock in update_paths; unload_all unloads every model before reporting the ones that stayed loaded - extension-install-utils / ipc-handlers: apply the scene-shape check, then the weight-group validation - test_generator_registry: keep the tests from both sides
- Treat an unreachable backend (ECONNREFUSED) as unloaded when removing weights: no Python process can hold the files, and failing here blocked deleting weights and uninstalling extensions while the backend was down - Removing an extension's weights unloads only that extension's models instead of calling /model/unload-all - Compare shared model dirs with str(Path(...)) so the extension process test passes on Windows - Add regression tests for both removal cases and for unload_all
Resolve conflicts with the scene artifacts (#357) and shared weight groups (#348) work, and address the review: Conflict resolution - Keep weight_groups and weight_variants side by side in the manifest validation (Python registry, model-sources, download plan, install utils, IPC listing), the preload API, the shared types and the Models UI - Rebuild model-sources.ts and the interleaved tests from dev, re-adding the variant code and tests unchanged - Download: keep dev's leased, target-aware flow and run the variant passes (shared files first, then the variant) in the legacy branch - ModelsPage: keep dev's install queue and shared groups, pass the variant id through, and use installedNodeIds everywhere Review fixes - Check the job's model for a missing weight variant (assert_weight_variant_installed(params, model_id)): pinned jobs only switch to their model once they run, so the active model is not a stand-in - deleteWeightVariant goes through the weight lease with a confirmed unload, and lists the variant files only once the node root is reserved - Picking a missing variant no longer navigates away from the graph or the Generate panel; an "install it" link is shown instead - Reject weight_variants combined with weight_groups with an explicit message - Disable other variant installs while one variant of the node downloads - README: undeclared repository variants are fetched by the shared pass
3 participants
Summary
This PR implements extension-scoped shared weight groups for model extensions.
Multiple model nodes within the same extension can now reuse one or more physical weight groups while retaining their own capability identity and optional node-private weights.
This supports both primary use cases described in #343:
Closes #343.
Motivation
Modly currently treats each model node as both a capability and the physical owner of its weights:
///
This causes every node to download its own copy of the same checkpoint, even when several nodes within an extension use an identical base model.
A simple node-owner alias is insufficient because some nodes need to combine a shared base checkpoint with additional private weights. It would also couple runtime identity and deletion behavior to a physical directory.
This PR separates:
Manifest contract
Model extensions can declare top-level "weight_groups". Each group owns its own validated "model_sources" plan.
Nodes reference the shared groups they require and may continue declaring private "model_sources".
Example:
{
"id": "pixal3d",
"type": "model",
"weight_groups": [
{
"id": "pixal3d-base",
"model_sources": [
{
"id": "pixal3d",
"provider": "huggingface",
"repo_id": "TencentARC/Pixal3D",
"revision": "",
"destination": ".",
"checks": [
"pipeline.json"
]
}
]
}
],
"nodes": [
{
"id": "generate",
"weight_groups": [
"pixal3d-base"
]
},
{
"id": "worldsculpt",
"weight_groups": [
"pixal3d-base"
],
"model_sources": [
{
"id": "worldsculpt-adapter",
"provider": "huggingface",
"repo_id": "AlayaLab/WorldSculpt",
"revision": "",
"destination": "worldsculpt",
"checks": [
"worldsculpt/"
]
}
]
}
]
}
Storage layout
Shared groups are stored independently from node-private weights:
/
/
_shared/
/
...
/
...
Rules enforced by the implementation:
Download behavior
Installing a node builds an effective download plan containing:
The download lifecycle now:
Readiness behavior
A node is ready only when:
This allows states such as:
Runtime contract
Capability identity is now explicitly separated from storage identity.
The runtime no longer depends on "MODEL_DIR.name" to determine which node is running.
Direct and subprocess extensions receive:
MODEL_ID=/
MODEL_NODE_ID=
MODEL_DIR=
SHARED_MODEL_DIRS={"":""}
The Python generator context also exposes the resolved "shared_model_dirs" mapping.
"MODEL_DIR" keeps its existing node-private meaning for backward compatibility.
When the configured model storage path changes, both private and shared directories are resolved again from the host configuration.
Deletion and uninstall behavior
Shared groups have an independent lifecycle:
UI changes
The Models UI and extension drawer now expose:
Manifest validation
The Electron and Python implementations apply equivalent normalization and validation rules.
Validation rejects:
Backward compatibility
Extensions that do not declare "weight_groups" retain their existing behavior and directory layout.
The following remain node-private and fully supported:
This PR does not migrate or deduplicate existing downloaded files automatically. Extension authors can provide migration guidance when adopting the new layout.
Test coverage
Regression coverage includes:
Validation results
Out of scope
This first phase intentionally does not add:
Cross-extension sharing can be designed separately once ownership, versioning, trust, reference counting and garbage-collection semantics have been defined.
Pixal3D consumer integration evidence
The current Pixal3D extension manifest was validated directly with this PR's Python normalizer. Its five groups (
pixal3d-base,pixal3d-mv,da3-base,sam3, andworldsculpt-adapters) all normalize successfully, every group is referenced, and no node mixes a shared-group reference with legacyhf_repo.An isolated integration merge with scene PR #357 and video PR #358 exposed three overlapping validation conflicts in
generator_registry.py,extension-install-utils.ts, andipc-handlers.ts. The combined resolution retains both artifact-type validation and node-specific shared-weight projection. It passed the exact six-node Pixal3D registry contract, 100 focused Python tests, 9 focused Node suites, TypeScript--noEmit, and the Electron/Vite production build. No shared-weight manifest change was required.