Skip to content

feat(models): install and remove weight variants per node - #353

Merged
lightningpixel merged 3 commits into
devfrom
feat/weight-variants
Oct 2, 2026
Merged

lightningpixel merged 3 commits into
devfrom
feat/weight-variants

Conversation

@Lorchie

@Lorchie Lorchie commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

A model node that publishes the same weights in several variants (quantizations, precisions) can declare weight_variants next to hf_repo. Each variant is downloaded and deleted on its own from the Extensions drawer, and the params_schema select that picks one labels the variants missing from disk.

Manifest

"params_schema": [
  { "id": "quant", "type": "select", "default": "Q5_K_M",
    "options": [{ "value": "Q4_K_M" }, { "value": "Q5_K_M" }] }
],
"weight_variants": {
  "param": "quant",
  "default": "Q5_K_M",
  "options": [
    { "id": "Q4_K_M", "size_gb": 2.4, "include_prefixes": ["dit/model_Q4_K_M.gguf"], "checks": ["dit/model_Q4_K_M.gguf"] },
    { "id": "Q5_K_M", "include_prefixes": ["dit/model_Q5_K_M.gguf"], "checks": ["dit/model_Q5_K_M.gguf"] }
  ]
}

Validation is mirrored in electron/main/model-sources.ts and api/services/model_sources.py: safe relative paths, no overlap between two variants, download_check outside every variant, no combination with model_sources, and the selecting param must exist (node or extension params_schema) and offer every variant id.

Behaviour

  • Every install downloads the shared files first (all variants excluded), then the requested variant — the default one when none is asked for. Files already complete on disk are skipped, so adding a second variant only fetches its own weights.
  • New IPC: model:installedWeightVariants (returns null when the install state cannot be read, so the renderer never mistakes it for "none installed") and model:deleteWeightVariant, which unloads the model first and removes only that variant's files, .part included.
  • A node counts as installed once its download_check and at least one variant are present.
  • Generation refuses a variant that is not installed (GeneratorRegistry.assert_weight_variant_installed, called at the top of _run_generation, so workflows and the agent are both covered). Picking a missing variant in a node opens the extension drawer.

Tests

npm run test:py (115 tests, OK) and npm run test:node (37 + 142, 0 fail) pass; npx eslint . is clean. tsc -b --noEmit reports the 29 errors already present on dev, none in the files touched here.

Not covered: a real Hugging Face download of a variant has not been exercised end to end — the variant files were faked on disk for the UI, deletion and generation-guard runs.

Lorchie and others added 3 commits September 18, 2026 10:55
A model node that publishes the same weights in several variants
(quantizations, precisions) can declare `weight_variants` next to
`hf_repo`. Each variant is downloaded and deleted on its own from the
Extensions drawer, and the params_schema select that picks one labels the
variants that are missing from disk.

- manifest validation mirrored in electron/main/model-sources.ts and
  api/services/model_sources.py: safe relative paths, no overlap between
  two variants, download_check outside every variant, no combination with
  model_sources, and the selecting param must exist and offer every id.
- every install downloads the shared files first (all variants excluded),
  then the requested variant, or the default one; files already complete
  on disk are skipped, so a second variant only fetches its own weights.
- new IPC: model:installedWeightVariants (null when unreadable) and
  model:deleteWeightVariant, which unloads the model before removal and
  touches only the files of that variant.
- generation refuses a variant that is not installed
  (GeneratorRegistry.assert_weight_variant_installed) and picking one in a
  node opens the extension drawer.
Resolve conflicts with the scene artifacts (#357) and shared weight groups
(#348) work, and address the review:

Conflict resolution
- Keep weight_groups and weight_variants side by side in the manifest
  validation (Python registry, model-sources, download plan, install utils,
  IPC listing), the preload API, the shared types and the Models UI
- Rebuild model-sources.ts and the interleaved tests from dev, re-adding the
  variant code and tests unchanged
- Download: keep dev's leased, target-aware flow and run the variant passes
  (shared files first, then the variant) in the legacy branch
- ModelsPage: keep dev's install queue and shared groups, pass the variant id
  through, and use installedNodeIds everywhere

Review fixes
- Check the job's model for a missing weight variant
  (assert_weight_variant_installed(params, model_id)): pinned jobs only switch
  to their model once they run, so the active model is not a stand-in
- deleteWeightVariant goes through the weight lease with a confirmed unload,
  and lists the variant files only once the node root is reserved
- Picking a missing variant no longer navigates away from the graph or the
  Generate panel; an "install it" link is shown instead
- Reject weight_variants combined with weight_groups with an explicit message
- Disable other variant installs while one variant of the node downloads
- README: undeclared repository variants are fetched by the shared pass
The capability destination is built from EXTENSIONS_DIR.resolve() (long
form) but was compared to os.path.abspath(ext_dir), which keeps the
configured form. With an 8.3 short EXTENSIONS_DIR (e.g. GitHub's Windows
runners: C:\Users\RUNNER~1\...) the same folder never matched, so an
authorized extension update was reported as an interrupted installation.

Normalize the extension folder the same way (resolved parent + name),
without resolving the folder itself, and cover it with a Windows test that
reruns the authorization through a short path.
@lightningpixel
lightningpixel merged commit 69e1014 into dev Oct 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants