feat(import): import an existing Windows-app installation

[jimisola]

Note

Status. The first review validated the ConvNeXt import path against a
real installation. Its one finding — OpenAI-mode models imported as
retrainable shells because this app had no per-model OpenAI mode — is now
implemented as requested: openai is a first-class model mode with the
API config at model level. A scripted end-to-end run against the real 1.3.8 installation on this
machine — including a genuine OpenAI-mode model — passed 56/56 checks;
details at the bottom.

Refs #98 — plus the follow-up work requested in the
first review.

What this does

Reads an existing WinForms ("AI Brass Sorter") installation and copies across
whatever the user ticks, so moving off the Windows app isn't a rebuild-by-hand.

Two ways in, one dialog:

• A once-ever first-run offer, shown only on a launch where an installation
  is actually found and this app still looks unused. A user who never ran the
  Windows app never sees anything.
• Settings → Import from Windows, permanently, for a custom install
  location, for someone who declined the offer, or for a second machine's data
  later.

Before — Settings	After — Settings

The dialog, with the per-item counts a survey of the install produced:

OpenAI models are now first-class (from the review)

The review's screenshots pinned down the Windows model: "OpenAI API" is a
Training Mode, a peer of the ConvNeXt sizes; several OpenAI models can
coexist (different cartridges, prompts, headstamp lists, providers) and the
endpoint/key/model/prompt live on the model. This app now mirrors that:

• model_mode = "openai" is a mode a row can carry — offered in
  Create/Edit Model, accepted by the repo and the schema (widening the
  models.model_mode CHECK on an existing database is a proper table
  rebuild, foreign-keys-off, since the children cascade on models).
• Runtime: an active openai model classifies over HTTP with its own
  ai_model_config — never the app-level AI Config (deliberately: leaking
  the app-level endpoint in would be the mirror image of the silent HTTP
  fallback this app already removed). No PyTorch is needed or ever prompted
  for; checkpoint_problem waves it through — there is no checkpoint to
  miss.
• UI: the Train/AI Config mode pair gains its third state. An active
  openai model leaves AI Config live, editing that model's settings — a
  caption above the form says whose settings they are, and the form rebinds
  only when the target actually changes, so a half-typed endpoint survives
  unrelated refreshes. Train shows an explainer naming the mode. Headstamps
  are the model's own, in the headstamps table like any model's.
• Import: legacy ModelMode 2 maps straight to openai — config,
  headstamps and all. The shell fallback and its warning are gone, and a row
  first imported as a shell by the previous revision self-heals on
  re-import.

Side by side with the Windows app (left: review screenshots; right: this PR):

Windows	This app

The model library with an openai model active, and Train's explainer for it:

Models	Train

(The blank checked Train button in that last shot is a pre-existing sidebar
inking defect — reproduced on main with a community model — tracked
separately, not introduced here.)

One deliberate boundary: SUPPORTED_MODEL_MODES keeps only the trainable
ConvNeXt backbones (the Train page assigns a mode straight into
training_config.model_name); MODEL_MODES = backbones + openai is what
the repo and the editor accept.

Answering the issue's open questions

1. Where does the Windows app store its data? In its own install
directory — %ProgramFiles%\SJSeth\AI Brass Sorter — not in a per-user folder:

<root>/Data/ConfigDB.sjdb.json      the whole database, one BOM-prefixed JSON document
<root>/Data/Settings.json           three app-level toggles, incl. the baud rate
<root>/training/images/<model id>/  training images, {label}__{ticks}.jpg
<root>/training/models/<model id>.zip   the trained checkpoint

ConfigDB.sjdb.json's top-level keys are the tables: Models, Headstamps,
Cartridges, HeadStampParents, HeadStampParentLinks, SlotConfigs,
SortingTemplates, CommunityNotes, plus a Defaults blob holding the
app-level settings (serial port, slot quantity, InitSettings, the IP_*
image-processing values, DefaultModelId, camera selection).

2. Copy or reference the images? Copy. The source installation is never
written to, so it keeps working. Re-copies are skipped by name and size, so a
second run is cheap rather than doubling disk again.

3. Re-import behaviour. Update in place. A community UID match wins first
(the same shared model is the same model, wherever it came from); failing that,
a per-install-root map of legacy-id → local-id recorded by an earlier run. Slot
assignments and sorting templates survive because the row keeps its id — the
same property import_model already relies on.

4. ML.NET-only models. Imported as a shell, not skipped. The images are the
expensive thing to recreate and they come across; the model just has no
checkpoint, which the existing NoLocalCheckpointError path already explains,
and Train is where the user picks it up. The dialog says which models this hit.

5. Is first-run the right moment? Both — offered once at first run, and
permanently available in Settings. Declining costs nothing.

The registry question

Nothing there is worth importing; the filesystem is the whole story. What
exists on a 1.3.8 install:

Key	Contents
HKCU\Software\AICaseSorter	Empty — no values, no subkeys
HKCU\Software\SJSeth\AI Brass Sorter\{A83B627C-…}	One value, DesktopFolder — an MSI installer artifact
HKLM\…\Uninstall\{C8B0E86E-…}	DisplayName, DisplayVersion — and InstallLocation is blank

So the registry can't even be used to locate a non-default install, which is
why discovery is a %ProgramFiles% probe plus "choose the folder yourself".
The MSI component database does contain the install path, but only spread
across ~45,000 per-file component keys — not something to depend on.

Two format findings worth reviewing

Legacy Models rows are already the shape we parse. They use the same
PascalCase spelling as an export ZIP's ModelInfo, so they go straight to
model_io.model_from_export_dict — enum-int ModelMode/ModelType mapping
included — rather than getting a second parser. ModelType 1/2 therefore lands
as ReadOnly/CommunityManaged, so a community model stays non-trainable here
exactly as a download would.

training/models/<id>.zip is not a ZIP of anything — it's a raw torch.save
archive (which is itself a zip container), so it copies straight to
<id>.pth. That matters because the older ML.NET pipeline writes its models
beside it under the same extension (TransformerChain/ wrapping a frozen
TensorFlow graph). The checkpoint is classified by looking inside before it is
copied — */data.pkl = torch, TransformerChain/ = ML.NET — so an ML.NET model
can never be installed as a PyTorch checkpoint.

Deliberately narrow

• Defaults.IP_* maps to image_proc.linescan only. Those values belong to
  the legacy line-scan pipeline; there is no Hough stage over there, so writing
  its numbers into our Hough parameters would silently detune a working crop.
  Hough and strategy are left alone.
• Camera settings are not imported. The legacy app stores a DirectShow
  moniker and a device name; we store an index. Mapping one to the other is a
  guess that would point the app at the wrong camera.
• The import never takes over an active model you already chose here. It
  only adopts the legacy DefaultModelId when nothing is active yet.
• Serial InitSettings keys we don't recognise are dropped — an unknown key
  would be written straight to the board on connect.

Still-open questions for @sjseth

Things one installation couldn't show me:

1. SlotConfigs[].ParentConfig — what is in it? It is [] in every row on
   this machine, even with parent classifications defined, so I can't confirm
   the entry shape. The parser accepts a bare string, {"Name": …} and a
   {"Parent": {"Name": …}} wrapper. Which is right?
2. Package-mode slot rows. Same problem: PackageMode: true rows exist but
   are all empty here. I'm assuming Config carries the same
   {"Headstamp": {"Name": …}} entries as standard mode. Correct?
3. SortingTemplates is empty on this install, so templates are not
   imported at all right now. Is the row shape stable enough to add, and does
   SlotConfigs[].TemplateId reference it?
4. Is the install location configurable, and if so is it recorded anywhere
   readable? Right now a non-default install is only reachable by the user
   pointing at the folder.
5. ModelMode 0 and 1 (DeepLearning/ResNet50 and Inception) currently
   fall back to convnext_tiny in model_io. For an import that fallback is
   also what the model row will claim it is. Is there a better answer than
   "treat it as an untrained shell"?
6. training/images/emulation/ (56 DATA__<ticks>.jpg files) and
   training/images/tuning/ — I've assumed these are app scratch space, not
   user data, and skipped them. Right call?

Testing

• tests/unit/data/test_winforms_import.py — 32 tests: discovery, survey
  counts, ML.NET and OpenAI-mode handling, ownership, per-model SAVEPOINT
  isolation, headstamps/parents/slots, package-mode slots, idempotent
  re-import, each settings block, source-tree immutability, the first-run gate.
• tests/unit/ui/test_winforms_import_dialog.py — 13 tests for the dialog.
• New for the openai mode: classifier routing (an active openai model calls
  the HTTP client with its own config; uses_local_inference False;
  checkpoint_problem None), the schema rebuild against a hand-written
  pre-openai database (proving the headstamp cascade does not fire),
  rebuild idempotence, MODEL_MODES acceptance, the AI Config page's
  per-model targeting (bind/save/retain-unsaved-edits/rebind), the mode
  pair's third state, Train's openai explainer, and the editor offering and
  persisting openai.
• Full suite green locally (1470 passed), ruff and ty clean, CI green.

Real-installation validation

A scripted run against the actual C:\Program Files\SJSeth\AI Brass Sorter
(1.3.8) on this machine, importing into a throwaway data root — the install
itself is only ever read. All 56 checks passed:

• Survey: 5 models, 62 images, 290 headstamps, no warnings; serial and
  image-processing detected.
• Import (0.9 s): 5 models, 62 images, 290 headstamps, 9 parents with
  their 26 links, 1 slot assignment, 1 torch checkpoint (198,823,697 bytes,
  byte-identical at the destination). The genuine OpenAI-mode model
  ("OpenAPI Test", legacy ModelMode 2) landed as Standard/openai,
  not trainable, and — being the legacy install's DefaultModelId — became
  the active model here, with uses_local_inference False and
  checkpoint_problem None. The community model kept its UID and stayed
  ReadOnly/convnext_small.
• Re-import (0.3 s): 0 imported / 5 updated, 0 images re-copied, slot
  assignments intact, library size unchanged. (The run also caught that
  checkpoints were re-copied unconditionally — fixed in 428764b with the
  same size-skip rule the images use.)
• Source untouched: 134 files under Data\ and training\
  snapshotted before and after two full runs — 0 added, 0 removed,
  0 changed.
• Settings: COM3, all 14 board init values, and the line-scan block
  landed exactly; Hough untouched. ai_config_imported was correctly False
  — every AIModelConfig blob in this install is null (the OpenAI API
  Configuration screen was never saved), and the importer reported that
  honestly instead of claiming success.

🤖 Generated with Claude Code

[github-actions]

Docs PDF for 17b368a — download

_{Rendered from this PR's docs. Kept up to date on each push; expires after 14 days.}

[sjseth]

I tested but was unable to import due to my source containing an openai model. The error in the screenshot happens whether or not AI Config is selected. I believe it relates to the Models import only as my windows install contains an openai model.

[jimisola]

Reproduced and fixed in 6ea3968 — thanks, that was a real bug and your install had exactly the row that finds it.

What happened. The legacy ModelMode enum value 2 (OpenAI) was mapped to a literal "openai", which isn't a mode ModelRepo accepts, so creating that one row raised — and since every model was imported inside one transaction, the whole thing rolled back. 15 models and 7,134 images imported as nothing. It made no difference whether AI Config was ticked because the failure is on the Models step, before any of that runs.

It's been latent in model_io since long before this PR; import_model clamps an unsupported mode to convnext_tiny, so the ZIP path never showed it. This import calls model_from_export_dict directly and is the first caller without that clamp.

What changed.

• ModelMode 2 now maps to convnext_tiny, like the other non-ConvNeXt legacy modes already did. Classifying over HTTP is AI Config mode here — the absence of an active model — so there's no model-row spelling of it to preserve. Your OpenAI model imports as a shell with its headstamps and images, ready to retrain locally. A test now asserts every entry in that map is a mode this app actually accepts, so it can't come back.
• The import says so, instead of silently turning it into a ConvNeXt: the survey warns which model this applied to and points at the AI Config tick, mirroring the existing ML.NET warning.
• AI Config now prefers the model that was actually classifying over HTTP. It used to take the first model carrying any non-empty AIModelConfig; since the legacy app writes that blob on every model and most are blank, ticking AI Config could import an empty endpoint over the one you'd configured and still report success. An all-blank blob is no longer treated as a config at all.
• One bad row no longer costs the install. Each model gets its own SAVEPOINT, so a model this app refuses is skipped with a warning in the completion summary and everything else still lands.

Guide updated for all of it. Worth a re-test against your install when you get a chance — I'd expect the 15 models, the images and your endpoint settings to come across now, with a warning naming the OpenAI model.

One thing I could not verify from here: whether your install's other 14 models hold anything else this app refuses. If the import still stops, the completion box will now name the model and the reason rather than showing a bare error, so that message would be the useful thing to paste.

[sjseth]

Ok, your latest changes have mostly resolved the import issue. I was able to pull all the models and settings for the convnext models, and validated that they were editable, functional once pulled in however the openai model did not come over correctly.

In windows the model is a loose term for a specific OpenAI API configuration with its own headstamps so there isn't really a model file behind the scenes but just a configuration.

One may have several openai models with different prompts, headstamp configurations (223, 9mm, etc) and different providers (openai, anthropic, local llm)

Recommend adding openai as a model type in the create model option. The OpenAI API config is at the model level and not a global config

[jimisola]

@sjseth Implemented as you recommended — openai is now a model mode, with the API config at model level (f227588). Thanks for the screenshots; they pinned the semantics down exactly.

What that means here:

• Create/Edit Model offers openai alongside the ConvNeXt sizes, mirroring your Training Mode dropdown. Several OpenAI models can coexist, each with its own cartridge, prompt, headstamp list and provider.
• The config lives on the model row (AIModelConfig, which the schema already carried per model). While an OpenAI model is active, the AI Config page edits that model's settings — a caption above the form says so by name — and its headstamps are ordinary model-scoped headstamps. This page is the equivalent of your per-model "OpenAI API Configuration" screen.
• Runtime: an active OpenAI model classifies over HTTP with its own config, needs no PyTorch, and is never blocked on a missing checkpoint. It is not trainable, whatever its ownership.
• The import now brings a ModelMode 2 row across faithfully — mode, config and headstamps — instead of the ConvNeXt-shell fallback you hit. A model already imported as a shell by the previous revision self-heals on re-import.

Side by side (left: your Windows shots; right: this branch):

Windows	This app

The model library with an OpenAI model active, and Train's explainer for it:

Models	Train

Validated end-to-end against the real 1.3.8 install on this machine, which now contains a genuine OpenAI-mode model ("OpenAPI Test", and it is the install's DefaultModelId): 56/56 scripted checks passed — all 5 models, 62 images, 290 headstamps, the 198 MB torch checkpoint byte-identical, the OpenAI model landing as openai/active with local inference correctly off, idempotent re-run, and the source install byte-for-byte untouched across two full runs. One thing the run surfaced and 428764b fixed: checkpoints were re-copied on every re-import; they now skip like the images do.

One detail from that install worth knowing: every model's AIModelConfig was null there — the app only writes the blob once the OpenAI API Configuration screen is saved — so the importer treats a null blob as "nothing to bring" and says so rather than claiming success. Your install's filled-in configs will come across on the same path.

Happy to hear whether this matches what you had in mind — a re-test of the import on your side would exercise the filled-config path mine can't.

[jimisola]

Thanks — both good calls. Pushed in e2fe3fb: the picker is now a tree, one branch per model.

Before	After

Same staged 5-model install in both shots.

Picking models

Every model is its own tick, with Training images, Headstamps and slot assignments and Trained model file underneath it. Select all / Select none are there so two out of fifteen is two clicks rather than thirteen, and the line under the tree totals what you've chosen before you commit to the wait.

Each model's row carries what you need to judge it without expanding anything — cartridge, image count, headstamp count, checkpoint size. 308 Experiment (old) · 38 image(s) · 0 headstamp(s) reads as junk at a glance, which is the decision being made.

On the implicit inheritance

Agreed, and the tree is how it's enforced rather than a rule the dialog has to explain: images, headstamps and the checkpoint are the model's branch, so unticking the model takes them with it. There's deliberately no leaf for the model's own row — it's the branch, so "images without the manifest" isn't expressible. A part the model has nothing for gets no row at all rather than a disabled one.

Here's a partial pick — two models declined outright, a third kept but without its 9 MB checkpoint, and the parents showing partial ticks:

On the ID conflict

You were right that there's a conflict, but it turned out not to be the id. ModelRepo.create allocates the rowid itself and the legacy Id never reaches the Model object at all — it survives only as a lookup key for training/images/<id> and the per-root remembered-pairing map. So an id collision with a model created on this side first is impossible by construction, and that was already true before this change.

The name was the real one. The ZIP import path has always resolved a colliding name with unique_model_name; this importer didn't, so importing onto a library that already had a "9mm" left you with two rows you couldn't tell apart. That function is now public and both paths use it — a colliding import lands as 9mm (2) and your existing model isn't touched. A re-import still matches on community UID or the remembered pairing and updates in place, so this doesn't turn repeat runs into a pile of copies (test pins that).

To make it visible instead of a surprise, each model's row now states its fate up front: new model here, or updates '' naming the row it would refresh.

The one thing I couldn't build: logs

There's nothing in a legacy install to populate that node. The layout is Data/ConfigDB.sjdb.json, Data/Settings.json, training/images/<id>/ and training/models/<id>.zip, and the config DB has no log or history table — so I left it out rather than shipping an empty branch. What did you have in mind there? Sort history, or the training confusion table (that one already rides along on the model row)?

Also fixed while in there

The ML.NET warning was install-wide, so declining an ML.NET model would still have told you it "was imported without a checkpoint" — a report on work that never happened. Warnings now hang off the model: the survey reports on the whole install (which is what informs your pick), the import reports only on what you actually took. Those models also say ML.NET model, retrain needed on their row now, since the absent Trained model file child otherwise reads as "never trained".

Docs updated in the same commit. Full suite green (1487 passed), ruff and ty clean.