Add some materialization tooling for mixtures - #313

Draft

undfined wants to merge 46 commits into

mainfrom

undfined/3p5-materialize

Commits on Aug 4, 2026

WIP: Plan, inventory, materilize
undfined
committed
efc50cc
Better plan visual
undfined
committed
64578b7
more cleanup and merge stages
undfined
committed
bfa2f36
More visual cleanup
undfined
committed
ca8a536
pmr stuff
undfined
committed
db5d784

Commits on Aug 5, 2026

Better sampling procedure
undfined
committed
9613647
Add document selector module
undfined
committed
797d4ba
Fix error with globs
undfined
committed
1b0e721
Refactor ergonomics a bit
undfined
committed
71aec49
Simplify reports and drop rep cap
undfined
committed
22f53fa
Drop some pointless stats and fix preflight checks
undfined
committed
d3ff698
Add materialize script
undfined
committed
55bd697
Prefix handling and exec unit planner fixes
undfined
committed
308b8e2
more cleanup and exit on completion
undfined
committed
11e5919
Refactor layout
undfined
committed
df9bc94
Use isolated uv run
undfined
committed
7d4bf06
Handle full instance lifecycle
undfined
committed
92fffab
Dolma fix
undfined
committed
88da739
Log and project fixes
undfined
committed
47acfc6
Better logging and tags
undfined
committed
5829615
Observability and disrtibution tweaks
undfined
committed
ebffcc4
Better transfer logs and default s5cmd config
undfined
committed
1d0e309
More log cleanup
undfined
committed
c31c276
Better shard allocation and sizing
undfined
committed
0d3b4c8
Allow running preflight as part of execution launch
undfined
committed
cba5f75
Log cleanup
undfined
committed
8fa677b
Report and scheduling concurrency
undfined
committed
46bd134
Batch launching controls and defaults
undfined
committed
f8d5928
Handle resumption as well
undfined
committed
76c43ce
More machine boostrapping tweaks
undfined
committed
43913a2
Add exclusion arg
undfined
committed
28faa16
Log size streaming
undfined
committed
32ff676
pmr bug for more than 100 instances
undfined
committed
87d2792
more boostrap fixes
undfined
committed
a7fc122
Handle rehydrating pool
undfined
committed
4e0b97b

Commits on Aug 6, 2026

Cleanup
undfined
committed
26e7775
Move report builder
undfined
committed
09f302d

Commits on Aug 10, 2026

V2 version fo the dolma3p5 mix with decon+html
undfined
committed
6169c5f
Fix dispatch assignment for map
undfined
committed
9dde655

Commits on Aug 24, 2026

Refactor a bit and cleanup documentation
undfined
committed
b816f87
More docs
undfined
committed
328ae2a
CI tweaks
undfined
committed
38e4225
CI issues, again
undfined
committed
8d17c86
More tweaks
undfined
committed
2857e10
Cleanup d3p5 readme
undfined
committed
9ad0d4c
Another pass on docs
undfined
committed
d53f103