lora - #2

Open

xrsrke wants to merge 118 commits into

mainfrom

phuc/multilora-native-nemorl

Commits on Feb 18, 2026

Add nano-v3 config
yfw
committed
3eb2c48
Cherry pick gym-related nano-v3 changes

yfw
and
pjin-nvidia
committed
1503eb7
Update config
yfw
committed
0b6757b

Commits on Feb 19, 2026

Merge remote-tracking branch 'origin/main' into yifu/nano-v3-config
yfw
committed
b86a31f

Commits on Feb 20, 2026

Remove unused
yfw
committed
4d35ce3
Merge remote-tracking branch 'origin/main' into yifu/nano-v3-config
yfw
committed
e5f2e7b
lint
yfw
committed
8f8857b

Commits on Feb 21, 2026

Super model running with mcore + mbridge bumps
yfw
committed
499108c
Support for ray spinup within gym

yfw
and
pjin-nvidia
committed
c722c91
add genrm rlhf to gym

committed
5bbf159
fix min&max metric

yfw
and
HeyyyyyyG
committed
a159d65
Add grpo_superv3.yaml
yfw
committed
750bd2a
async nemo gym (#1999)

authored
6bbd083
skip log probs + offload before refit

yfw
and
gshennvm
committed
3f30b80
Use prompt length to find groups for advantage calculation

yfw
and
HeyyyyyyG
committed
241ddde
Allow replay buffer checkpointing and reloading capability

yfw
and
parthchadha
committed
0bb29f4
custom reasoning parser

yfw
and
HeyyyyyyG
committed
657a1b4
Skip calculating prev_logprob when force_onpolicy_ratio is true

yfw
and
HeyyyyyyG
committed
118de81
super ckpt conversion
yfw
committed
b556e9c
skip loading ref model when kl>0

yfw
and
HeyyyyyyG
committed
28ecbb8
only calculate loss on assistant turns generated by model

yfw
and
HeyyyyyyG
committed
16b4890
Add seq level logprob error metrics

yfw
and
HeyyyyyyG
committed
145c744

Commits on Feb 22, 2026

Fix for ray state API usage (list_nodes with limit). Enable debug logging.

yfw
and
pjin-nvidia
committed
5c6d5e4
effort work

yfw
and
gshennvm
committed
1693cc1
fix genrm reasoning off

yfw
and
HeyyyyyyG
committed
48fe510
don't count fully masked sequences

yfw
and
HeyyyyyyG
committed
b856899
add overlong filtering for gym rollout + configurable advantage clipping

yfw
and
HeyyyyyyG
committed
57bf919
increase last_target_weight_already_generated only when whole batch is generated and consumed

yfw
and
HeyyyyyyG
committed
3186fb4
merge swe openhands + fix misc env issues + fix nemo skills sandbox

yfw
and
HeyyyyyyG
committed
3fa3e32
Respect external VLLM_CACHE_ROOT and add cache optimization guide

yfw
and
yashaswikarnati
committed
ae1ab08
penalize invalid toolcall + log TIS clipped fraction

yfw
and
HeyyyyyyG
committed
2da2585
POC ray put

yfw
and
guyueh1
committed
d667613

Commits on Feb 23, 2026

Update grpo_superv3.yaml with latest `main` changes
yfw
committed
6439707
Fix penalize invalid toolcall with prorl changes
yfw
committed
d6b27b5
MTP changes
yfw
committed
8106050
overlap NeMo Gym init with vLLM init

yfw
and
yashaswikarnati
committed
eb401ba
fix missing logger

yfw
and
HeyyyyyyG
committed
6efed5b
Pass explicit PG to gym

yfw
and
yashaswikarnati
committed
46e5ec5
penalize toolcall in reasoning when it should be outside + invalid thinking tag

yfw
and
HeyyyyyyG
committed
b941004
fix: add support to configure RL+Gym+vLLM port ranges to avoid port contention failures

yfw
and
ananthsub
committed
855550a
feat: add [GPU_DIAG] memory diagnostics logging during worker init

yfw
and
yashaswikarnati
committed
d9a212c
Fix setuptools error
yfw
committed
e95ea74

Commits on Feb 24, 2026

Update ray.sub
yfw
committed
5f87e2b
Update gym
yfw
committed
234df47
Update Megatron setup.py
yfw
committed
ec81c6b
Remove vllm version check
yfw
committed
fdf34df
Update uv.lock
yfw
committed
471bf18
Bump mbridge for mtp fix
yfw
committed
ab77301
Add stage 1 recipe
yfw
committed
f4163a6
bump mcore for mtp detach fix
yfw
committed
13559c7

Commits on Feb 27, 2026

effort level updates

haifengqian
authored and
yfw
committed
76ddd97
Fix genrm for async gym
yfw
committed
3f391da

Commits on Mar 2, 2026

Bump Megatron-Bridge to copy reasoning parser in conversion
yfw
committed
5417b36

Commits on Mar 3, 2026

Bump mbridge to fix fp32 expert bias
yfw
committed
55880f1
feat: caching gym venvs and apptainer for super branch (#2050)
terrykong
authored
05eb916

Commits on Mar 7, 2026

build: Bump dependencies to address CVEs (#2082)
chtruong814
authored
01d0f5d
SWE stage changes + update recipes / docs (#2083)

yfw
and
sdevare-nv
authored
92208dc
Update doc
yfw
committed
df4d391
Remove auto mount
yfw
committed
279f2e4
Point gym to public super-v3
yfw
committed
211dc1f

Commits on Mar 8, 2026

build: Ensure xgrammar==0.1.32 in vllm and gym venvs (#2086)
chtruong814
authored
f7fe11c

Commits on Mar 9, 2026

Use super configs for prefetch envs
yfw
committed
7b30253
Handle partial download
yfw
committed
ddf2d9b
Address Terry's feedback
yfw
committed
c6702b2
Update swe2
yfw
committed
0a7b1d3

Commits on Mar 10, 2026

100 samples for validation
yfw
committed
a90de92

Commits on Mar 11, 2026

Fix penalize_invalid_tool_call and penalize_malformed_thinking
yfw
committed
c21c87e
Rebuild swe prefetch container
yfw
committed
3ab7a2c
Update dataset name
yfw
committed
7933ea6
Update launch script
yfw
committed
61cb0f3

Commits on Mar 12, 2026

Fix: apptainer download script (#2101)
sdevare-nv
authored
298fea1
Remove unused configs
yfw
committed
347ac6a

Commits on Mar 19, 2026

docs(super): Add note about vllm bug prior to 0.17.0 (#2127)
yfw
authored
9c77bef

Commits on May 27, 2026

Commits on May 28, 2026

sft: add 1-step debug instrumentation for token_mask=0 diagnosis on Super-120B
xrsrke
committed
0925aa0

Commits on May 29, 2026

bit-eq diag: NLLLoss + worker + train fingerprints (symmetric with nousnet MultiAdapterLoss)
Phuc Nguyen
committed
a79ce1e
worker: thread dump_phase + step to aggregate_lora_fingerprints
Phuc Nguyen
committed
d769538
fix(automodel/train): skip non-numeric metrics in per-mb division
xrsrke
committed
7ee631a

Commits on Jun 11, 2026

debug(multi-lora): NOUSNET_DUMP_EVERYTHING hooks in dtensor worker v2
xrsrke
committed
d64656d

Commits on Jul 17, 2026

diag: add exact-init and per-adapter clip probes
xrsrke
committed
9d927fc
diag: log worker source hash for exact-init probes
xrsrke
committed
1da6d85
fix: preserve masked SFT rows in bit-equivalence probes
xrsrke
committed
8625745
fix: match stock norm semantics for per-adapter clipping
xrsrke
committed
fff22a9
fix: per-adapter clip in-place scaling with correct norm
xrsrke
committed
1e5189b
diag: emit compact single-adapter loss traces
xrsrke
committed
2e06046
diag: support lightweight pairwise loss tracing
xrsrke
committed
4c3c1b8
fix: compute per-adapter global valid-token counts for DP>1
xrsrke
committed
ca95312
fix: all-reduce dense adapter token counts
xrsrke
committed
c323e67
fix: stripe multi-adapter rows across DP ranks
xrsrke
committed
7400de2
diag: support trace-only single-adapter loss capture
xrsrke
committed
2456200

Commits on Jul 23, 2026

Commits on Jul 24, 2026

Commits on Jul 28, 2026

multi-lora: fail closed on adapter-axis FSDP sharding + parity report
xrsrke
committed
c0841cb
multi-lora: add final evidence reproduction bundle
xrsrke
committed
9db1502
multi-lora: make reproduction reruns collision-safe
xrsrke
committed
000d2ac