Skip to content

[bugfix] LoRA + TP>1 + sequence_parallel: sum grads of the replicated LoRA factor over the TP group (exported adapter had last-layer linear_proj.lora_B == 0) - #173

Merged
hjh0119 merged 1 commit into
modelscope:mainfrom
hojuna:fix/lora-sp-tp-grad-allreduce
Sep 7, 2026
Merged

Commits

Commits on Aug 26, 2026