[Dev] Add zero-CTA all-gather policy for Megatron-FSDP#5657

bobboli wants to merge 2 commits into

NVIDIA:devNVIDIA/Megatron-LM:devfrom

bobboli:perf/zero-sm-allgather-fsdpbobboli/Megatron-LM:perf/zero-sm-allgather-fsdpCopy head branch name to clipboard