Skip to content

[megatron] Fix router replay CP partition and expert index overflow - #9981

Open
taking-lying-flat wants to merge 1 commit into
modelscope:mainfrom
taking-lying-flat:fix/router-replay-cp-expert-id
Open

[megatron] Fix router replay CP partition and expert index overflow#9981
taking-lying-flat wants to merge 1 commit into
modelscope:mainfrom
taking-lying-flat:fix/router-replay-cp-expert-id

Conversation

@taking-lying-flat

Copy link
Copy Markdown
Contributor

Summary

  • use the configured CP partition mode when slicing replayed expert indices, so contiguous-CP hidden states and router targets remain token-aligned
  • validate recorded expert indices before compression
  • retain uint8 storage for models with at most 256 experts and use int32 when more experts are configured

Problem

Router replay sliced global expert IDs with the default zigzag CP layout even when model inputs used contiguous CP. This could silently assign each local token the experts selected for a different token.

Recorded router indices were also always converted to uint8. Models with more than 256 experts therefore wrapped IDs above 255 before replay.

Compatibility

The CP mode argument defaults to zigzag, and the keyword is only forwarded for contiguous mode. This preserves the existing default path and compatibility with older mcore-bridge versions.

Verification

  • checked zigzag and contiguous token ownership for both ranks with CP=2
  • checked lossless expert IDs at 1, 2, 255, 256, 257, 301, and 65536 experts
  • checked rejection of negative and out-of-range expert IDs
  • ran the repository pre-commit hooks on both modified files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant