fix: normalize rewards by explicit sample groups

[morluto]

Problem

RolloutManager._post_process_rewards inferred GRPO reward groups from the total sample count. When a custom rollout returns uneven numbers of samples per prompt, the fallback reshaped the reward vector to one row and centered rewards across the entire batch.

For groups of sizes 4, 3, and 4, a prompt whose three rewards are all 5.0 received advantages of -1.090909 instead of zero.

Changes

• Normalize rewards using each sample's group_index and restore results in input order.
• Keep fixed-size compatibility for legacy custom samples where every group_index is missing; reject mixed or uneven unidentified groups instead of silently using a global baseline.
• Return zero for singleton groups when standard-deviation normalization is enabled.
• Remove the fanout test's custom reward-normalization workaround so that it exercises the production path.
• Add the CPU regression test to the regular PR test matrix.

Validation

\n- uvx --from pre-commit pre-commit run --all-files --show-diff-on-failure --color=always

• PYTHONPATH=. python3 tests/test_reward_utils.py (5 passed)
• ruff check slime/ray/rollout.py slime/rollout/reward_utils.py slime/rollout/_fanout_test_helpers.py tests/test_reward_utils.py tests/test_qwen2.5_0.5B_fanout_short.py
• python3 -m py_compile slime/ray/rollout.py slime/rollout/reward_utils.py slime/rollout/_fanout_test_helpers.py tests/test_reward_utils.py tests/test_qwen2.5_0.5B_fanout_short.py
• python3 .github/workflows/generate_github_workflows.py
• Autoreview on the complete staged diff: no accepted or actionable findings

tests/test_qwen2.5_0.5B_fanout_short.py was not run locally because it requires four GPUs; it remains registered in the GPU CI matrix.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.