You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
MLX DPO, ORPO and SimPO cut the prompt to max_length // 2 even when the row fits (follow-up to #1209) #2199
Blocked by #1209 and #1196. Please do not start before both have landed.
The maintainer ruled on #1209 that DPO and ORPO on the transformers backend cut the prompt only when
prompt plus answer overflow max_length; #1196 puts the same rule on SimPO, KTO, IPO and BCO. The
review threads of both left the MLX files out on purpose, as a separate change.
The MLX trainers mirror TRL's tokenization so that one soup.yaml trains on the same tokens on
either backend (see the module docstring of src/soup_cli/trainer/mlx_dpo.py). Today they cut the
prompt by the prompt alone:
DPO: src/soup_cli/trainer/mlx_dpo.py:160 (_truncate_tokens(prompt_ids, max_prompt_length, ...)
with max_prompt_length=max_length // 2 from _build_example, :504-507).
ORPO: src/soup_cli/trainer/mlx_orpo.py:162 (build_orpo_example, called with max_length // 2 from _build_example, :313-316). mlx_simpo.py builds its rows with the ORPO
wrapper, so SimPO shares this site.
Once #1209 and #1196 are in, a long-prompt row that fits trains on different tokens on MLX and on
transformers.
Proposed fix
Apply the same rule in the two MLX builders: cut the prompt only when the prompt plus the answer
exceed max_length, from the same end as today (DPO keeps the start of the prompt, ORPO and SimPO
the end), and let the answers keep the room that remains. Reuse the rule #1196 puts in trainer/_trl_compat.py if it works on token lists, otherwise a small pure function with the same
cases. The row-building tests run without Apple hardware (fake tokenizer); a live MLX run is a
welcome extra for someone with Apple Silicon, not a requirement.
Acceptance
A DPO row and an ORPO/SimPO row that fit in data.max_length keep their whole prompt in build_preference_example and build_orpo_example; a test fails if the max_length // 2 cap touches them.
A row that overflows is still cut from the same end as today.
The preference-prompt wording in docs/backends-and-ops.md says both backends cut under the same rule.
A changed changelog fragment says long-prompt MLX configs now keep the whole prompt when the row fits.
Activity