Fix preference max length enforcement across TRL wrappers

changelog.d/0.77.0/1196.fixed.md

@@ -0,0 +1 @@
+- Restored `data.max_length` enforcement for SimPO and KTO on TRL releases that removed `max_prompt_length`, rebuilding the KTO/BCO completion from the untruncated `answer_input_ids` so an over-length prompt no longer discards it, and covering the same measured over-length path for IPO and BCO. For SimPO, KTO, IPO and BCO a prompt is cut only when the row does not fit in `data.max_length`, which replaces the unconditional cut that #2116 introduced (#1185 by @ousamabenyounes in #1196).

docs/backends-and-ops.md

@@ -278,10 +278,12 @@ That shows the objective moving, not alignment quality.
   transformers Trainer does for TRL's CPO.
 - **Tokens** are ORPO's above: TRL's CPO and ORPO trainers tokenize rows with
   the same code, so rows that fit in `data.max_length` train on the same tokens
-  on either backend. Over-long rows are cut with Soup's cap (prompt to
-  `max_length // 2`, keeping its end, answers to the remaining room). The
-  transformers SimPO path applies no prompt cap: TRL 0.29.1 cuts only the
-  answers there, and Soup refuses a row whose shorter answer that cut empties.
+  on either backend. MLX SimPO, DPO and ORPO cut over-long rows by the prompt
+  alone (prompt to `max_length // 2`, keeping its end, answers to the remaining
+  room). The transformers SimPO path now applies that same cap, but only when a
+  row does not fit `data.max_length` (TRL 0.29.1 cuts only the answers, so a
+  long prompt reached the model whole); Soup refuses a row whose shorter answer
+  that cut empties.
 - **Ties train**, as in TRL: the preference term is the constant
   `softplus(simpo_gamma)` and the NLL still learns.
 - **Stops on a nonfinite loss** instead of training on NaN.
@@ -375,8 +377,9 @@ training:
   completion located even when the tokenizer merges the last prompt token
   into it, BOS prepended to the prompt and EOS appended to the completion when
   missing. A row longer than `data.max_length` loses the end of its completion;
-  the prompt is never cut (TRL 0.29.1 has no prompt cap, and the transformers
-  path adds none). A KL row is re-cut to its own prompt's room.
+  the prompt is never cut (TRL 0.29.1 has no prompt cap; the transformers path
+  now caps the prompt only when a row does not fit `data.max_length`). A KL row
+  is re-cut to its own prompt's room.
 - **Labels** are read by the loader's `format: kto` converter, exactly as on the
   transformers backend (`true` / `false`, and the strings and numbers that
   converter accepts); MLX adds no reading of its own. Only a row handed to the

docs/training.md

@@ -1361,6 +1361,12 @@ frozen reference.
 Same input format as DPO; rows are split internally to TRL's BCO
 unpaired schema (`{prompt, completion, label}`).
 
+When TRL no longer exposes `max_prompt_length`, Soup still enforces
+`data.max_length`: BCO, IPO, KTO, and SimPO cut prompts only when a row
+would exceed `max_length`. BCO, KTO, and SimPO keep the prompt end; IPO
+follows DPO's `truncation_mode` (`keep_start` by default). DPO/ORPO keep
+their legacy `max_length // 2` prompt cap.
+
 ```yaml
 task: bco
 data:
@@ -1935,14 +1941,12 @@ training:
   quantization: 4bit
 ```
 
-**Long prompts are capped as in DPO and ORPO.** trl's CPO trainer cuts only the
-answers, so after it tokenizes, `soup train` cuts the prompt to
-`data.max_length // 2` tokens, keeping its end, and the answers to the room
-left, for the train and validation splits. Every sequence is then at most
-`data.max_length`. The cut goes by the prompt alone: a prompt longer than
-`data.max_length // 2` tokens loses its start even when prompt and answers
-would fit in `data.max_length`. A row whose prompt is at most
-`data.max_length // 2` tokens and which fits is unchanged.
+**Long prompts are capped only when a row overflows.** trl's CPO trainer cuts
+only the answers, so a long prompt reached the model whole. A SimPO prompt is
+now cut only when the row does not fit in `data.max_length`: it is then cut to
+`data.max_length // 2` tokens, keeping its end, and the answers get the room
+left, for the train and validation splits, so every sequence is at most
+`data.max_length`. A row that fits is unchanged, whatever its prompt length.
 
 **Rows whose completion trl truncates away are refused.** trl's CPO trainer
 truncates each answer to `data.max_length` minus the **longer** answer's length,