Repository navigation
Replace the deprecated torch_dtype= argument with dtype= at the transformers model load sites #2200
Description
Context
transformers 5.x logs "torch_dtype is deprecated! Use dtype instead!" for a from_pretrained
call that passes torch_dtype=; soup serve showed it in the v0.77.0 smoke run (transformers
5.17). On v0.77.0, 15 files under src/soup_cli pass the keyword torch_dtype= and 19 lines build
a "torch_dtype" dict key for a load call (for example commands/serve.py, commands/chat.py,
commands/infer.py, trainer/sft.py, trainer/dpo.py, trainer/ppo.py,
utils/layer_stream_runtime.py). The declared floor is transformers>=5.16.1
(pyproject.toml:90), which accepts dtype=.
A static guard enforces the old spelling today:
tests/test_transformers_floor_compat.py::test_no_production_transformers_model_load_uses_dtype_kwarg
fails when a load site uses dtype= (it dates from the older floor, #478), and
tests/test_issue339_load_dtype.py, tests/test_issue491_frozen_base_load_dtype.py and a few more
assert on the torch_dtype keyword. The warning is cosmetic today.
Proposed fix
Do it as one sweep, in one pull request, because a half-migrated tree fails the guard either way:
change the spelling at each transformers load site, invert the guard so it requires dtype= for
from_pretrained and from_config on the transformers Auto classes (it must still flag a
deliberate torch_dtype= mutation), and update the tests that assert on the keyword name. Leave
alone Unsloth's FastLanguageModel (it already takes dtype), other libraries' APIs, and the
config-file field names in utils/kimi_k2.py, which describe a checkpoint's config.json, not a
load argument.
Acceptance
- No transformers load site passes
torch_dtype=; a CPU load of a tiny model with warnings turned into errors emits no deprecation message. - The floor-compat guard requires
dtype=and flags a deliberatetorch_dtype=mutation. - The CI cell that installs the transformers floor (5.16.1) passes.
Found during the v0.77.0 release review.
Activity
MakazhanAlpamys commentedon Oct 27, 2026
Confirmed, @Nari-Ab: #2200 is yours. #2200 stays with @Nari-Ab, and #2257 by @Nari-Ab is its pull request.
You did not comment here first, but the description of #2257 says what it changes and closes the issue, so we read it as your claim; nothing else holds the issue.
On scope: the issue asks for one sweep in one pull request, because a half-migrated tree fails the guard either way, so we will not ask you to split #2257, even at 44 files. It covers the load sites in the issue's list, leaves Unsloth's
FastLanguageModel, other libraries and the config-file names inutils/kimi_k2.pyalone, inverts the floor-compatibility guard so that it flags a deliberatetorch_dtype=, and loads a real tiny model on the CPU with warnings turned into errors (tests/test_transformers_load_dtype.py). Running the floor tests against an installed Transformers 5.16.1 in a separate target, as the description says, is the evidence we were hoping for.The hosted floor cell (Transformers 5.16.1) runs only with the
ci:fulllabel, and the issue's acceptance list asks for it to pass, so we will add the label when the review is done. A pull request this size needs a clean rebase right before it merges, so expect us to ask for one. When the review starts we will put atorch_dtype=spelling back in one direct call and in onemodel_kwargsdictionary and check that the guard flags both, and we will read the new scanner intests/test_transformers_floor_compat.py(+180 -47 in that file); it has to stay readable for whoever edits it next.You have five open pull requests (#2189, #2206, #2214, #2256 and #2257), which is your line; #2206 is waiting for your changes. Reviews go in issue-number order. We cannot give a date; a merge may wait for the v0.77.1 release that is being prepared, and we say here when it moves. This reply is not a promise to merge.
MakazhanAlpamys commentedon Oct 28, 2026
#2200 is fixed by #2257: no Transformers model load in
src/soup_clipassestorch_dtype=any more (35 load forms in 29 files moved todtype=, values unchanged), and a CPU load of a tiny model throughlive_evaland afrom_configskeleton through the layer-streaming runtime both run with warnings turned into errors and emit no deprecation message. The floor-compat guard now flagstorch_dtypeon a Transformers load, including inside locally built kwargs dictionaries and loader aliases; its docstring names the shapes it does not follow (dictionary union,setdefault, partials, dictionaries from another scope or a helper,pipeline(...), library config kwargs). Both were checked locally on Transformers 5.16.1 and 5.17.0, and the Transformers floor CI cell is part of the merge requirements. Left out of scope: three benchmark scripts underbenchmarks/harness/still use the old spelling. It lands with the next patch release.fix(models): use dtype consistently for Transformers loads (MakazhanA…