From 0449b34143af69b159e0146e1fb54f71bb68c7fa Mon Sep 17 00:00:00 2001 From: Vineeth Sai Date: Wed, 12 Aug 2026 10:58:59 -0700 Subject: [PATCH] Drop the documented grad_hooks ZeRO option, which does not exist config-json.md documents grad_hooks as a ZeRO option with a default of True, but DeepSpeedZeroConfig has no such field and DeepSpeedConfigModel sets extra="forbid", so following the docs is a hard failure: DeepSpeedZeroConfig(**{"stage": 1, "grad_hooks": False}) pydantic_core._pydantic_core.ValidationError: 1 validation error grad_hooks Extra inputs are not permitted The option was never wired up rather than removed later. cfa63f5da ("ZeRO stage 1 refresh", #1042) added the doc entry and DeepSpeedEngine.zero_grad_hooks() in the same commit without ever adding the field to the config or to the constants module, and it is absent from deepspeed/runtime/zero/config.py in every release back to v0.3.0. So there is nothing to restore, and zero_grad_hooks() reads zero_config.grad_hooks, which can only raise AttributeError. Nothing in the package or the tests calls it. Remove the doc entry and the dead accessor. Signed-off-by: Vineeth Sai --- deepspeed/runtime/engine.py | 3 --- docs/_pages/config-json.md | 6 ------ 2 files changed, 9 deletions(-) mode change 100755 => 100644 deepspeed/runtime/engine.py mode change 100755 => 100644 docs/_pages/config-json.md diff --git a/deepspeed/runtime/engine.py b/deepspeed/runtime/engine.py old mode 100755 new mode 100644 index 5db39d237967..41761d772707 --- a/deepspeed/runtime/engine.py +++ b/deepspeed/runtime/engine.py @@ -1287,9 +1287,6 @@ def zero_model_persistence_threshold(self): def zero_gather_16bit_weights_on_model_save(self): return self._config.zero_config.gather_16bit_weights_on_model_save - def zero_grad_hooks(self): - return self._config.zero_config.grad_hooks - def zero_legacy_stage1(self): return self._config.zero_config.legacy_stage1 diff --git a/docs/_pages/config-json.md b/docs/_pages/config-json.md old mode 100755 new mode 100644 index 8f91414abd8f..c9abd8d817eb --- a/docs/_pages/config-json.md +++ b/docs/_pages/config-json.md @@ -531,12 +531,6 @@ Enabling and configuring ZeRO memory optimizations | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | | Initialize fp32 master weights from fp32 copies in checkpoint (no precision loss) or from model's fp16 copies (with precision loss). This can be used to initialize optimizer state even when checkpoint is missing optimizer state. | `True` | -**grad_hooks**: [boolean] - -| Description | Default | -| ----------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| For use with ZeRO stage 1, enable backward hooks to reduce gradients during the backward pass or wait until the end of the backward pass. | `True` | - ***round_robin_gradients***: [boolean] | Description | Default |