Skip to content

fix: align timeout bootstrap with final observation contract - #240

Merged
TATP-233 merged 5 commits into
mainfrom
fix/issue-236-time-limit-bootstrap
Apr 12, 2026
Merged

TATP-233 merged 5 commits into
mainfrom
fix/issue-236-time-limit-bootstrap

Conversation

@TATP-233

Copy link
Copy Markdown
Collaborator

Fixes #236

Summary

  • align synchronous PPO timeout bootstrap with the env final_observation contract instead of reusing V(obs_t)
  • keep APPO and off-policy collector actor inputs on the post-reset observation while still storing true terminal observations for learning targets
  • move final_observation handling behind a shared transition-bootstrap contract instead of duplicating worker-side patching rules

Impact

  • PPO / RSL-RL
  • MLX PPO
  • APPO collector / learner
  • off-policy collector

Validation

  • UV_CACHE_DIR=/tmp/uv-cache make check
  • UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/utils/test_final_observation.py tests/algos/test_rsl_rl_ppo.py tests/algos/test_appo_worker.py tests/config/test_locomotion_params.py -q
  • UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/utils/test_final_observation.py tests/algos/test_rsl_rl_ppo.py tests/algos/test_appo_worker.py tests/config/test_locomotion_params.py tests/scripts/test_train_scripts.py -k 'test_play_wrapper_step_exports_timeout_bootstrap_obs or test_play_wrapper_policy_obs_mode_actor or test_play_wrapper_uses_current_reset_contract or test_ppo_hydra_default_algo_log_name' -q

Notes

  • tests/scripts/test_train_scripts.py -k test_mlx_time_limit_bootstrap_values_use_final_observation still aborts locally because importing scripts/train_mlx_ppo.py triggers a native mlx extension crash in this machine's environment.

@TATP-233
TATP-233 merged commit d45b607 into main Apr 12, 2026
5 checks passed
@TATP-233
TATP-233 deleted the fix/issue-236-time-limit-bootstrap branch April 12, 2026 16:57
TATP-233 added a commit that referenced this pull request May 4, 2026
…trap

fix: align timeout bootstrap with final observation contract
TATP-233 added a commit that referenced this pull request May 30, 2026
…trap

fix: align timeout bootstrap with final observation contract
TATP-233 added a commit that referenced this pull request Sep 4, 2026
…trap

fix: align timeout bootstrap with final observation contract
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: time-limit bootstrap ignores final_observation in synchronous PPO, and APPO/off-policy workers reuse patched terminal obs as next actor input

1 participant