Skip to content

Fix deprecated RLlib API and CNN policy definition in pistonball tutotial per #1253 - #1418

Open
jangroter wants to merge 4 commits into
Farama-Foundation:mainfrom
jangroter:main
Open

Fix deprecated RLlib API and CNN policy definition in pistonball tutotial per #1253 #1418
jangroter wants to merge 4 commits into
Farama-Foundation:mainfrom
jangroter:main

Conversation

@jangroter

Copy link
Copy Markdown

Description

The pistonball rllib tutorial (tutorials/ray/rllib_pistonball.py) used an old deprecated API stack for interfacing with RLlib, additionally, the custom CNN policy was registered, but never defined to be used by RLlib. This PR updates this tutorial to use the new API stack (since ray 2.56).

This also changes the way the CNN policy has to be defined, which is now a list of [[num_out_channels, kernel, stride],...]. The resulting CNN policy should be identical to the one originally defined in the tutorial.

One caveat: I do not have the required compute available right now to train a model, as a result there is no trained policy to observe, which is present in the original tutorial in the documentation.

Fixes #1253

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Screenshots

Please attach before and after screenshots of the change if applicable.
To upload images to a PR -- simply drag and drop or copy paste.

Checklist:

  • I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
  • I have run pytest -v and no errors are present.
    I did have errors for not having all ROMs installed through autoROM, but because my changes do not affect any environments I considered it non-critical to check
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@virgilt

virgilt commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

@jangroter the pre-commit is failing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Proposal] Proper initialization of the custom CNNmodel in the RLlib tutorial

2 participants