[Feature] Add flat replay writes to collectors

[vmoens]

Summary

• add an opt-in flatten_data collector argument that writes batched rollouts to 1-D replay storage with one reshape and one extend
• support local, asynchronous, multi-process, and Ray collector paths, including Hydra configs and lazy shared-storage initialization
• document flat 1-D replay buffers as TorchRL's preferred layout and add regression and benchmark coverage

Motivation

Batched environments produce [N, T] rollouts. Direct replay writes currently preserve that batch shape, so 1-D storage records N items with length-T payloads instead of N * T transitions. This makes transition sampling and frame accounting awkward, particularly for Ray-backed collectors.

With flatten_data=True, collectors reshape the rollout once immediately before replay insertion, preserving one replay-buffer extend call while storing individual transitions.

Compatibility

The option defaults to False, so existing multidimensional replay-buffer layouts are unchanged. The deprecated extend_buffer=False path is rejected with flattening, and padded split_trajs=True output remains separate from flat replay writes.

Testing

• pre-commit run --files <changed files>
• python -m pytest test/test_collectors.py::TestCollectorRB -q (19 passed, 25 skipped)
• python -m pytest test/test_collectors.py::TestTrajsPerBatchReplayBuffer -q (28 passed)

Ray and config regression tests were added but not executed locally because their optional dependencies were unavailable.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4020

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 3794707 with merge base f2a2337 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Libs Tests on Linux / unittests-gym (3.10, 12.8) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.