Feat/wandb experiment tracking

[caozx1110]

No description provided.

[Copilot]

Pull request overview

Adds first-class Weights & Biases (W&B) experiment tracking across training entrypoints, with consistent local run artifacts (config + summary) and optional video upload.

Changes:

• Introduces ExperimentTracker utilities for writing run_config.json / run_summary.json and initializing/reusing W&B runs.
• Extends on-/off-policy loggers + runners/scripts to reuse an existing W&B run and record a final training summary.
• Adds Hydra config defaults + tests + README documentation for new training.wandb_* fields, and adds wandb to dependencies.

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
uv.lock	Locks new transitive dependencies for wandb (and related HTTP/pydantic/protobuf packages).
pyproject.toml	Adds wandb as a project dependency.
README.md	Documents how to enable/configure W&B tracking and what gets recorded.
tests/utils/test_experiment_tracking.py	Adds unit tests for ExperimentTracker + W&B run reuse behavior.
tests/scripts/test_train_scripts.py	Verifies Hydra default values for new training.wandb_* fields.
src/unilab/utils/onpolicy_logger.py	Adds W&B entity/group/job_type/tags/notes support and avoids finishing a reused run.
src/unilab/utils/offpolicy_logger.py	Same W&B enhancements as on-policy logger, including safe run reuse.
src/unilab/utils/experiment_tracking.py	New shared tracking module (local artifacts, W&B settings, rsl-rl writer patch).
src/unilab/algos/torch/offpolicy/runner.py	Records last_run_summary (status, rewards, time, checkpoint) for downstream tracking.
src/unilab/algos/torch/appo/runner.py	Records last_run_summary (status, rewards, time, checkpoint) for downstream tracking.
scripts/train_rsl_rl.py	Integrates ExperimentTracker, patches rsl-rl W&B writer to reuse the tracker’s run, uploads play video.
scripts/train_offpolicy.py	Integrates ExperimentTracker, logs runner summary + optional play video.
scripts/train_mlx_ppo.py	Replaces ad-hoc W&B + JSON logging with ExperimentTracker and summary/video reporting.
scripts/train_appo.py	Integrates ExperimentTracker, logs runner summary + optional play video.
conf/ppo/config.yaml	Adds default training.wandb_* fields for PPO configs.
conf/ppo/config_mlx.yaml	Adds default training.wandb_* fields for MLX PPO configs.
conf/offpolicy/config.yaml	Adds default training.wandb_* fields for off-policy configs.
conf/appo/config.yaml	Adds default training.wandb_* fields for APPO configs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

patch_rsl_rl_wandb_writer() imports torch.utils.tensorboard.SummaryWriter unconditionally. Elsewhere (e.g., OnPolicyLogger/OffPolicyLogger) TensorBoard is treated as optional, so calling this patch in a W&B-only environment can raise ImportError and abort training. Consider wrapping this import in a try/except ImportError (and skipping the patch with a warning) so W&B logging can still work without tensorboard installed.

Suggested change

	from torch.utils.tensorboard import SummaryWriter as TensorboardSummaryWriter
	try:
	from torch.utils.tensorboard import SummaryWriter as TensorboardSummaryWriter
	except ImportError:
	import warnings
	warnings.warn(
	"TensorBoard is not available; skipping rsl-rl W&B writer patch. "
	"W&B logging will continue without TensorBoard integration.",
	RuntimeWarning,
	)
	return

[Copilot]

ExperimentTracker and log_fp are created/started, but tracker.finish() and log_fp.close() are not protected by a try/finally in this function. If training throws (e.g., env init, update loop), the run summary may never be written/flushed and the log file can remain open. Consider wrapping the training + optional play section in try/finally to always call tracker.finish() and close the file handle.

[Copilot]

learner_device is computed (and used for the ExperimentTracker), but the runner is still constructed with device=cfg.training.device (which can be null). This can cause the tracker metadata to disagree with the actual device selection and ignores the computed fallback. Consider passing learner_device into APPORunner(..., device=learner_device, ...) for consistency.

Suggested change

	device=cfg.training.device,
	device=learner_device,