experiments.md

Simple-SciBuddy experiment guide

This guide describes the configured double-recursive RSI experiment. Read the algorithm guide for the learning procedure and evaluation semantics, or return to the ScienceBuddy overview. Commands below run from the repository root.

Maintained recipe

The checked-in experiment configuration and SkyRL recipe define the current experiment.

Component	Setting
Task model	Qwen3.5-4B
Auxiliary improver	Remote Chat Completions model; configured as gpt-6-astra, medium reasoning
Auxiliary output budget	8,192 tokens
Schedule	h0001 → r0001 → h0002 → r0002 → h0003 → r0003
Harness steps per stage	3
Train interactions per step	16
Proposals per step	3 from the same parent
Candidate selection	Complete 90-task Val comparison against the parent
RL updates per stage	30
RL batch	8 task instances × 8 attempts
Learning rate	1e-6
Backend	SkyRL, FSDP and vLLM
KL loss / KL reward penalty	Both disabled
Task-model sampling	Temperature 1, top-p 1, thinking disabled
Evaluation sampling	Temperature 0, one attempt per task
Rollout concurrency	64; candidate Val jobs share that limit
Evaluation schedule	Harness start/end; RL baseline and stage-end export load
Checkpoints	Stage-end HF model/tokenizer export, without optimizer state

These settings describe the maintained recipe, not measured performance. A completed fresh run contains three harness stages and three RL stages. A continuation reuses the validated stages recorded in its source run.

Dataset and budgets

The frozen release is data/releases/id715-val90-test90-v1:

Family	Train	Harness Val	Test
DbQA	411	50	50
GWAS	140	20	20
LitQA2	76	10	10
ProtocolQA	88	10	10
Total	715	90	90

Harness interactions and RL use Train. Harness selection uses Val. Both learning phases use the same Test assignment for evaluation. Val never enters RL updates. The split is by task ID; its material-overlap limitations are explained below and in the algorithm guide.

Episode limit	Budget
Context	24,576 tokens
Generated tokens per model call	4,096
Model calls / tool calls	9 / 9
Visible tool response	2,048 tokens
Total visible tool history	8,192 tokens
Python call timeout	60 seconds
Episode timeout	900 seconds
Researcher replies	At most 1 during harness interaction; 0 during RL/evaluation

Setup and run

Clone this repository as its own checkout, including its pinned SkyRL submodule:

git clone --recurse-submodules https://github.com/Gen-Verse/ScienceBuddy-RSI.git
cd ScienceBuddy-RSI

The training recipe targets Linux, Python 3.12, CUDA 13, Docker and eight A100 GPUs. Place Qwen3.5-4B at models/Qwen3.5-4B and the authorized frozen task release at data/releases/id715-val90-test90-v1. Weights and task assets are supplied separately. Machine-specific paths can be overridden in the ignored configs/local.json.

1. Set your improver endpoint and a fresh experiment ID in configs/train.toml.
2. Supply SCIENCEBUDDY_IMPROVER_API_KEY and WANDB_API_KEY through the environment, or literal assignments in .secrets/improver.env and .secrets/wandb.env.
3. Prepare the environment and launch from the repository root:

bash scripts/setup.sh
python scripts/train.py configs/train.toml

For an unattended run, use the exact experiment ID configured in the TOML:

experiment_id=qwen35-4b-coevolve-YYYYMMDD-01
mkdir -p runs/logs
nohup bash -c 'python scripts/train.py configs/train.toml; status=$?; printf "%s\n" "$status" > "runs/logs/$1.exit"; exit "$status"' _ "$experiment_id" > "runs/logs/$experiment_id.launch.log" 2>&1 < /dev/null &
echo "$!" > "runs/logs/$experiment_id.pid"

The defaults in configs/defaults.json are overridden by configs/local.json, then explicit experiment settings. All project paths in configuration must be relative: TOML paths are relative to the TOML file; JSON paths are relative to the repository root. Absolute settings and references or symlinks escaping this repository are rejected. Place the model and data inside this checkout rather than linking to files in another workspace. configs/skyrl.json holds the backend recipe. N_SAMPLES_PER_PROMPT=16 selects sixteen attempts instead of eight.

The task release must also be self-contained. Its environment.lock.json uses a relative data_lake.path (for example, resources) inside the release directory, and task-index task_dir fields are relative to their JSONL file. Keep public assets, private references and frozen runtime build files inside that release. The host's installed Python/CUDA/Docker toolchain and fixed paths inside the container are runtime prerequisites; no sibling project is imported. The launcher does not force a machine-specific CUDA or shared-library location. Optional SIMPLE_SCIBUDDY_CUDA_HOME and SIMPLE_SCIBUDDY_CUDA_COMPAT overrides must point to relative locations within this checkout.

The frozen task-ID split is 715 Train / 90 Val / 90 Test; Val is excluded from RL. Related material groups overlap across Train–Val (20 groups) and Train–Test (18 groups), so task-ID separation is not a claim of independent test material. All assignments are preserved; setup does not reconstruct or randomly resplit data.

Each experiment records input identities, selected harnesses, model exports and measurements under runs/. Stage-end exports omit optimizer state. An optional continue_from in the TOML starts a new run from a verified completed boundary; unsaved optimizer updates are not restored.

CPU checks and repository layout

python3.12 -m venv .venv-check
.venv-check/bin/python -m pip install -e '.[dev]'
bash scripts/check.sh

The default checks use synthetic data and mocked components; they exclude tests/integration/ and need no model, task release, GPU or Docker.

assets/                  Paper illustrations and the usage video
docs/                    Algorithm and experiment guides
configs/                 Runtime defaults and experiment recipe
src/simple_scibuddy/      Standalone scientific-agent RSI implementation
scripts/                 Setup, launch and CPU checks
tests/                   Unit tests and explicit integration checks
skyrl/                   Pinned upstream SkyRL submodule

Inspect a run

The experiment preserves the launch configuration, identities and measurements under its own directory in runs/. The following files are useful for checking algorithm behavior without starting additional model calls:

Record	What it establishes
config.toml, config.json, identities.json	Requested schedule and pinned inputs
harness_evolve/hNNNN/selection-tasks.json	Fixed Val task membership
harness_evolve/hNNNN/step-NNNN/selection.json	Candidate validity, paired scores and selection
harness_evolve/hNNNN/summary.json	Stage-boundary Test measurements and selected harness
harness_evolve-history.json	Accepted/rejected proposals and aggregate Val comparisons
shared/rounds/rNNNN/request.json	Frozen model/harness/runtime handoff contract
shared/rounds/rNNNN/rl-train/settings.json	Effective RL settings and input identities
shared/rounds/rNNNN/rl-train/evaluation-history.jsonl	In-trainer evaluations, including the RL baseline
shared/rounds/rNNNN/rl-export-eval/evaluation.json	Test evaluation from the actual exported model
shared/rounds/rNNNN/result.json	Validated stage completion and model export
continuation.json	Inherited stage boundary and recovery provenance

Raw episode records preserve first answers, verifier outcomes, model calls, observations and termination reasons. Training accuracy describes changing sampled batches; it should not be treated as a fixed-panel Test curve.

Other configured modes

• mode = "harness_evolve": one harness-learning phase with the task model fixed.
• mode = "model": train or evaluate with a fixed harness, selected through [model].mode and optional relative model/harness paths.
• mode = "coevolve": alternate harness and model learning for the configured number of complete cycles.

smoke and overfit model modes execute actual rollouts. The CPU check script and configuration parsing are separate from these model-executing modes.