Skip to content

Support generic supplied document injection#6

Open
lilyjge wants to merge 1 commit into
castorini:mainfrom
lilyjge:allow-document-injections
Open

Support generic supplied document injection#6
lilyjge wants to merge 1 commit into
castorini:mainfrom
lilyjge:allow-document-injections

Conversation

@lilyjge

@lilyjge lilyjge commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

  • add generic supplied document bundles for benchmark runs via --supplied-doc-bundle / SUPPLIED_DOC_BUNDLE
  • inject grouped supplied evidence into the agent prompt and record metadata.supplied_docids
  • count supplied docids in surfaced, previewed, and agent-visible retrieval views
  • document the bundle JSONL format and wire the high-level query-set launcher to forward the option

Verification

  • npm run typecheck
  • npm run format:check
  • npm run lint
  • git diff --check
  • npx tsx --test tests/benchmark_query_set_launch.test.ts tests/launcher_wrappers.test.ts tests/run_pi_benchmark_pi_search_contract.test.ts

Note

  • Full npm test is 229/230 locally; the unrelated failing test expects missing local fixture data/msmarco-v1-passage/queries/dl19.tsv in compare_bm25_runs.test.ts.

@lilyjge

lilyjge commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

@lingwei-gu for context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant