🏈 PickSix

An NFL analyst agent that answers natural language queries, originally built to study inference vs. retrieval latency tradeoffs in tool-calling agents.

Runs on a Google Cloud TPU VM via vLLM with Qwen3-4B (although other models can be configured via serve.sh). Designed to answer three query types:

• Win/loss analysis: game outcomes, season records, playoff runs (e.g. "Why did the Ravens lose the 2024 AFC Divisional game?")
• Player performance: individual stats, situational splits, career trends (e.g. "How did Derrick Henry perform in the 2020 season?")
• Predictions: forward-looking questions about upcoming seasons, backed by web search (e.g. "Will the Seahawks win the 2026 NFC West?")

PickSix achieves ~51% accuracy on the evaluation set with Qwen3-4B.

Tool layer was built around ddgs and nfl-data-py.

Usage

Install dependencies and start the model server:

./setup.sh
./serve.sh

Run the default agent with a query like: python3 agent.py --query "Why did the Baltimore Ravens lose the 2024 AFC Divisional playoff game?"

View its thought/action/observation steps with: python3 agent.py --query "..." --trace

Evaluation

Run the agent against evaluation data with commands like:

python3 eval/run_eval.py # eval all categories
python3 eval/run_eval.py --category win_loss # eval one category
python3 eval/run_eval.py --id winloss_001 # eval one question

Results files are tagged with the agent name, timestamp, and run type.

To evaluate the (less effective) planning-and-decomposition agent, pass --agent single_pass as an argumenet to run_eval.

Single-pass agent

This repo contains agent.py and agent_single_pass.py. Both agent implementations share the same tools and the same run() / _get_model() interface:

• agent.py: Leverages a ReAct loop. The model observes each tool result before deciding the next action, enabling adaptive multi-step reasoning.
• agent_single_pass.py: Original plan-execute-synthesize pipeline. The planner emits all tool calls upfront, they execute in parallel, then the synthesizer writes the answer. Supports one bounded re-plan round for dependent tool calls.