Skip to content

Capture vllm server log into test_vllm_inference output#270

Open
atnair-amd wants to merge 1 commit into
dev/dtnifrom
atnair/vllm-server-log
Open

Capture vllm server log into test_vllm_inference output#270
atnair-amd wants to merge 1 commit into
dev/dtnifrom
atnair/vllm-server-log

Conversation

@atnair-amd

Copy link
Copy Markdown
Collaborator

Summary

_check_early_failure() only tails the vLLM server log during the
startup/readiness wait; nothing re-checks it afterward, so a
mid-benchmark server crash (e.g. EngineDeadError) never appears in
test_vllm_inference's captured output.

Change

  • Added VllmJob.dump_server_log(), mirroring the existing
    dump_client_log() pattern but per-rank, skipping ray-backend
    headless workers (rank > 0), which never run vllm serve
  • Wired into test_vllm_inference's failure path, via
    lifecycle.live_server_job so the dump targets whichever job
    actually owns the running server (on the server-reuse path, an
    earlier cell's job -- not the current cell's, which never issued
    build_server_cmd()/start_server())
  • Out of scope: no success-path dump -- the server log is per-server,
    not per-cell, so dumping on every passing cell sharing a reused
    server would re-emit the same growing log repeatedly

Tests

  • New TestDumpServerLog (3 cases: full content, mp-backend dumps
    every rank, ray-backend skips worker ranks) in
    test_vllm_job_server_reuse.py
  • Gate: .test_venv/bin/python -m unittest discover -s cvs/lib/inference/unittests -p "test_vllm_job*.py" (92 passed)
  • Full cvs/lib/inference/unittests discovery: 205/207 passed (2
    pre-existing failures in test_inferencing_config_loader.py,
    unrelated, reproduced identically on unmodified dev/dtni)

_check_early_failure() only tails the server log during startup, so a
mid-benchmark crash (e.g. EngineDeadError) currently vanishes from the
captured pytest output. Add VllmJob.dump_server_log(), mirroring the
existing dump_client_log() pattern but per-rank, skipping ray-backend
headless workers (rank > 0) which never run vllm serve.

Wire it into test_vllm_inference's failure path only, tracking
lifecycle.live_server_job so the dump targets whichever job actually
owns the running server -- on the reuse path (cells that only differ
by concurrency) that's an earlier cell's job, not the current one.
@atnair-amd
atnair-amd marked this pull request as ready for review July 24, 2026 19:30
@atnair-amd atnair-amd self-assigned this Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant