feat(inference): lm-eval-harness based accuracy evaluation + vllm suite integration

[atnair-amd]

Jira: AIMVT-269

Summary

Adds an lm-eval-harness based accuracy evaluation capability, opt-in per suite via config.json.

• Config schema (accuracy_config.py): AccuracyTask/AccuracyConfig (pydantic, extra=forbid) hold task selection only — id/task/num_fewshot/metadata/include_path/num_concurrent/apply_chat_template/gen_kwargs. Rejects duplicate task ids.
• Result parsing (lm_eval_parsing.py): project() auto-discovers real numeric metrics from an lm-eval results.json payload and flattens them into a single dict, keyed task.metric__filter.
• Command construction + orchestration (lm_eval_job.py): builds the lm_eval CLI invocation (local-completions vs local-chat-completions, model_args, gen_kwargs, metadata) and runs it via orch.exec_on_head against an already-live inference server, then locates and parses the resulting results*.json.
• Lifecycle stage (inference_suite_lifecycle.py::test_accuracy_eval): shared, opt-in pytest stage — skips cleanly if a prior stage failed or no accuracy tasks are configured, records per-task results via lifecycle.record, and gates each task independently against threshold.json's accuracy.<task_id> block when enforce_thresholds is set (record-only otherwise).
• Suite wiring: accuracy: AccuracyConfig field added to both VariantConfig (vllm) and InferenceXAtomVariantConfig (inferencex_atom), defaulting to empty so existing configs are unaffected. test_accuracy_eval imported into both suites' test modules and inserted into collection order right after the perf-metric stage.

Not included in this PR: the long_isl_osl_stress custom lm-eval task (its gibberish/repetition scoring logic needs its own design pass — deferred).

Test plan

• ./.cvs_venv/bin/python -m pytest cvs/lib/inference/unittests/ --cluster_file /dev/null --config_file /dev/null — 249 passed (2 pre-existing, unrelated failures in test_inferencing_config_loader.py persist with or without this branch)
• Verified vllm/inferencex_atom suite modules and conftest.py collection ordering import cleanly with test_accuracy_eval inserted

[solaiys]

Review (head 66e57af0, base dev/dtni)

Regular review — blocking issues only.

Blocking issues

✅ No blocking issues found.

What I checked:

• Command construction / injection (lm_eval_job.py) — every lm_eval arg is shlex.quoted, and model_args/gen_kwargs are joined into a single quoted token; the find/cat follow-ups quote task_out_dir and result_path. No shell-injection or interpolation gaps. exec_on_head single-result unpacking is guarded ((x,) = out.values() wrapped with a clear RuntimeError), and a missing results*.json is treated as a run failure.
• Result parsing (lm_eval_parsing.py) — pure function; skips alias/non-real numbers (bool/NaN excluded), flattens numeric metrics only. No crash paths.
• Config schema (accuracy_config.py) — Pydantic v2 _Forbid; mutable defaults ({}/[]) are per-instance-copied by Pydantic (not a shared-mutable-state bug), and duplicate task ids are rejected. The new accuracy field defaults to empty via default_factory, so existing configs are unaffected.
• Threshold refactor (vllm_config_loader.py → shared validate_thresholds_cover_sweep) — no regression: import warnings was removed from vllm_config_loader.py and it has no remaining warnings. use, while the shared function imports/uses warnings. New gated_gpu_metrics param defaults to None (backward-compatible for other callers); vllm passes GATED_GPU_METRICS to preserve prior gpu gating. accuracy is excluded from sweep-cell coverage via NON_SWEEP_THRESHOLD_KEYS.
• Lifecycle stage (test_accuracy_eval) — skips on prior failure / missing task, records per-task independently (doesn't set the shared lifecycle.failed), and only gates via evaluate_all when enforce_thresholds is set (record-only otherwise). vllm collection order/parametrization (pytest_generate_tests empty-list auto-skip, conftest rank) is consistent.
• Concurrency — no new threads/async; per-task output dirs are isolated (output_dir/<task.id>), so no shared-file races in normal (non-xdist) collection.

Non-blocking note (not a merge blocker, but worth confirming)

The PR description states the accuracy field was added to InferenceXAtomVariantConfig and test_accuracy_eval was "imported into both suites' test modules." The diff only wires the vllm suite — the inferencex_atom suite (inferencex_atom_single.py / _shared.py / conftest.py) has no accuracy references, and no inferencex_atom config/suite file is changed. This isn't a bug in the code that's present (nothing breaks; accuracy is simply unavailable for inferencex_atom), but the described scope for inferencex_atom isn't delivered here — worth confirming whether that was intended for this PR or a follow-up.

Reviewed 1 PR: 0 blocking issues — clean (with one scope-accuracy note).