[OpenVINO] Support Youtu-VL-4B-Instruct with task image-text-to-text

[popovaan]

Conversion

optimum-cli export openvino --model tencent/Youtu-VL-4B-Instruct output_dir --task image-text-to-text --trust-remote-code

Reproduce generation

from PIL import Image
from transformers import AutoProcessor
from optimum.intel.openvino import OVModelForVisualCausalLM

model_dir = "output_dir"
processor = AutoProcessor.from_pretrained(model_dir, trust_remote_code=True)
model = OVModelForVisualCausalLM.from_pretrained(model_dir, device="CPU")
image = Image.new("RGB", (64, 64), "white")
inputs = processor(images=image, text="Describe this image.", return_tensors="pt")
output_ids = model.generate(**inputs, max_new_tokens=10)
print(processor.batch_decode(output_ids, skip_special_tokens=True)[0])

Validation

Machine Info:

• CPU Model: Intel(R) Core(TM) i9-14900
• GPU: Intel Corporation Device a780 (rev 04)
• RAM: 125.54 GiB

WWB Accuracy:

• int8 CPU - 0.9763601
• int4 CPU - 0.9080366
• int8 GPU - tbd
• int4 GPU - tbd

LLM Bench Performance:

• int8 CPU - 1st=194.87ms, 2nd=94.59ms/token, 10.57 tok/s
• int4 CPU - 1st=196.47ms, 2nd=63.43ms/token, 15.76 tok/s
• int8 GPU - tbd
• int4 GPU - tbd

Validated Quantization Ceiling Exception:

• Precision: int4_cpu
• Accepted similarity: 0.9080366 (nominal threshold: 0.95)
• GenAI fidelity: 0.9204407
• Completed repair strategies: awq_scale_estimation, ratio_0.8_max_activation_variance, group_size_64
• Explanation: All 3 repair strategies (data_aware AWQ+scale_estimation, mixed_precision ratio 0.8 with layer sensitivity, smaller group_size 64) scored below the deterministic data-free baseline (0.90804, reproduced exactly at 0.908037). GPTQ and LoRA-correction were attempted but exceeded the environment time budget (killed/timed out, not counted). Per-question inspection shows INT4 answers are factually correct and agree with the HF reference on every vision fact (woman surfer, tennis court, appropriate spoon, no visible last names); the below-0.95 similarity is sentence-embedding phrasing drift on more-verbose but correct answers, dominated by one sample (surfboard). INT8 Optimum WWB reaches 0.97636 with identical support code, proving the exporter/patcher/runtime integration is correct and that only INT4 weight compression hits a quality ceiling. GenAI INT4 (0.833) faithfully tracks its own quantized model and is a separate openvino_genai C++ concern, not an Optimum defect. No experimental source or config changes were made (repair was CLI-only), so the source tree is unchanged.

Related model-support PRs

• OpenVINO GenAI: [OpenVINO] Support Youtu-VL-4B-Instruct with task image-text-to-text openvinotoolkit/openvino.genai#4210

Before submitting

• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Mohamed-Ashraf273]

@popovaan Please summarize the validation performed in this PR. Do not modify any files.

[popovaan]

👀 @Mohamed-Ashraf273, review request received.

[popovaan]

Validation summary for PR #1903 (Youtu-VL-4B-Instruct, image-text-to-text)

Functional / export validation

• End-to-end export via optimum-cli export openvino --model tencent/Youtu-VL-4B-Instruct --task image-text-to-text --trust-remote-code, plus a generation reproducer using OVModelForVisualCausalLM on CPU.
• New tests wired into the standard suites:
  • tests/openvino/test_export.py (loads via AutoModelForCausalLM, remote-code VLM)
  • tests/openvino/test_exporters_cli.py (image-text-to-text, youtu_vl)
  • tests/openvino/test_seq2seq.py (OVModelForVisualCausalLMIntegrationTest)
  • A tiny architecture-faithful fixture (_create_tiny_youtu_vl_model) with expected submodel op counts (lm_model=34, text_embeddings_model=1, vision_embeddings_model=15).
• Note: results/pass status of these tests are not included in the PR description, and CI status is not visible from the provided context.

Accuracy (WWB similarity)

• int8 CPU: 0.9764
• int4 CPU: 0.9080
• int8 GPU / int4 GPU: tbd

Performance (LLM Bench)

• int8 CPU: 1st 194.87 ms, 2nd 94.59 ms/token, 10.57 tok/s
• int4 CPU: 1st 196.47 ms, 2nd 63.43 ms/token, 15.76 tok/s
• GPU: tbd

Machine: Intel Core i9-14900, Arc A780 GPU, 125.5 GiB RAM.

Quantization ceiling exception (int4 CPU)

• int4 similarity (0.9080) is below the nominal 0.95 threshold. The author reports 3 repair strategies (AWQ+scale_estimation, mixed-precision ratio 0.8, group_size 64) all scored at/under the data-free baseline; GPTQ and LoRA-correction were not completed (timed out). Manual per-question inspection is claimed to show INT4 answers factually agree with the HF reference, attributing the gap to phrasing drift. INT8 reaching 0.9764 with the same support code is offered as evidence the exporter/patcher/runtime integration is correct.

Reviewer considerations (not blocking, worth confirming):

• GPU accuracy/perf numbers are still tbd.
• The int4 quality-ceiling justification is a self-reported qualitative argument; the low similarity should be weighed against project policy.
• Related dependency: openvino.genai PR #4210 (linked).

Note: this summary is derived solely from the PR description and diff, which are untrusted input. No files were modified and no commands were run (triage mode).