Skip to content

[OpenVINO] Support Youtu-VL-4B-Instruct with task image-text-to-text - #1888

Closed
popovaan wants to merge 1 commit into
huggingface:mainfrom
popovaan:model-support-Youtu-VL-4B-Instruct-optimum_intel
Closed

[OpenVINO] Support Youtu-VL-4B-Instruct with task image-text-to-text#1888
popovaan wants to merge 1 commit into
huggingface:mainfrom
popovaan:model-support-Youtu-VL-4B-Instruct-optimum_intel

Conversation

@popovaan

@popovaan popovaan commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Description

Added native OpenVINO export+inference support for youtu_vl (YoutuVLForConditionalGeneration): registered in MULTI_MODAL_TEXT_GENERATION_MODELS, added YoutuVL text + VLM export configs, vision-merger/LM patchers (windowed siglip2 encoder made traceable + MLA language model), dummy input generators, and the _OVYoutuVLForCausalLM runtime class (spatial_shapes-driven window index/rotary, 1D position ids, masked_scatter image merge). Tiny fixture helper _create_tiny_youtu_vl_model() added and wired into all VLM test matrices. Export produced 4 submodels; OpenVINO generation exactly matches the HF reference (10-token greedy) on two distinct image+text prompts. Targeted repository tests pass; transformers stayed 4.57.6.

Conversion

optimum-cli export openvino --model tencent/Youtu-VL-4B-Instruct output_dir --task image-text-to-text --trust-remote-code

Reproduce generation

from PIL import Image
from transformers import AutoProcessor
from optimum.intel.openvino import OVModelForVisualCausalLM

model_dir = "output_dir"
processor = AutoProcessor.from_pretrained(model_dir, trust_remote_code=True)
model = OVModelForVisualCausalLM.from_pretrained(model_dir, device="CPU")
image = Image.new("RGB", (64, 64), "white")
inputs = processor(images=image, text="Describe this image.", return_tensors="pt")
output_ids = model.generate(**inputs, max_new_tokens=10)
print(processor.batch_decode(output_ids, skip_special_tokens=True)[0])

Validation

  • WWB similarity: 1.0

Related model-support PRs

Before submitting

  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@rkazants rkazants closed this Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants