[OpenVINO] Add image-to-video support for LTX2

[goyaladitya05]

What does this PR do?

This PR adds image-to-video pipeline support for LTX2, enabling users to create videos conditioned on an input image combined with a text prompt. Adding image conditioning provides a strong visual anchor, improving control over composition and style.

Image-to-Video Code and Sample

import torch
import scipy.io.wavfile

from optimum.intel.openvino import OVLTX2ImageToVideoPipeline
from diffusers.utils import export_to_video, load_image

f32_config = {"INFERENCE_PRECISION_HINT": "f32"}

pipe = OVLTX2ImageToVideoPipeline.from_pretrained(
    "ltx2_i2v",# must be exported with --task image-to-video (has vae_encoder)
    device='CPU',
    ov_config=f32_config,
)

image = load_image(
    "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/"
    "resolve/main/in_paint/overture-creations-5sI6fQgYIuo.png"
)

prompt = (
    "A golden retriever in a sunlit bedroom slowly stands up, stretches, yawns, then turns its head "
    "to look at the camera, ears perking up, tail wagging, warm afternoon window light, cinematic, "
    "shallow depth of field, photorealistic, detailed fur. Soft ambient room tone, a gentle bark."
)
negative_prompt = "blurry, low quality, watermark, text, logo, static, motionless"

output = pipe(
    image=image,
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=50,
    height=480,
    width=704,
    num_inference_steps=20,
    guidance_scale=4.0,
    audio_guidance_scale=7.0,
    generator=torch.manual_seed(42),
)

from diffusers.pipelines.ltx2.export_utils import encode_video   # requires: pip install av

video = output.frames[0]
audio = output.audio

encode_video(
    video,
    fps=24,
    audio=audio[0].cpu().float(),
    audio_sample_rate=24000,         
    output_path="ltx2_i2v_fp32.mp4",       # single mp4, audio muxed in
)
print("saved ltx2_i2v.mp4 (video+audio)")

ltx2_i2v_f32.mp4

Text-to-Video Code and Sample

import torch
from optimum.intel.openvino import OVLTX2Pipeline
from diffusers.pipelines.ltx2.export_utils import encode_video   # pip install av

f32_config = {"INFERENCE_PRECISION_HINT": "f32"}

pipe = OVLTX2Pipeline.from_pretrained(
    "ltx2_i2v",          # reuse the i2v export, or "ltx2_t2v"
    device="CPU",
    ov_config=f32_config,
)

prompt = (
    "A golden retriever in a sunlit bedroom slowly stands up, stretches, yawns, then turns its head "
    "to look at the camera, ears perking up, tail wagging, warm afternoon window light, cinematic, "
    "shallow depth of field, photorealistic, detailed fur. Soft ambient room tone, a gentle bark."
)
negative_prompt = "blurry, low quality, watermark, text, logo, static, motionless"

output = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=100,               # must be 8k+1
    height=480,                  # divisible by 32
    width=704,
    num_inference_steps=50,
    guidance_scale=4.0,
    audio_guidance_scale=7.0,
    generator=torch.manual_seed(42),
)

encode_video(
    output.frames[0],
    fps=24,
    audio=output.audio[0].cpu().float(),
    audio_sample_rate=24000,
    output_path="ltx2_t2v_fp32.mp4",
)
print("saved ltx2_t2v_fp32.mp4 (video+audio)")

ltx2_t2v.mp4

Similarity Scores (CPU,fp32)

Text-to-Video hf vs optimum 0.996 (same as main branch, no regressions)
Image-to-Video hf vs optimum 0.991

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[Copilot]

Pull request overview

Adds OpenVINO image-to-video pipeline support for the LTX2 diffusion architecture, enabling video generation conditioned on an input image + prompt, alongside the corresponding exporter plumbing and documentation.

Changes:

• Introduces OVLTX2ImageToVideoPipeline (and shared LTX2 base) plus runtime handling for LTX2 transformer timestep/audio_timestep shapes.
• Extends OpenVINO exporter configuration and input generation to export the additional VAE-encoder component required for image conditioning.
• Updates CLI/exporter tests, diffusion pipeline tests, and inference documentation to cover the new task/model pairing.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/openvino/test_exporters_cli.py	Adds LTX2 image-to-video entry to the exporter CLI coverage matrix.
tests/openvino/test_diffusion.py	Extends image-to-video pipeline tests to include LTX2 (diffusers >= 0.38.0).
optimum/intel/utils/dummy_openvino_and_diffusers_objects.py	Adds dummy stub for OVLTX2ImageToVideoPipeline when optional deps are missing.
optimum/intel/openvino/modeling_diffusion.py	Implements OVLTX2ImageToVideoPipeline, shared LTX2 base wiring (incl. VAE encoder), and transformer timestep/audio_timestep handling.
optimum/intel/openvino/init.py	Exposes OVLTX2ImageToVideoPipeline from the OpenVINO module when diffusers is available.
optimum/intel/init.py	Adds the new pipeline to lazy import structure and dummy fallbacks.
optimum/exporters/openvino/model_configs.py	Registers image-to-video diffusers task mapping and updates LTX2 transformer input signatures (timestep/audio_timestep).
optimum/exporters/openvino/input_generators.py	Updates dummy inputs for LTX2 VAE encoder and transformer (timestep/audio_timestep).
optimum/exporters/openvino/convert.py	Exports an additional LTX2 VAE encoder submodel needed for image conditioning.
docs/source/openvino/inference.mdx	Documents the new OpenVINO pipeline class ↔ task mapping for LTX2 image-to-video.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[goyaladitya05]

Hi @anatyrova,
I've added the WWB scores and sample outputs to the PR description. The text-to-video scores match the main branch, confirming that this PR introduces no regressions for existing code. Please take a look whenever you have time.

[rkazants]

na

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[rkazants]

@goyaladitya05, please put scripts and results into PR description. Avoid links that are not convinient.

[rkazants]

also check that common class OVPipelineForImage2Video is working for your implementation

[goyaladitya05]

I checked both OVPipelineForImage2Video and OVPipelineForText2Video. Both work fine as before.

[goyaladitya05]

@goyaladitya05, please put scripts and results into PR description. Avoid links that are not convinient.

@rkazants I've updated the PR desciption.

[goyaladitya05]

For the CI failures, none of them are releted to my changes.

[anatyrova]

@goyaladitya05 is it right that vae encoder exports for both t2v and i2v (as it should)?

also dw about CI, i will reload it and it should pass eventually.

[goyaladitya05]

@goyaladitya05 is it right that vae encoder exports for both t2v and i2v (as it should)?

@anatyrova yes it does. Similar to how we did for LTX-Video.

[anatyrova]

is this comment valid, since you said vae encoder gets exported for both tasks?

[goyaladitya05]

I have corrected it. It is exported for both tasks, but only loaded for image-to-video.