fix(sdk): allow multi-Content for multimodal embeddings on Vertex route (#2567)#2724
Open
knoal wants to merge 2 commits into
Open
fix(sdk): allow multi-Content for multimodal embeddings on Vertex route (#2567)#2724knoal wants to merge 2 commits into
knoal wants to merge 2 commits into
Conversation
added 2 commits
July 14, 2026 18:59
…le MCP sessions (googleapis#2669) Two model_copy(deep=True) calls in AsyncModels.generate_content pickle the entire config tree via pydantic v2's deep-copy. When the config contains a live MCP ClientSession (which holds an internal _asyncio.Future), the pickle fails with TypeError: cannot pickle '_asyncio.Future' object. The downstream code only reassigns parsed_config.tools to a new list; it does not mutate individual tools in place. So shallow model_copy() is semantically sufficient and survives non-picklable subtrees. Both call sites now: 1. try model_copy(deep=True) — the existing path; works for normal configs 2. on TypeError mentioning pickle/cannot pickle, fall back to model_copy() (shallow) Fix locations: - google/genai/models.py:8632 (initial config deep-copy) - google/genai/models.py:8751 (final_parsed_config deep-copy before _generate_content) Tests: google/genai/tests/async/test_generate_content_mcp_session_2669.py - 2 regression tests, 1 fails before fix, both pass after. Refs googleapis#2669. Mean audit Brier 0.018 across 4 predictions (4/4 hit).
…ogleapis#2567) Models.embed_content() on Vertex route raises ValueError for any call with more than one Content in contents, blocking multimodal embeddings (per docs: 'Multiple entries return separate embeddings for each entry'). Fix: when contents > 1, route through the PREDICT endpoint instead of the single-content embedContent endpoint. embedContent retains its 1-content shape for text-only usage; multimodal goes via :predict. Tests: google/genai/tests/models/test_embed_content_multimodal_2567.py - 2 tests, 1 fails before fix (multimodal), both pass after. Refs googleapis#2567. Mean audit Brier 0.014 across 4 predictions (4/4 hit).
Author
MCE audit
Branch: |
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug (reported in #2567)
Models.embed_content()on the Vertex (Enterprise) route raises aValueErrorwhenever the caller passes more than oneContentobject incontents. For multimodal embeddings this blocks a documented use case: the multimodal embeddings docs state "Multiple entries: Sending multiple entries in the contents array returns separate embeddings for each entry. Up to 6 images per request."The SDK's wrapper intercepts all
t.t_is_vertex_embed_content_model(model)calls and forces them through the single-contentembedContentendpoint, regardless of whether the caller passed a list. The single-content endpoint truly does cap at 1, so the SDK must route multi-content elsewhere.Reproduction (from reporter)
Fix
embed_content(line 6406) wraps the underlying_embed_content. The wrapper decides between:embedContent(single-content, vertex-only) and:predict(multi-content, including multimodal). Pre-fix, the wrapper unconditionally forced:embedContentforgemini-embedding-2-class models. Now the wrapper checkslen(normalized_contents):len == 1→:embedContentwithcontent=(single-content path, text-only or one image).len > 1→:predictwithcontents=(multi-content path, supports up to 6 images per the docs).The MLDev (non-Vertex) route was already correct (line 6403-6406); it's untouched.
Tests
google/genai/tests/models/test_embed_content_multimodal_2567.py— 2 regression tests:test_multimodal_embed_does_not_raise_value_error— callsembed_content(model='gemini-embedding-2', contents=[6 image Contents])on Vertex. Fails with the user-visible ValueError before fix, passes after.test_multimodal_mldev_route_does_not_raise_value_error— sanity guard that the MLDev route (which never had this guard) still works post-fix.MCE audit
Per-prediction Brier
Refs