fix: Expose image_edit_oai to Agents That Select It Directly

[RutgerLubbers]

Pull Request

Summary

An agent that enables only image_edit_oai cannot edit images. image_edit_oai is a child of the image_gen_oai toolkit: it is not a manifest plugin/toolkit key, so the definition loader drops it, and the shared toolkit constructor still exposes both image tools with no reference image. Selecting the toolkit (image_gen_oai) works because that key is treated as built-in.

This PR makes the directly selected toolkit child work end to end: the definition is emitted, the conversation's images are primed for it, and only the tools the agent selected are loaded.

How it works

Recognition comes from the toolkit parent mapping rather than the manifest, so a child selected on its own is no longer skipped:

-      if (!isBuiltInTool(toolName)) {
+      if (!isBuiltInTool(toolName) && !toolkitParent[toolName]) {
         continue;
       }

An image-edit tool references its input by file_id, and the edit usually targets an image uploaded on an earlier turn. That image is neither a request attachment nor a persisted agent file, so it is absent from every source primeResources already walks. primeResources now primes it through an injected loader, keeping the DB read and file selection in packages/api and applying the same access filtering (filterFiles) and policy screening (screenPersistentFiles) as any other historical file:

+    await applyConversationImageEditFallback();
     const provisionState = await computeProvisionState({ ... });

initialize.ts supplies that loader: it reads the conversation's referenced file ids, hydrates them under the requesting owner, and passes the result into primeResources. The legacy /api loader (handleTools.js) stays a pure adapter — it only reads tool_resources[image_edit].files and exposes the tools the agent selected, expanding toolkit keys so selecting the toolkit still yields its child:

-      const wanted = new Set(tools);
+      const wanted = new Set(tools.flatMap((name) => [name, ...(toolkitExpansion[name] ?? [])]));
       return built.filter((t) => wanted.has(t.name));

Type of change

• Bug fix
• Tests / tooling / CI

Testing

Automated tests:

• packages/api/src/tools/definitions.spec.ts: a toolkit child selected on its own is included (fails without the definition-loader change).
• packages/api/src/agents/resources.test.ts: primeResources primes a conversation's images for an image-edit agent, passes them through filterFiles, skips when the agent cannot edit images or an image is already primed, and ignores records without dimensions.
• api/app/clients/tools/util/handleTools.test.js: an edit-only agent receives only image_edit_oai, and the toolkit still exposes both tools (the second fails without the expansion-aware filter).

All suites pass locally (jest), along with tsc and eslint on the changed files.

Tested environments/configuration:

• Provider/model: a local OpenAI-compatible image endpoint serving a FLUX.2 editing model.
• Database: MongoDB.
• Agents endpoint with an agent whose only tool is image_edit_oai.

Manual verification: upload an image in a conversation, then ask the edit-only agent to modify it, and confirm the model calls image_edit_oai with the real file_id and returns an edited image.

Screenshots / recordings

No user-facing change.

Risk / compatibility

None notable. The definition gate only widens recognition to names already present in the toolkit parent mapping. The conversation-image fallback runs only for agents whose tools include an image tool and only when no image is primed yet; it reuses the existing owner-scoped hydration and access/policy checks, so it adds no new file-loading path.

Checklist

• I reviewed my own changes
• Relevant tests have been added or updated
• Existing relevant tests pass
• The change does not introduce new warnings or errors
• User-facing or complex behavior is documented where necessary
• Required dependency changes have been merged/published
• Required documentation PR: N/A

[Copilot]

🟡 Changes recommended

The fallback bypasses active-branch scoping and the established inspected file-resource pipeline.

1 open finding

• Move file-loading fallback out of legacy API wiring · New

What changed in this PR

Enables agents to select and use image_edit_oai independently.

Changes:

• Recognizes toolkit child tools directly.
• Filters toolkit output to selected tools.
• Adds conversation-image fallback and regression tests.

File	Description
packages/​api/​src/​tools/​definitions.ts	Recognizes toolkit children.
packages/​api/​src/​tools/​definitions.spec.ts	Tests child recognition.
api/​app/​clients/​tools/​util/​handleTools.js	Filters tools and loads image references.
api/​app/​clients/​tools/​util/​handleTools.test.js	Tests toolkit filtering.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[RutgerLubbers]

Moved the fallback out of the legacy /api loader. primeResources now primes the conversation's images behind an injected getConversationImageFiles, applying the existing filterFiles access check and screenPersistentFiles policy screening; initialize.ts supplies the loader (owner-scoped message read + hydration). handleTools.js is back to a pure adapter that only reads tool_resources[image_edit].files. Added resources.test.ts cases that pass a conversationId and exercise the fallback, plus the skip/access paths.