Skip to content

Commit cf45453

Browse files
jerm-droclaude
jerm-dro
and
claude
committed
Add opt-in code mode decorator for vMCP
Agents today make one tool call per backend tool with model inference between each, so multi-service workflows cost many round-trips and tokens. Code mode lets an agent submit one Starlark script (loops, conditionals, parallel() fan-out) that calls many tools server-side and returns a single aggregated result via the execute_tool_script virtual tool. Implement it as a core.VMCP decorator rather than the parked HTTP middleware (#4821): the engine in pkg/script is stateless, so it belongs in the core, and wrapping there places it BELOW any Serve-layer optimizer — ListTools advertises execute_tool_script alongside backend tools, so the optimizer indexes and routes to it like any other tool. A script's inner calls route back through inner.CallTool, so the core admission seam authorizes each one by its real name; a script can only reach tools the caller already may use. - pkg/vmcp/codemode: NewDecorator (ListTools/LookupTool/CallTool overrides, per-call timeout, vmcp<->mcp-go result conversion) and FromConfig - pkg/vmcp/config: codeMode block on the shared CRD/file config type - server.Config.CodeModeConfig wired in New, just after core.New - cli/serve.go populates it from the parsed config - regenerated deepcopy, CRD manifests, and CRD API docs; example manifest Part of #4742 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 37a188f commit cf45453

13 files changed

Lines changed: 1082 additions & 0 deletions

File tree

‎deploy/charts/operator-crds/files/crds/toolhive.stacklok.dev_virtualmcpservers.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1226,6 +1226,43 @@ spec:
12261226
- url
12271227
type: object
12281228
type: array
1229+
codeMode:
1230+
description: |-
1231+
CodeMode configures vMCP code mode: server-side execution of Starlark scripts that
1232+
orchestrate multiple backend tool calls in a single request via the execute_tool_script
1233+
virtual tool. When enabled, execute_tool_script is advertised alongside the backend
1234+
tools; a script's inner tool calls are authorized individually, so a script can only
1235+
reach tools the caller is already permitted to use. Disabled by default.
1236+
properties:
1237+
enabled:
1238+
description: |-
1239+
Enabled turns code mode on. When false (the default), execute_tool_script is not
1240+
advertised in tools/list and scripts cannot be executed.
1241+
type: boolean
1242+
parallelMaxConcurrency:
1243+
default: 10
1244+
description: |-
1245+
ParallelMaxConcurrency caps the number of goroutines a script's parallel() builtin
1246+
may run concurrently. Defaults to 10 if unset or zero.
1247+
minimum: 1
1248+
type: integer
1249+
stepLimit:
1250+
default: 100000
1251+
description: |-
1252+
StepLimit is the maximum number of Starlark execution steps per script. It bounds
1253+
runaway loops and computation. Defaults to 100000 if unset or zero.
1254+
format: int64
1255+
minimum: 1
1256+
type: integer
1257+
toolCallTimeout:
1258+
default: 30s
1259+
description: |-
1260+
ToolCallTimeout bounds each individual backend tool call made from within a script.
1261+
A call exceeding it is cancelled and surfaces a timeout error to the script.
1262+
Defaults to 30s if unset.
1263+
pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$
1264+
type: string
1265+
type: object
12291266
compositeToolRefs:
12301267
description: |-
12311268
CompositeToolRefs references VirtualMCPCompositeToolDefinition resources
@@ -4277,6 +4314,43 @@ spec:
42774314
- url
42784315
type: object
42794316
type: array
4317+
codeMode:
4318+
description: |-
4319+
CodeMode configures vMCP code mode: server-side execution of Starlark scripts that
4320+
orchestrate multiple backend tool calls in a single request via the execute_tool_script
4321+
virtual tool. When enabled, execute_tool_script is advertised alongside the backend
4322+
tools; a script's inner tool calls are authorized individually, so a script can only
4323+
reach tools the caller is already permitted to use. Disabled by default.
4324+
properties:
4325+
enabled:
4326+
description: |-
4327+
Enabled turns code mode on. When false (the default), execute_tool_script is not
4328+
advertised in tools/list and scripts cannot be executed.
4329+
type: boolean
4330+
parallelMaxConcurrency:
4331+
default: 10
4332+
description: |-
4333+
ParallelMaxConcurrency caps the number of goroutines a script's parallel() builtin
4334+
may run concurrently. Defaults to 10 if unset or zero.
4335+
minimum: 1
4336+
type: integer
4337+
stepLimit:
4338+
default: 100000
4339+
description: |-
4340+
StepLimit is the maximum number of Starlark execution steps per script. It bounds
4341+
runaway loops and computation. Defaults to 100000 if unset or zero.
4342+
format: int64
4343+
minimum: 1
4344+
type: integer
4345+
toolCallTimeout:
4346+
default: 30s
4347+
description: |-
4348+
ToolCallTimeout bounds each individual backend tool call made from within a script.
4349+
A call exceeding it is cancelled and surfaces a timeout error to the script.
4350+
Defaults to 30s if unset.
4351+
pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$
4352+
type: string
4353+
type: object
42804354
compositeToolRefs:
42814355
description: |-
42824356
CompositeToolRefs references VirtualMCPCompositeToolDefinition resources

‎deploy/charts/operator-crds/templates/toolhive.stacklok.dev_virtualmcpservers.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1229,6 +1229,43 @@ spec:
12291229
- url
12301230
type: object
12311231
type: array
1232+
codeMode:
1233+
description: |-
1234+
CodeMode configures vMCP code mode: server-side execution of Starlark scripts that
1235+
orchestrate multiple backend tool calls in a single request via the execute_tool_script
1236+
virtual tool. When enabled, execute_tool_script is advertised alongside the backend
1237+
tools; a script's inner tool calls are authorized individually, so a script can only
1238+
reach tools the caller is already permitted to use. Disabled by default.
1239+
properties:
1240+
enabled:
1241+
description: |-
1242+
Enabled turns code mode on. When false (the default), execute_tool_script is not
1243+
advertised in tools/list and scripts cannot be executed.
1244+
type: boolean
1245+
parallelMaxConcurrency:
1246+
default: 10
1247+
description: |-
1248+
ParallelMaxConcurrency caps the number of goroutines a script's parallel() builtin
1249+
may run concurrently. Defaults to 10 if unset or zero.
1250+
minimum: 1
1251+
type: integer
1252+
stepLimit:
1253+
default: 100000
1254+
description: |-
1255+
StepLimit is the maximum number of Starlark execution steps per script. It bounds
1256+
runaway loops and computation. Defaults to 100000 if unset or zero.
1257+
format: int64
1258+
minimum: 1
1259+
type: integer
1260+
toolCallTimeout:
1261+
default: 30s
1262+
description: |-
1263+
ToolCallTimeout bounds each individual backend tool call made from within a script.
1264+
A call exceeding it is cancelled and surfaces a timeout error to the script.
1265+
Defaults to 30s if unset.
1266+
pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$
1267+
type: string
1268+
type: object
12321269
compositeToolRefs:
12331270
description: |-
12341271
CompositeToolRefs references VirtualMCPCompositeToolDefinition resources
@@ -4280,6 +4317,43 @@ spec:
42804317
- url
42814318
type: object
42824319
type: array
4320+
codeMode:
4321+
description: |-
4322+
CodeMode configures vMCP code mode: server-side execution of Starlark scripts that
4323+
orchestrate multiple backend tool calls in a single request via the execute_tool_script
4324+
virtual tool. When enabled, execute_tool_script is advertised alongside the backend
4325+
tools; a script's inner tool calls are authorized individually, so a script can only
4326+
reach tools the caller is already permitted to use. Disabled by default.
4327+
properties:
4328+
enabled:
4329+
description: |-
4330+
Enabled turns code mode on. When false (the default), execute_tool_script is not
4331+
advertised in tools/list and scripts cannot be executed.
4332+
type: boolean
4333+
parallelMaxConcurrency:
4334+
default: 10
4335+
description: |-
4336+
ParallelMaxConcurrency caps the number of goroutines a script's parallel() builtin
4337+
may run concurrently. Defaults to 10 if unset or zero.
4338+
minimum: 1
4339+
type: integer
4340+
stepLimit:
4341+
default: 100000
4342+
description: |-
4343+
StepLimit is the maximum number of Starlark execution steps per script. It bounds
4344+
runaway loops and computation. Defaults to 100000 if unset or zero.
4345+
format: int64
4346+
minimum: 1
4347+
type: integer
4348+
toolCallTimeout:
4349+
default: 30s
4350+
description: |-
4351+
ToolCallTimeout bounds each individual backend tool call made from within a script.
4352+
A call exceeding it is cancelled and surfaces a timeout error to the script.
4353+
Defaults to 30s if unset.
4354+
pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$
4355+
type: string
4356+
type: object
42834357
compositeToolRefs:
42844358
description: |-
42854359
CompositeToolRefs references VirtualMCPCompositeToolDefinition resources

‎docs/operator/crd-api.md‎

Lines changed: 23 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/operator/virtual-mcps/vmcp_codemode.yaml‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Example: VirtualMCPServer with Code Mode enabled
2+
#
3+
# Code mode adds a single virtual tool, execute_tool_script, to the aggregated
4+
# tool list. Instead of issuing one tools/call per backend tool (with model
5+
# inference between each), an agent submits a Starlark script that calls multiple
6+
# tools server-side — with loops, conditionals, and parallel() fan-out — and
7+
# receives a single aggregated result. This collapses multi-tool workflows into
8+
# one round-trip and cuts token usage.
9+
#
10+
# Security: a script can only call tools the caller is already permitted to use.
11+
# Each inner tool call is authorized individually through the same admission path
12+
# as a normal tools/call, so code mode never widens reachability — it only changes
13+
# how many round-trips a workflow takes.
14+
#
15+
# Code mode composes with the optimizer: when both are enabled, execute_tool_script
16+
# is indexed alongside the backend tools and can be discovered via find_tool and
17+
# invoked via call_tool. This example keeps it simple (no optimizer).
18+
#
19+
# Example script an agent might submit to execute_tool_script:
20+
#
21+
# results = parallel([
22+
# lambda: osv_query_vulnerability(package="lodash", version="4.17.20"),
23+
# lambda: osv_query_vulnerability(package="express", version="4.17.1"),
24+
# ])
25+
# vulnerable = [r for r in results if r.get("vulns")]
26+
# return {"checked": len(results), "vulnerable": vulnerable}
27+
#
28+
# Usage:
29+
# kubectl apply -f vmcp_codemode.yaml
30+
31+
---
32+
# Step 1: Create MCPGroup
33+
apiVersion: toolhive.stacklok.dev/v1beta1
34+
kind: MCPGroup
35+
metadata:
36+
name: codemode-services
37+
namespace: default
38+
spec:
39+
description: Backend services for a code-mode-enabled VirtualMCPServer
40+
41+
---
42+
# Step 2a: MCPServer backend - fetch (URL content fetching)
43+
apiVersion: toolhive.stacklok.dev/v1beta1
44+
kind: MCPServer
45+
metadata:
46+
name: fetch
47+
namespace: default
48+
spec:
49+
groupRef:
50+
name: codemode-services
51+
image: ghcr.io/stackloklabs/gofetch/server
52+
transport: streamable-http
53+
proxyPort: 8080
54+
mcpPort: 8080
55+
resources:
56+
limits:
57+
cpu: "100m"
58+
memory: "128Mi"
59+
requests:
60+
cpu: "50m"
61+
memory: "64Mi"
62+
63+
---
64+
# Step 2b: MCPServer backend - osv (OSV vulnerability database)
65+
apiVersion: toolhive.stacklok.dev/v1beta1
66+
kind: MCPServer
67+
metadata:
68+
name: osv
69+
namespace: default
70+
spec:
71+
groupRef:
72+
name: codemode-services
73+
image: ghcr.io/stackloklabs/osv-mcp/server:0.0.7
74+
transport: streamable-http
75+
proxyPort: 8080
76+
resources:
77+
limits:
78+
cpu: "100m"
79+
memory: "128Mi"
80+
requests:
81+
cpu: "50m"
82+
memory: "64Mi"
83+
84+
---
85+
# Step 3: Create VirtualMCPServer with code mode enabled
86+
apiVersion: toolhive.stacklok.dev/v1beta1
87+
kind: VirtualMCPServer
88+
metadata:
89+
name: codemode-vmcp
90+
namespace: default
91+
spec:
92+
groupRef:
93+
name: codemode-services
94+
config:
95+
# Aggregation: prefix strategy prevents tool name conflicts. The prefixed
96+
# names (e.g. osv_query_vulnerability) are exactly what scripts call.
97+
aggregation:
98+
conflictResolution: prefix
99+
conflictResolutionConfig:
100+
prefixFormat: "{workload}_"
101+
102+
# Code mode: opt in and tune the execution limits. All fields below are
103+
# optional and shown with their defaults.
104+
codeMode:
105+
enabled: true
106+
stepLimit: 100000 # max Starlark execution steps per script
107+
parallelMaxConcurrency: 10 # max goroutines parallel() may run at once
108+
toolCallTimeout: 30s # per-inner-tool-call deadline
109+
110+
# Incoming authentication (client -> vMCP). Anonymous for easy local testing.
111+
incomingAuth:
112+
type: anonymous
113+
authzConfig:
114+
type: inline
115+
inline:
116+
policies:
117+
- 'permit(principal, action, resource);'
118+
119+
# Outgoing authentication (vMCP -> backends). Discovered mode auto-discovers
120+
# auth from the backend MCPServers.
121+
outgoingAuth:
122+
source: discovered

‎pkg/vmcp/cli/serve.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ import (
3838
vmcpauth "github.com/stacklok/toolhive/pkg/vmcp/auth"
3939
authfactory "github.com/stacklok/toolhive/pkg/vmcp/auth/factory"
4040
vmcpclient "github.com/stacklok/toolhive/pkg/vmcp/client"
41+
"github.com/stacklok/toolhive/pkg/vmcp/codemode"
4142
"github.com/stacklok/toolhive/pkg/vmcp/config"
4243
"github.com/stacklok/toolhive/pkg/vmcp/discovery"
4344
"github.com/stacklok/toolhive/pkg/vmcp/health"
@@ -433,6 +434,7 @@ func Serve(ctx context.Context, cfg ServeConfig) error {
433434
Watcher: nil, // set below if backendWatcher is non-nil
434435
StatusReporter: statusReporter,
435436
OptimizerConfig: optCfg,
437+
CodeModeConfig: codemode.FromConfig(vmcpCfg.CodeMode),
436438
SessionFactory: sessionFactory,
437439
SessionStorage: vmcpCfg.SessionStorage,
438440
// Core collaborators: server.New routes through core.New + Serve, so the core

0 commit comments

Comments
 (0)