feat: complete Kthena inference service workflows - #495
Conversation
There was a problem hiding this comment.
Pull request overview
本 PR 面向 Crater 的“在线模型部署/推理服务”能力补齐了 Kthena v1（ModelBooster 资源模型）适配与端到端闭环：后端引入特性开关与统一 Workload API（Volcano Job + Kthena ModelBooster），前端新增模型部署入口、列表页与管理员开关配置，并补齐必要的 RBAC、持久化表与迁移测试。
Changes:
- 新增
kthenaInferenceEnabled系统配置开关（默认关闭），并在前后端同时做“关闭即拒绝访问”的防护。 - 引入统一 Workload 列表/Facet API，将 Volcano 作业与 Kthena ModelBooster 以统一表格行返回，同时限制模型部署不参与作业批量删除。
- 前端新增模型部署相关 API client、路由与页面（含资源占用 Grafana iframe），并补齐多语言文案。
Reviewed changes
Copilot reviewed 44 out of 47 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/services/client.ts | 扩展 v1 POST 支持 options，并新增 v1 PATCH helper |
| frontend/src/services/api/vcjob.ts | 增加统一 Workload 类型/枚举与新接口封装 |
| frontend/src/services/api/system-config.ts | 新增 Kthena 推理开关状态的获取/设置 API |
| frontend/src/services/api/inference.ts | 新增 Kthena 推理服务、模板、会话、转发调用等 API 定义 |
| frontend/src/routeTree.gen.ts | 注入模型部署相关路由（生成文件） |
| frontend/src/routes/portal/route.tsx | 侧边栏按开关动态展示“模型部署”入口 |
| frontend/src/routes/portal/overview/index.tsx | 总览页从 Job 切换为 Workload（支持模型部署行） |
| frontend/src/routes/portal/inference-services/route.tsx | 路由守卫：开关关闭/不可用时重定向 |
| frontend/src/routes/portal/inference-services/index.tsx | 新增模型部署列表页与删除交互 |
| frontend/src/routes/portal/inference-services/-components/resource-usage.tsx | Grafana iframe 资源占用面板（按 Pod 选择） |
| frontend/src/routes/admin/more/index.tsx | 管理员“更多设置”中增加开关配置与请求联动 |
| frontend/src/routes/admin/more/-components/kthena-inference-settings.tsx | 新增开关设置卡片组件 |
| frontend/src/i18n/locales/zhCN/translation.json | 新增 Kthena/开关相关中文翻译 |
| frontend/src/i18n/locales/ko/translation.json | 新增 Kthena/开关相关韩文翻译 |
| frontend/src/i18n/locales/ja/translation.json | 新增 Kthena/开关相关日文翻译 |
| frontend/src/i18n/locales/enUS/translation.json | 新增 Kthena/开关相关英文翻译 |
| frontend/src/components/query-table/remote.tsx | RemoteDataTable 支持按行禁用选择（避免误批量操作） |
| frontend/src/components/job/overview/custom-jobs.tsx | 作业概览表升级为 Workload，并禁止模型部署进入批量删除 |
| frontend/src/components/form/env-form-field.tsx | 修复按钮默认 submit：显式 type="button" |
| frontend/src/components/badge/kthena-status-badge.tsx | 新增 Kthena 状态 Badge 与状态归一化展示 |
| frontend/src/components/badge/job-type-badge.tsx | 增加“模型部署”JobType 展示 |
| charts/crater/templates/crater-backend/serviceaccount.yaml | 增加 Kthena/Volcano serving 相关 RBAC 权限 |
| backend/internal/service/config_service.go | 增加 Kthena 推理开关读取/设置 |
| backend/internal/service/config_service_test.go | 增加开关默认值与更新测试 |
| backend/internal/handler/vcjob/workload.go | 新增统一 Workload 列表/Facet 处理与 ModelBooster 转换逻辑 |
| backend/internal/handler/vcjob/workload_test.go | 覆盖 ModelBooster 转换、过滤与开关屏蔽逻辑 |
| backend/internal/handler/vcjob/vcjob.go | 注册统一 Workload 路由（protected/admin） |
| backend/internal/handler/vcjob/list.go | job list query 增加 workload_kind 过滤支持并校验枚举 |
| backend/internal/handler/system_config.go | 新增 Kthena 推理开关状态 GET/PUT 路由与 handler |
| backend/internal/handler/system_config_test.go | 覆盖开关状态路由测试 |
| backend/internal/handler/kthena_inference_template.go | 新增用户私有推理模板 CRUD（强隔离） |
| backend/internal/handler/kthena_inference_template_test.go | 覆盖模板私有性与校验逻辑 |
| backend/internal/handler/inference_service_test.go | 覆盖 v1 请求校验、资源构建、网关/诊断等核心逻辑 |
| backend/internal/handler/inference_conversation_test.go | 覆盖会话持久化、隔离、幂等 turn 等逻辑 |
| backend/dao/model/system_config.go | 新增系统配置 key：ENABLE_KTHENA_INFERENCE |
| backend/dao/model/kthena_inference_template.go | 新增推理模板表模型 |
| backend/dao/model/kthena_chat.go | 新增会话/消息表模型（含幂等 turn 约束字段） |
| backend/cmd/gorm-gen/models/migrate.go | 增加新表迁移与回滚，并加入 AutoMigrate 列表 |
| backend/cmd/gorm-gen/models/migrate_test.go | 覆盖迁移与回滚的幂等性测试 |
| backend/cmd/gorm-gen/curd/generate.go | 将新模型纳入 gorm-gen 代码生成清单 |
| to="/portal/inference-services/$name" | ||
| params={{ name: workload.jobName }} | ||
| preload="intent" | ||
| className="text-primary hover:underline" | ||
| title={`查看模型部署 ${workload.name}`} |
| { | ||
| value: 'model-deployment', | ||
| label: '模型部署', | ||
| }, |
| - apiGroups: ["workload.serving.volcano.sh"] | ||
| resources: ["modelboosters", "modelservings", "autoscalingpolicies"] | ||
| verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] | ||
| - apiGroups: ["networking.serving.volcano.sh"] | ||
| resources: ["modelroutes", "modelservers"] | ||
| verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83f6688280
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return | ||
| } | ||
|
|
||
| rawResp, err := mgr.proxyKthenaRouter(c.Request.Context(), c.Request.Method, targetPath, body, c.Request.Header) |
There was a problem hiding this comment.
When an OpenAI-compatible request sets stream: true, DoRaw collects the entire router response into a byte slice before the handler writes anything to the client. The caller therefore receives no incremental SSE tokens, and line 616 subsequently labels the completed event stream as application/json. This breaks streaming clients and can make long generations appear stalled or hit timeouts; the proxy needs to forward the upstream response body and content type as a stream.
Useful? React with 👍 / 👎.
| resputil.HandleError(c, bizerr.BadRequest.InvalidRequest.Wrap(err, "failed to read request body")) | ||
| return | ||
| } | ||
| body, err = withDefaultModel(body, servedModelFromModelBooster(obj)) |
There was a problem hiding this comment.
When a caller omits model, this supplies the vLLM served-model-name, but Kthena routes by the ModelBooster/ModelRoute name: buildKthenaAccess exposes booster.GetName() as the route model, and the persisted conversation path deliberately sends scope.RouteModelName for the same reason. Deployments whose service name differs from their served-model name—the default for many external models—will therefore return a route-not-found error for otherwise valid proxied requests that rely on this advertised default.
Useful? React with 👍 / 👎.
| rawCompletion, err := mgr.proxyKthenaRouter( | ||
| c.Request.Context(), http.MethodPost, "v1/chat/completions", body, c.Request.Header, | ||
| ) |
There was a problem hiding this comment.
When a client retries the same clientTurnId while the original request is still in flight, both requests can pass findTurn before either persists anything and both execute this expensive router call. appendTurn deduplicates only afterward, so the API's retry idempotency still consumes inference capacity twice and may discard a different second completion. Claim or lock the turn ID before calling the router, or make concurrent retries wait for the in-progress result.
Useful? React with 👍 / 👎.
| return applyJobFilters(ctx, scope, &jobRequest, defaultDays). | ||
| Preload(query.Job.User). | ||
| Preload(query.Job.Account). | ||
| Find() |
There was a problem hiding this comment.
The unified list fetches all matching job rows with Find() and only paginates the merged slice afterward. In the batch-workload view the self endpoint uses a default of days=-1, so every table refresh loads, preloads, converts, and sorts a user's entire job history just to return one page; users with large histories can make this endpoint increasingly slow and memory-heavy. Preserve database pagination or merge bounded, sorted result sets instead of materializing the complete job set.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83f6688280
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if servedModel != "" && (name == servedModel || strings.Contains(name, sanitizeKubeName(servedModel))) { | ||
| return true |
There was a problem hiding this comment.
When two tenants have resources whose names share a common served-model fragment, this namespace-wide matcher treats the other tenant's resource as related; for example, a service serving qwen will accept an unrelated other-qwen-route. That resource is then included in the response and, for a matched ModelServing, its pod metadata and diagnostics are also collected, producing incorrect status/access data and potentially exposing another tenant's runtime details. Restrict association to an exact booster label or owner reference rather than a substring match.
Useful? React with 👍 / 👎.
| for i := range pod.Status.ContainerStatuses { | ||
| status := &pod.Status.ContainerStatuses[i] | ||
| logTail := mgr.containerLogTail(ctx, pod, status.Name, status.RestartCount) | ||
| diagnostics = append(diagnostics, diagnosticsFromContainerStatus(pod, status, false, logTail)...) |
There was a problem hiding this comment.
For every healthy running container, this calls containerLogTail and opens a Kubernetes pod-log stream before diagnosticsFromContainerStatus determines that no diagnostic is needed. Both service list and detail conversion execute this path, while the new list page polls every 10 seconds, so installations with multiple deployments repeatedly read logs for every container and can substantially increase API-server load and response latency. Inspect the container state first and fetch logs only for waiting or failed containers.
Useful? React with 👍 / 👎.
背景与目标
Kthena 已升级到 v1 ModelBooster 资源模型。本 PR 将 Crater 的模型部署能力按该版本重新适配，并把部署、运行、调用、监测、会话历史和作业视图串成完整闭环。
模型部署默认保持关闭；只有管理员在“更多设置”中显式开启后，用户侧菜单、路由和接口才会生效，避免未完成配置的集群直接暴露该功能。
后端实现
Kthena v1 与部署生命周期
ModelBooster生成和解析部署资源，固定使用 Volcano 调度器，并兼容 vLLM 单服务端 worker 形态。requests与limits，保证 CPU、内存与 GPU 资源能被调度和资源页正确识别。权限、开关与数据持久化
kthenaInferenceEnabled系统配置，默认false；功能关闭时，Kthena 模型部署相关路由会拒绝访问。作业与门户整合
前端体验
创建与管理
详情、调用与观测
兼容性与安全边界
验证
pnpm run lintpnpm buildpython3 hack/format_translation.py --checkgolangci-lint全量与增量检查页面截图
以下截图仅作为 PR 描述附件，未提交到仓库。
模型部署列表与创建
部署详情与调用
资源观测与诊断
作业、门户概览与管理开关