Skip to content

feat: add Kubernetes deployment support with a Helm chart - #1838

Draft
eetoc wants to merge 4 commits into
mainfrom
feat/k8s-helm-deployment
Draft

eetoc wants to merge 4 commits into
mainfrom
feat/k8s-helm-deployment

Conversation

@eetoc

@eetoc eetoc commented Oct 8, 2026

Copy link
Copy Markdown

变更类型

  • 新功能
  • 文档更新
  • 测试用例
  • 其他 (后端增加若干部署相关开关)

变更内容

  1. 后端新增一组默认值等于现状的开关与生命周期支持,供 Kubernetes 部署使用,docker compose 行为不变:

    • /healthz 与 /readyz 走独立端口(health.port,默认关闭),探针流量不进入 API 路由,也不经过鉴权、session 与请求日志中间件
    • 收到 SIGTERM/SIGINT 时优雅关闭并排空在途请求,同时停止 cron 调度与 telemetry 上报
    • SQL 迁移与 raglite 建库可从 Pod 启动路径剥离(pg.auto_migrate、pg.create_raglite_db),改由一个 Job 执行一次;并容忍并发建库
    • Caddy admin 端点支持 unix / http / tcp,reverse_proxy 上游可配置(caddy.upstreams),配置重载时重述 admin 监听
    • 没有知识库时推送空路由表,删除最后一个知识库不再在 Caddy 里留下残留规则
    • 容忍 MinIO bucket 与 NATS stream 的并发创建;telemetry 可关闭(telemetry.enabled);自签证书生成可跳过(setup.init_cert)
  2. 修正管理端 nginx 的流式响应列表。原先只覆盖两个端点,且混入了返回普通 JSON 的 /share/v1/chat/widget/search,同时漏掉了会流式的 /share/v1/chat/completions(OpenAI 风格流式因此被缓冲)。现与后端实际设置 text/event-stream 的六个端点对齐;并修正 EXPOSE 为 server.conf 实际监听的 8080。

  3. 新增 helm/panda-wiki/。部署 api、consumer、管理端 nginx、wiki 站点、Caddy、迁移 Job,以及 raglite 与 crawler 两个闭源镜像;PostgreSQL、Redis、NATS、MinIO、Qdrant 可作为单副本 StatefulSet 内置部署,也可通过 *.external.* 指向外部服务。Service 名与后端硬编码名一致,因此后端源码无需改动(代价是一个 namespace 对应一个 release,已在文档说明)。

测试情况

  • 已本地测试
    • go build ./...、go vet ./...、golangci-lint run(改动包 0 issues)、go test(含 -race)
    • helm lint;默认值渲染后 kubectl apply --dry-run=server 通过 33 个对象;四种 caddy.exposure 与外部中间件场景均可渲染
  • 已添加测试用例
    • repo/pg:caddyAdminEndpoint 的表驱动测试(含默认 unix socket 路径的兼容性用例)
    • health:探针聚合、超时上界与并发注册(-race 下通过)
  • 已在真实集群验证(单节点 k8s v1.30)
    • 12 个工作负载全部就绪;迁移 Job 建出 32 张表
    • /readyz 返回各依赖状态;知识库创建 → Caddy 路由下发(上游三个地址正确)→ 从集群外访问返回 200;wiki 站点静态资源(/panda-wiki-app-assets/...)全部 200,无需网关 rewrite
    • 删除最后一个知识库后 Caddy 路由被清空;Caddy 单独重启后路由保留(--resume)
    • 管理端的 Caddy admin API 在 NodePort/LoadBalancer 下不对外暴露(独立 ClusterIP Service)

其他说明

兼容性:所有新增开关默认值等于现有行为,compose 部署不受影响。

两点与已发布镜像相关的说明(均非本 PR 引入,但用官方镜像测试时会遇到):

  1. 建知识库会报 no default embedding model available —— raglite 需要先有默认 embedding 模型。这是既有产品行为。
  2. 已发布镜像是本分支改动之前的构建,因此 /healthz、Caddy over TCP 等在上述镜像上不生效;要验证这些能力需用本分支重新构建 api 与 consumer 镜像。

caddy.exposure 默认 hostNetwork,与 compose 一致,以保留「知识库可使用任意端口」这一产品能力。Caddy 启动时配置为空(镜像内 Caddyfile 全部被注释),不会绑定 80/443,因此默认安装在任何集群都能成功;只有在节点上已被占用的端口上发布知识库时会失败(Caddy 整份拒绝配置,控制台提示保存配置失败)。若不希望依赖节点端口,可改为 NodePort,代价是自定义端口失效。

顺带修复的两个既有缺陷:handler/mq/cron.go 中 cron 实例被局部变量遮蔽导致无法停止;Caddy 同步路径创建临时 transport 后未释放连接。

已知的既有问题(与本 PR 无关):pkg/bot/discord 的 TestDiscord 结尾为无条件的 select {},会永久阻塞,因此 go test ./... 无法跑完,需要排除该包。

eetoc added 4 commits October 8, 2026 06:47
Adds the switches and lifecycle hooks a Kubernetes deployment needs, keeping the
docker compose behaviour as the default everywhere: every new option defaults to
what the code already did.

- serve /healthz and /readyz on a separate port (health.port, off by default) so
  probe traffic stays out of the API router and never runs through the auth,
  session or request-logging middleware
- drain on SIGTERM and SIGINT instead of dying on the signal, and stop the cron
  scheduler and the telemetry reporter on the way out. The cron instance had been
  lost to a shadowed variable, so it could not be stopped at all
- move the SQL migrations and the raglite database creation out of pod startup
  behind pg.auto_migrate and pg.create_raglite_db, so they can run as one job
  instead of once per replica, and treat a concurrent database creation as success
- reach the Caddy admin endpoint over unix, http or tcp rather than a unix socket
  only, make the reverse_proxy upstreams configurable instead of deriving them
  from a subnet prefix, and restate the admin listener when the configuration is
  reloaded
- push an empty routing table when no knowledge base is left, so deleting the last
  one no longer leaves its rule in Caddy
- tolerate a MinIO bucket and a NATS stream being created concurrently
- make telemetry optional and add setup.init_cert, which skips the self-signed
  certificate that only the bundled admin nginx consumed through a shared volume
- stop building a transport for every Caddy sync without releasing its connections
The unbuffered list covered two endpoints and had drifted both ways:
/share/v1/chat/widget/search answers with plain JSON, while
/share/v1/chat/completions streams when the request asks for it and was missing,
so OpenAI-style streaming arrived in a single chunk once generation finished.

List the six endpoints that actually answer with text/event-stream, and correct
EXPOSE to 8080, the port server.conf listens on.
Deploys what the docker compose file runs — api, consumer, admin nginx, wiki app,
caddy, a migration job, and the raglite and crawler images — plus optionally
PostgreSQL, Redis, NATS, MinIO and Qdrant as single-replica StatefulSets. Pointing
any of those at a managed service is a values change.

- Services are named exactly as the backend hardcodes them, so the Go source needs
  no change; the cost, one release per namespace, is documented
- the api entrypoint is overridden and migrations are off on the serving pods. A
  Helm hook job runs them instead, after waiting for PostgreSQL, Redis, NATS and
  MinIO, because the migration binary builds its whole dependency graph on start
- the backend config.yml and the admin nginx config are rendered from values, which
  is also what applies the streaming fix to already-published images
- the Caddy admin API sits on a ClusterIP Service of its own, so choosing NodePort
  or LoadBalancer exposure never publishes an endpoint that accepts unauthenticated
  config writes
- the Secret given as secrets.existingSecret is validated on install and upgrade,
  because one of its failure modes is silent: the backend only creates the admin
  account when ADMIN_PASSWORD is non-empty, so a key that exists but is empty would
  start a console with no way in and no error anywhere. A missing key, or an empty
  admin-password, now fails while rendering, before anything is created
- secrets are generated once and kept across upgrades, and rendering without a
  cluster connection fails with instructions rather than inventing new values on
  every render
- caddy.resume is on, so a Caddy-only restart keeps the routing table, and the
  closed-source health endpoints are used for the raglite and qdrant probes
- the chart refuses to render values that would fail quietly: a consumer replica
  count its scheduled jobs cannot support, an Ingress with no hosts or one whose
  target component is disabled, or a disabled component with no external address.
  Turning off the migration job or caddy warns through the install notes instead,
  since running migrations elsewhere is legitimate
The new health package had none. These four cases run without a database, cache
or object store client, since they only exercise Register and Run: every probe is
reported, an all-passing set is healthy, a hanging probe is bounded by its own
deadline instead of holding readiness open, and Register is safe while Run is
serving. The last one is what the mutex added in the previous commit is for, and
it passes under -race.
@eetoc
eetoc marked this pull request as draft October 8, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant