Skip to content

README.en.md

Latest commit

 

History

History
212 lines (162 loc) · 8.29 KB

README.en.md

File metadata and controls

212 lines (162 loc) · 8.29 KB
Raw

Local LLM Control Plane (Controller)

README Guide & Links

This README is intentionally trimmed to be a quick-start entry.

Detailed documentation has been moved to:

Architecture design details, system-shape sections, capability tiers, feature deep-dives, and troubleshooting guidance are now consolidated in doc/Schema.md, organized as:

  • Front chapters: current architecture design
  • Later chapters: implemented/planned capabilities, feature details, and troubleshooting

Brief Description

Local LLM Control Plane is a local multi-node LLM control plane based on controller + agent + managed node, providing unified model/node/runtime/task management.

Project Positioning

  • Built for local/LAN multi-node model operations
  • Short-term position: a model resource scheduling system; long-term evolution: a multi-model orchestration runtime for complex tasks
  • Controller owns unified API, state, scheduling, and orchestration
  • Agent executes node-local actions and reports facts back
  • Supports Docker/Portainer/LM Studio runtime integrations
  • Provides Web console + REST API + SQLite persistence

Zero-to-Deployment

Target directory example: /tank/docker_data/model_control_plane

sudo mkdir -p /tank/docker_data/model_control_plane
sudo chown -R $USER:$USER /tank/docker_data/model_control_plane
rsync -a --delete /home/whoami/Dev/model-control-plane/ /tank/docker_data/model_control_plane/
cd /tank/docker_data/model_control_plane

Initialize:

cp resources/docker/compose.example.env .env
mkdir -p resources/config resources/models testsystem/logs
touch resources/config/controller.db
chmod 777 resources/config testsystem/logs
chmod 666 resources/config/controller.db

Start:

# Core (recommended, local-agent on controller node is enabled by default)
./scripts/one-click-up.sh

# Explicitly enable local-agent (same as default behavior)
./scripts/one-click-up.sh --local-agent

# Disable local-agent (compatibility/troubleshooting only; falls back to controller direct/self-check)
./scripts/one-click-up.sh --no-local-agent

# Core + addons
./scripts/one-click-up.sh --addons

# Core + download containers
./scripts/one-click-up.sh --download

# Core + vLLM runtime template container
./scripts/one-click-up.sh --vllm

Verify:

curl -sS http://127.0.0.1:59081/healthz
curl -sS http://127.0.0.1:59081/api/v1/models
curl -sS http://127.0.0.1:59081/api/v1/nodes

Stop:

./scripts/one-click-down.sh

Key Project Files

  • Orchestration: docker-compose.yml
  • Build: Dockerfile
  • Control-plane config: resources/config/config.example.yaml
  • Compose env template: resources/docker/compose.example.env
  • Gateway config: resources/nginx/nginx.example.conf
  • Frontend: resources/web/index.html / resources/web/app.css / resources/web/app.js
  • One-click scripts: scripts/one-click-up.sh / scripts/one-click-down.sh
  • Architecture schema: doc/Schema.md
  • Change log: doc/LOG.md
  • Research index: doc/Research-Index.md
  • Enhancement roadmap: doc/Enhancement-Roadmap.md

Key Config Keys

  • MCP_EXTERNAL_PORT: external gateway port (default 59081)
  • MCP_SQLITE_PATH: SQLite file path
  • MCP_MODEL_DIR_HOST: host model directory (default ./resources/models)
  • MCP_MODEL_ROOT_DIR: in-container model directory (default /opt/controller/models)
  • MCP_TEST_LOG_ROOT_HOST: host test log directory (default ./testsystem/logs)
  • MCP_TEST_LOG_ROOT_DIR: in-container test log directory (default /opt/controller/test-logs)
  • MCP_LMSTUDIO_ENDPOINT: LM Studio endpoint
  • MCP_DOCKER_ENDPOINT: Docker endpoint
  • MCP_CONTAINER_HOST_ALIAS: in-container host alias (default host.docker.internal)
  • MCP_VLLM_EXTERNAL_PORT: vLLM external port
  • MCP_VLLM_MODEL: default vLLM model
  • HUGGING_FACE_HUB_TOKEN: optional token for private/gated models

API

Base:

  • GET /healthz
  • GET /api/v1/version

Nodes and models:

  • GET /api/v1/nodes
  • GET /api/v1/models
  • GET /api/v1/models/{id}
  • POST /api/v1/models/{id}/load
  • POST /api/v1/models/{id}/unload
  • POST /api/v1/models/{id}/start
  • POST /api/v1/models/{id}/stop

Runtime templates:

  • GET /api/v1/runtime-templates
  • POST /api/v1/runtime-templates/validate
  • POST /api/v1/runtime-templates

Runtime objects (instance-first):

  • GET /api/v1/runtime-bindings
  • GET /api/v1/runtime-instances
  • GET /api/v1/runtime-instances/{id}
  • GET /api/v1/runtime-instances/{id}/tasks
  • GET /api/v1/runtime-instances/{id}/summary
  • GET /api/v1/runtime-instances/{id}/reconcile-summary

Agents and tasks:

  • GET /api/v1/agents
  • POST /api/v1/agents/register
  • POST /api/v1/agents/{id}/heartbeat
  • POST /api/v1/agents/{id}/capabilities
  • GET /api/v1/agents/{id}/tasks/next
  • POST /api/v1/agents/{id}/tasks/{taskID}/report
  • GET /api/v1/tasks
  • GET /api/v1/tasks/{id}
  • POST /api/v1/tasks/runtime/start
  • POST /api/v1/tasks/runtime/stop
  • POST /api/v1/tasks/runtime/restart
  • POST /api/v1/tasks/runtime/refresh
  • POST /api/v1/tasks/agent/runtime-readiness
  • POST /api/v1/tasks/agent/node-local

Node execution task types (submitted via POST /api/v1/tasks/agent/node-local):

  • agent.runtime_precheck
  • agent.resource_snapshot
  • agent.docker_inspect
  • agent.docker_start_container
  • agent.docker_stop_container

Test runs:

  • GET /api/v1/test-runs/scenarios
  • GET /api/v1/test-runs
  • GET /api/v1/test-runs/{id}
  • POST /api/v1/test-runs

Scripted smoke checks (local-agent path):

  • scripts/controller_api_smoke.sh <base_url> [token] [agent_id] [model_id]
  • testsystem/scenarios/stage0_to_b_full_smoke.sh
  • testsystem/scenarios/stage0_runtime_object_smoke.sh
  • testsystem/scenarios/local_agent_execution_smoke.sh
  • testsystem/scenarios/e5_embedding_smoke.sh
  • testsystem/scenarios/e5_gating_blocked_smoke.sh

Stage A closure (current):

  • agent.runtime_precheck is now manifest-driven preflight (mount/env/script/port/compatibility/policy/custom_bundle-min checks).
  • Agent check/execution task details now use a unified structured envelope (overall_status + structured_result + manifest_summary).
  • Agent outcomes now land on RuntimeInstance first (precheck/readiness/drift/resolved state + latest task summary).
  • Node keeps node-level resource/agent liveness facts; Model primarily consumes instance projection.
  • testsystem/scenarios/local_agent_execution_smoke.sh now covers the Stage A closure chain on the default E5 sample model.
  • Stage B kickoff is now in place: a controller-side instance-first reconcile loop continuously updates desired/observed/readiness/health/drift and last_reconciled_at.
  • precheck_* is now a formal reconcile input, and agent_offline/observation_stale are surfaced in instance-level reconcile conclusions.
  • Stage B deepening is now active: RuntimeInstance now carries structured conflict/gating/last_plan_* state, and reconcile emits minimal load/unload/deferred/blocked lifecycle plans.
  • runtime.start/restart now uses gating fail-fast (blocked requests are rejected before enqueue), and planner metadata is attached to runtime task payload/detail.
  • Added testsystem/scenarios/e5_gating_blocked_smoke.sh to validate the chain: binding/script conflict -> gating blocked -> runtime.start fail-fast.
  • Added stage0_runtime_object_smoke and stage0_to_b_full_smoke to cover Stage 0 through Stage B in one test matrix.
  • The web console now supports manual scenario triggering via a scenario selector, including a quick "Stage 0~B" run entry.
  • See doc/Schema.md (Stage B semantics) and doc/LOG.md (this round changes).