Excalidraw AI Proxy

A local Node.js proxy that enables Excalidraw OSS AI features without exposing your OpenAI API key to the browser.

This repository is distributed as source through GitHub releases and is also published as the @mcamner/excalidraw-ai-proxy npm package on GitHub Packages.

It implements Excalidraw-compatible text-to-diagram and diagram-to-code endpoints, repairs common Mermaid failures, and keeps model access and secrets on the server.

Important

This repository is the AI backend proxy, not the Excalidraw editor. To use the AI features in the UI, install a local checkout of Excalidraw OSS and connect it to this proxy.

Why use it?

• Keep secrets server-side. Excalidraw only connects to the proxy; the OpenAI API key remains in the proxy's local .env.
• Use separate models per task. Configure fast text-to-diagram and stronger multimodal diagram-to-code models independently.
• Get import-ready Mermaid. Deterministic sanitizing and optional model repair normalize output before Excalidraw receives it.
• Run locally by default. The proxy binds to 127.0.0.1 and only allows configured Excalidraw origins.
• Diagnose compatibility quickly. Health and capability endpoints expose the supported API surface without exposing secrets.

Example

Prompt:

Show a user request flowing through the proxy to OpenAI and back to Excalidraw.

Normalized Mermaid returned to Excalidraw:

flowchart LR
  A[Excalidraw] -->|AI request| B[Local proxy]
  B -->|Server-side API call| C[OpenAI]
  C -->|Model output| B
  B -->|Repaired Mermaid| A

Loading

The text-to-diagram endpoint buffers the model stream, removes incompatible Mermaid syntax, optionally repairs invalid output, and then returns normalized SSE chunks. See examples for request and response shapes.

When the sanitized output still does not parse, the proxy classifies why — unbalanced subgraph, end used as a node ID, a node ID with a space, an unquoted label, a diagram type Excalidraw cannot import, too many nodes — and sends that specific error back to the model as a targeted repair instruction instead of a generic retry. It keeps the best candidate it has seen, so a retry can never make the response worse than the first attempt. See the AI output contract for the failure classes and their severity.

Buffering also makes the endpoint tolerant of a truncated upstream stream: if the OpenAI connection closes prematurely after content has arrived, the proxy normalizes what it received instead of failing the request. A premature close with no content is still an error.

Prompt contracts

Text-to-diagram uses one of two system prompts. The default contract asks for a plain flowchart. The architecture contract additionally requires subgraph grouping and quotes every label. Neither contract states a node count: diagram size is owned by MERMAID_MAX_NODES alone, so the prompt and the runtime budget cannot disagree.

The proxy selects the contract per request. Send "mode": "architecture" or "mode": "default" in the request body to choose explicitly; otherwise a bilingual Swedish/English heuristic inspects the prompt. Terms like arkitektur, architecture, microservice, or component diagram select the architecture contract on their own, while weaker terms like component or data flow only do so alongside architecture context — show the system components routes to architecture, show the components of a form does not.

Because the architecture contract is more constrained, use an explicit mode when the choice matters. Available modes are listed in GET /v1/ai/capabilities under features.promptContractModes. See the AI output contract for both contracts in full.

Quick start

Requirements: Node.js 20 or later, npm, an OpenAI API key, and a local Excalidraw OSS checkout.

Clone and install Excalidraw OSS in a separate directory:

git clone https://github.com/excalidraw/excalidraw.git
cd excalidraw
yarn

Then clone and install the proxy:

git clone https://github.com/MCamner/excalidraw-ai-proxy.git
cd excalidraw-ai-proxy
cp .env.example .env
npm install

Set your own key in the proxy's .env:

OPENAI_API_KEY=your-openai-api-key

Start the proxy:

npm run dev

In the Excalidraw checkout, add this to .env.local:

VITE_APP_AI_BACKEND=http://localhost:3016
VITE_APP_PORT=3003

Then start Excalidraw from that checkout:

yarn start

The default local addresses are:

• proxy: http://localhost:3016
• Excalidraw: http://localhost:3003

For prerequisites, verification, and custom origins, read the full installation guide.

Supported API

Endpoint	Purpose
GET /health	Basic liveness check
GET /v1/ai/capabilities	Supported features, models, limits, and streaming mode
POST /v1/ai/text-to-diagram/chat-streaming	Prompt to repaired Mermaid over SSE
POST /v1/ai/diagram-to-code/generate	Excalidraw image to self-contained HTML

Architecture

flowchart LR
  Browser[Excalidraw browser UI]
  Proxy[Local Express proxy]
  OpenAI[OpenAI API]
  Env[Server-side .env]

  Browser -->|AI requests| Proxy
  Env -->|API key and settings| Proxy
  Proxy -->|Model requests| OpenAI
  OpenAI -->|Generated output| Proxy
  Proxy -->|Normalized response| Browser

Loading

The browser never receives OPENAI_API_KEY. The proxy applies explicit CORS origins, body limits, prompt limits, rate limiting, model timeouts, and Mermaid normalization. It does not provide authentication or TLS termination; review the security policy before exposing it beyond localhost.

Configuration

.env.example is the source of truth for runtime settings. The main controls are:

Setting	Default	Purpose
PORT	3016	Proxy port
HOST	127.0.0.1	Bind address
ALLOWED_ORIGINS	localhost on port 3003	Browser origins allowed by CORS
OPENAI_MODEL	gpt-4.1-mini	Fallback model for both tasks
OPENAI_TEXT_TO_DIAGRAM_MODEL	OPENAI_MODEL	Text-to-diagram model
OPENAI_DIAGRAM_TO_CODE_MODEL	OPENAI_MODEL	Diagram-to-code model
OPENAI_ARCHITECTURE_MODEL	text-to-diagram model	Model for architecture-contract requests
OPENAI_REPAIR_MODEL	text-to-diagram model	Model for the Mermaid repair pass
OPENAI_MODEL_POOL	empty	Models the proxy may pick from per task
MERMAID_AUTO_REPAIR	true	Enable model-assisted repair fallback
MERMAID_MAX_REPAIR_ATTEMPTS	1	Targeted repair passes, 0 disables them, 2 is the cap
MERMAID_MAX_NODES	60	Soft node budget that triggers one smaller-diagram retry, 0 disables
MAX_PROMPT_CHARS	6000	Maximum text prompt length
RATE_LIMIT_MAX_REQUESTS	20	Requests allowed per rate-limit window

For higher-quality diagram-to-code output, use a stronger multimodal model for OPENAI_DIAGRAM_TO_CODE_MODEL while keeping text-to-diagram on a faster model.

Model routing

The proxy resolves a model per task rather than using one model for everything. Four tasks are routed: text-to-diagram, text-to-diagram:architecture, mermaid-repair, and diagram-to-code. Resolution order per task is configured model → pool → fallback:

1. The task's own setting wins if set.
2. Otherwise, if OPENAI_MODEL_POOL lists models, the proxy picks from that list using the capabilities in lib/model-registry.js: the cheapest model that can stream for a plain flowchart, the strongest one for architecture, the cheapest small one for the repair pass, and one that accepts image input for diagram-to-code.
3. Otherwise it falls back to the text-to-diagram model, then OPENAI_MODEL.

The pool is opt-in on purpose. Without it the proxy only ever calls a model you named explicitly — it never upgrades to a more expensive model on its own.

GET /v1/ai/capabilities reports the resolved model for each task under modelRouting, together with why it was chosen (configured, pool, or fallback) and its capability tiers, so a wrong model is visible without reading the logs. Known models also clamp the configured token budget to their documented output limit, and a model routed to a task it cannot perform is reported at startup.

The registry is a table, not a provider abstraction: the API key stays server-side, there is one OpenAI client, and adding another provider means adding rows and a client — not rewriting the routes. Quality and cost tiers are routing labels, not benchmarks; edit them to match what you actually pay for.

The two endpoints call different OpenAI APIs, so the models are not interchangeable:

Setting	OpenAI API	Model must support
OPENAI_TEXT_TO_DIAGRAM_MODEL	Chat Completions	streaming
OPENAI_DIAGRAM_TO_CODE_MODEL	Responses	image input

Documentation

Document	Contents
Quick start	Minimal local setup
Installation	Prerequisites, configuration, and verification
Examples	API request and response examples
AI output contract	Mermaid generation and repair rules
Architecture	Repository flow and editable Excalidraw diagram
Roadmap	Completed phases and future constraints
Security policy	Private reporting and deployment scope
Contributing	Development and pull-request workflow

Development

Run the complete test suite:

npm test

The suite covers capabilities, HTTP endpoints, mocked OpenAI responses, streaming semantics, prompt contracts, and Mermaid repair regressions. GitHub Actions runs the same command for pushes and pull requests to main.

License

Released under the MIT License.

Acknowledgements

Built for compatibility with Excalidraw OSS.

Thanks to the Excalidraw maintainers and contributors for the open-source editor and its AI integration surface. This project is independent and is not affiliated with or endorsed by Excalidraw.