Skip to content

DOCS.md

Latest commit

 

History

History
320 lines (219 loc) · 12.4 KB

DOCS.md

File metadata and controls

320 lines (219 loc) · 12.4 KB
Raw

Duma - Business Requirements

"Duma" — Hungarian for "talking" (dumalni a gephez)

Duma is a local macOS application that lets AI agents ask the user questions asynchronously via MCP. The user answers at their own pace through a native app window, and agents poll back later to retrieve responses.

1. High-Level Workflow

External flow (agent side)

  1. An agent encounters a question it cannot resolve on its own.
  2. The agent calls the ask_question MCP tool, providing the question content and its agent ID.
  3. Duma returns an auto-generated question ID.
  4. The agent continues with other work and periodically calls get_response with its agent ID and the question ID.
  5. When the user has answered, get_response returns the full response. The question status moves to "retrieved".

Internal flow (user side)

  1. A new question arrives via MCP.
  2. A native macOS notification appears: "New question from {agent_name}".
  3. The menu bar icon changes from idle to pending state.
  4. The user clicks the notification or menu bar icon to open/focus the app window.
  5. The app window shows the question with response options.
  6. The user responds (see Response Mechanisms below) and clicks "Send Response".
  7. The app saves the response. If voice was included, it is transcribed via OpenAI Whisper API.
  8. The UI jumps to the next unanswered question. If none remain, it shows a "yay, you are done" state.

2. MCP Tools

All tools are exposed via MCP. Agents identify themselves with a self-provided agent_id. Questions can only be created by agents via MCP — the user interface is for answering only.

ask_question

Submit a new question for the user to answer.

Parameters:

Field Type Required Description
agent_id string yes Self-reported agent identifier (e.g. recipe-agent-01)
question_short_description string yes What the question is about, max 5-10 words. Used as display title.
question_body string yes The full question in Markdown format.
proposed_answers string[] no List of quick-response options the user can select from.
high_priority bool no Defaults to false. Affects ordering and visual treatment.

Returns: { question_id: string } — auto-generated unique ID (e.g. 0aa1ea9a5a04b78d4581dd6d17742627).

get_response

Check if a question has been answered and retrieve the response.

Parameters:

Field Type Required Description
agent_id string yes Must match the agent_id used when asking.
question_id string yes The ID returned by ask_question.
mark_retrieved bool no Defaults to true. When true, the question status transitions to "retrieved" on first access. Set to false to read the response without changing its status.

Returns:

  • If not yet answered: { status: "pending" }
  • If answered: { status: "answered", selected_options: string[], text_response: string | null, voice_transcript: string | null }

When mark_retrieved is true (the default), the first retrieval of an answered question moves its status internally to "retrieved". When false, the status remains "answered" regardless of how many times the response is read. Subsequent calls with mark_retrieved: true still trigger the transition if it hasn't happened yet.

list_my_pending_questions

List all unanswered questions for a given agent.

Parameters:

Field Type Required Description
agent_id string yes The agent's identifier.

Returns: { questions: [{ question_id: string, short_description: string, status: string }] }

list_all_my_questions

List all questions (any status) for a given agent.

Parameters:

Field Type Required Description
agent_id string yes The agent's identifier.

Returns: { questions: [{ question_id: string, short_description: string, status: string }] }

3. Question Lifecycle & Statuses

pending  -->  answered  -->  retrieved
  • pending: Question submitted, awaiting user response.
  • answered: User has responded, agent has not yet called get_response.
  • retrieved: Agent has called get_response and received the answer.

Questions are kept indefinitely. They are never deleted or expired.

4. Response Mechanisms

When answering a question, the user has three input methods. At least one must be provided before the response can be submitted.

Proposed answer selection (optional)

  • Displayed as checkboxes (multi-select).
  • The user can select zero or more of the agent-provided options.
  • Options are returned to the agent as their original label text.

Free-text input (optional)

  • Multi-line textarea.
  • For when none of the proposed answers fit and the user prefers typing.

Voice recording (optional)

  • A record/stop button (audio is recorded Python-side, not in the browser).
  • Maximum recording length: 2 minutes.
  • Audio is saved to disk at ~/.duma/audio/{question_id}.* (format determined by the platform).
  • On submit, the audio is sent server-side to the OpenAI Whisper API for transcription. Language is switchable via a 🇭🇺 HU | 🇬🇧 EN toggle next to the record button (defaults to Hungarian).
  • On transcription failure: retry once, then fall back to the transcript text "user answered with voice input but transcription failed".
  • The raw audio file is kept on disk for reference but is not used by any other functionality.

Submission rules

  • The "Send Response" button is disabled until at least one input is provided (a selected option, text, or a voice recording).
  • Once submitted, the question becomes read-only.

5. User Interface

Technology

  • Native macOS window via pywebview (WebKit-based).
  • Dark theme throughout (similar to VS Code dark color scheme, good contrast).
  • HTML/CSS/JS frontend, no build step.

Layout

Two-panel layout:

  • Left sidebar: Scrollable list of questions.
  • Right panel: Selected question detail and response area.

Sidebar (question list)

Each item displays:

  • Short description as the title (bold/prominent).
  • Truncated question body below it (like an email preview).
  • Source agent name and timestamp.
  • Status indicator (pending / answered / retrieved).
  • High priority gets distinct visual treatment (e.g. colored accent/dot).

Sort order

  1. Unanswered, high priority (newest first)
  2. Unanswered, normal priority (newest first)
  3. Answered, not yet retrieved (newest first)
  4. Answered, retrieved (newest first)

Question detail panel

  • Short description as heading.
  • Source agent name and timestamp metadata.
  • Question body displayed as raw Markdown in a monospace/preformatted block (not rendered as HTML).
  • Proposed answers as checkboxes (if any were provided).
  • Free-text textarea.
  • Voice recorder (record/stop button, visual indicator of recording state, duration counter, language toggle).
  • Send Response button (disabled until at least one input is provided).

Completed question view

  • Read-only. Shows the question and the response that was given.
  • Shows: selected options, text response, voice transcript (if any).

Empty state

  • When no unanswered questions remain, show a "yay, you are done" message.
  • The user can still navigate to past answered questions in the sidebar.

Navigation behavior

  • On launch, the first item selected is the newest unanswered question.
  • After answering a question, automatically jump to the next unanswered question.
  • When no unanswered questions remain, show the empty state.

6. macOS Integration

Menu bar icon

  • Persistent icon next to the system clock.
  • Two states: idle (no pending questions) and pending (one or more unanswered questions).
  • Clicking the icon opens/focuses the app window.

Window behavior

  • Closing the app window keeps the app running in the menu bar.
  • The window can be re-opened by clicking the menu bar icon or a notification.
  • "Quit" from the menu bar fully terminates the app.

Notifications

  • A native macOS notification is shown each time a new question arrives.
  • Notification title: "Duma"
  • Notification body: "New question from {agent_name}: {short_description}"
  • Clicking the notification opens/focuses the app window.
  • Only one app window exists. If already open, it is focused (not duplicated).

Dock

  • The app appears with its own icon in the Dock when running.
  • Achieved via .app bundle built by PyInstaller (ad-hoc code-signed).

7. Architecture

+-----------------------------------+
|  Menu bar icon (PyObjC)           |  idle/pending status
+----------------+------------------+
                 | click -> focus/show window
+----------------v------------------+
|  Native window (pywebview)        |  HTML/CSS/JS UI
|  +-- Question list (sidebar)      |
|  +-- Question detail + response   |
+----------------+------------------+
                 | HTTP to localhost
+----------------v------------------+
|  Backend (FastAPI)                |
|  +-- Serves static web UI files   |
|  +-- REST API for UI actions      |
|  +-- MCP server (for agents)      |
|  +-- SQLite database              |
|  +-- Audio file storage           |
+-----------------------------------+

All components run in a single process. pywebview owns the main thread (Cocoa run loop). The HTTP server runs in a daemon thread. The menu bar icon and notifications use PyObjC directly.

Tech stack

Component Technology
Backend Python, FastAPI
Database SQLite (via Python built-in sqlite3)
Frontend Vanilla HTML / CSS / JS (no framework, no build step)
Native window pywebview
Menu bar icon PyObjC (NSStatusItem)
Notifications PyObjC (UserNotifications framework)
Voice recording Python-side via sounddevice + soundfile
Transcription OpenAI Whisper API (server-side)
MCP transport Streamable HTTP
App bundle PyInstaller (via build.sh)

Data storage

  • Configuration: ~/.duma/config.json — API keys and settings.
  • SQLite database: ~/.duma/duma.db — stores questions, responses, metadata.
  • Audio files: ~/.duma/audio/{question_id}.* — raw voice recordings, kept for reference only.

Configuration

  • OPENAI_API_KEY for Whisper API access (via ~/.duma/config.json or environment variable).
  • DUMA_WHISPER_LANGUAGE for transcription language (defaults to "hu").
  • Port: configurable, default 31299.
  • Bound to localhost only — not accessible from the network.
  • Only whitelisted config keys are accepted; unknown keys are ignored.

8. Security

Threat model

Duma is a single-user localhost tool intended only for use by the machine owner and that owner's own locally running agents. It does not attempt to defend against malicious local processes, browser extensions, browser tabs, or other software running under the same user account. agent_id is metadata, not authentication. Localhost binding is used to avoid external network exposure, not to provide security isolation between local actors.

  • No authentication is a deliberate tradeoff for a trusted single-user local environment.
  • Any code able to access the localhost port may be able to interact with Duma.
  • This is acceptable for v1 because hostile-local-code protection is out of scope.
  • If the threat model changes (e.g. multi-user, network exposure), authentication and request validation become required, not optional.

Security reviews should evaluate Duma against this trusted-local threat model unless the product scope changes.

Local robustness (required regardless of threat model)

  • The backend never trusts client-supplied file paths. Audio file locations are determined server-side.
  • All agent-provided text is rendered safely (no innerHTML). Prevents accidental script injection even from buggy agents.
  • Development-only endpoints are disabled outside debug mode.
  • The OpenAI API key is never exposed to the frontend.

Network

  • The HTTP server binds to 127.0.0.1 only. No network access.
  • Requests with unexpected Origin headers are rejected.

9. Non-goals (explicitly out of scope)

  • No AI-generated text or summaries within Duma itself.
  • No search or filtering in the question list (may be added later).
  • No launchd/auto-start (manual launch for now).
  • No multi-user or network access.
  • No question expiry or deletion.
  • No editing of already-submitted responses.