Skip to content

Latest commit

 

History

History
92 lines (73 loc) · 7 KB

CLAUDE.md

File metadata and controls

92 lines (73 loc) · 7 KB

Project: pidcast

Single-shot CLI that turns a URL or audio file into an Obsidian-ready Markdown transcript with optional LLM analysis. See README.md for user-facing overview and docs/architecture.md for system structure.

Tech stack

  • Runtime: Python ≥ 3.10, managed via uv
  • Audio: yt-dlp, ffmpeg, ffprobe
  • Transcription providers: whisper.cpp (local, default) and ElevenLabs Scribe v2 (cloud)
  • Diarization: pyannote.audio (whisper path, needs HF token) or built-in (ElevenLabs path)
  • LLM providers: Groq (default, JSON mode) or Claude via the Claude Code CLI

Core commands

Run with uv run pidcast <input>. Useful entry points:

Command Purpose
pidcast <URL_OR_PATH> Transcribe only (bare = transcribe)
pidcast transcribe <URL_OR_PATH> [--analyze] Transcribe; analysis is opt-in via --analyze or any analysis flag (-a, --provider, -o, ...)
pidcast setup Interactive setup wizard (env, deps, API keys)
pidcast doctor Diagnose tooling/env configuration
pidcast info Print resolved data/config dirs
pidcast analyze transcript.md Re-analyze without re-transcribing
pidcast diarize transcript.md [--audio file] Retry diarization only
pidcast discord transcript.md [--style {play,digest}] [--pick 1,3] [--model opus] Story picker + Ukrainian Discord recap (<= 1900 chars) via the Claude CLI
pidcast notes transcript.md [--pick 2,5] [--focus TEXT] [-o] [--model opus] Theme picker + English knowledge notes (exec summary, detailed focus themes, glossary) via the Claude CLI
pidcast transcribe <input> --test [MIN] [--start-at MIN] Dry-run transcription on a slice
pidcast transcribe <input> --transcription-provider {whisper,elevenlabs} Pick transcription backend
pidcast transcribe <input> --provider {groq,claude} Pick LLM analysis backend
pidcast lib {add,list,show,remove,sync,process,digest} Manage podcast subscriptions
pidcast list {analyses,models,whisper-models,presets,profiles} List analysis types / LLM models / Whisper models / presets / Chrome profiles

The CLI is verb-first (one argparse subparser per verb, wired in cli.py via set_defaults(func=...)); bare pidcast <input> injects the transcribe verb. Advanced whisper-tuning flags (--vad, --vad-threshold, --temperature, --no-speech-thold, --no-fallback, --whisper-threads) are hidden from --help (still settable; supply via config.yaml presets).

Environment: copy .env.example → .env. See docs/development-guide.md for the full var list.

Build, test, lint

uv sync --group dev          # install runtime + dev deps
uv run pytest                # run tests
uv run ruff check src/       # lint
uv run ruff format src/      # format
pre-commit install           # enable pre-commit hooks (ruff, mermaid)

Module map

Source lives under src/pidcast/. Hot-path modules (read these first when debugging a workflow):

  • cli.py — thin argparse surface: builds the verb subparser tree + shared parent parsers, then args.func(args)
  • commands/ — one module per verb (transcribe, analyze, diarize, discord, notes, listing, info, lib); handler bodies live here, not in cli.py
  • discord_recap.py + config/discord/*.yaml — two-pass Discord recap: Haiku lists stories, user picks, Opus/Sonnet writes in a preset style; one auto-shorten retry if over budget
  • knowledge_notes.py + config/notes/*.yaml — long-episode notes: Haiku maps themes (reusing the discord story parser), user checks focus themes, Opus gets the full transcript and writes deep sections for focus themes, one line for the rest
  • providers/claude_provider.py — run_claude() shells out to the claude CLI in isolation (temp cwd, replaced system prompt, no tools/MCP) so the user's CLAUDE.md never reaches analysis prompts
  • duplicate.py — duplicate-detection prompt UI (consumed by commands/transcribe.py)
  • workflow.py — orchestration: download → transcribe → diarize → analyze → write
  • transcription.py + providers/ — provider dispatch for whisper/ElevenLabs
  • diarization.py — pyannote integration; consumes whisper JSON or ElevenLabs speakers
  • analysis.py + summarization.py — Groq/Claude prompt execution, JSON validation
  • chunking.py — semantic chunking for transcripts > 120k chars
  • markdown.py — front matter assembly, smart filename construction
  • library.py + sync.py + rss.py + apple_podcasts.py + discovery.py — podcast library subsystem
  • digest.py + history.py — processing history + digest generation
  • download.py + cookies.py — yt-dlp wrapper, browser cookie extraction
  • config.py + config_manager.py + model_selector.py + setup.py — config, model fallback chain, setup wizard
  • evals/ — provider comparison evals (pidcast-eval entry point)
  • exceptions.py + utils.py — shared error types, filename/duplicate helpers

Engineering conventions

  • Workflow: "plan then execute" — propose changes before editing.
  • Audio pipeline: standardize to 16kHz mono WAV before transcription regardless of source format.
  • Whisper anti-hallucination: --suppress-nst is ON by default; --vad (opt-in, needs a Silero model) strips silence pre-decode; repeated identical lines (3+) are collapsed in post-processing. These attack Whisper's silence-hallucination at the source rather than censoring phrases.
  • Chunking threshold: transcripts > 120k chars use semantic chunking with synthesis.
  • LLM responses: every analysis prompt returns JSON with analysis and contextual_tags fields.
  • Filenames: smart-filtered with YYYY-MM-DD_Title.md date prefix.
  • Transcripts location: the canonical store is the XDG data dir, $XDG_DATA_HOME/pidcast/transcripts/ (default ~/.local/share/pidcast/transcripts/; override with PIDCAST_DATA_DIR; run pidcast info to print the resolved dirs). Audio, logs, and the unified run history (state/runs.json) live alongside it under the data dir. Never drop stray transcripts in the repo root (ignored via .gitignore).
  • Linting: ruff is the single source of truth (config in pyproject.toml under [tool.ruff]).

Data handling

The transcripts dir ($XDG_DATA_HOME/pidcast/transcripts/, run pidcast info) contains large text files (some > 200 KB). The repo's data/transcripts/ may still hold pre-migration files. Do NOT read these into context unless the user explicitly approves a specific file. For test fixtures, ask the user to point at one or create a dedicated minimal fixture under tests/.

Further reading

  • docs/architecture.md — components, data flow, diagrams
  • docs/development-guide.md — dev environment, testing, CI
  • docs/adr/ — Architecture Decision Records (provider abstractions)
  • config/prompts.yaml — analysis prompt templates
  • config/models.yaml — LLM model definitions and fallback chains