🪞 JournalMirror

Behavioral-Narrative Divergence Scorer — a tool that compares what you say in your journal with what your habit data shows, surfacing gaps you might not notice.

JournalMirror is not a diagnostic tool. It describes the gap, never the person.

---

Table of Contents

• Quick Start
• Installation
• Usage
  • CLI
  • REST API
• Input Formats
• How It Works
  • Pipeline Overview
  • Alignment Scoring
  • Divergence Classification
  • Evidence Gate
  • Safety Filter
• Example Personas
• Output Format
• Running Tests
• Tech Stack
• Design Decisions

---

Quick Start

# Clone and set up
cd journalmirror
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Run on an example
python run.py \
  --journal data/examples/alice/journal.jsonl \
  --habits  data/examples/alice/habits.csv \
  --output  results/alice_report.json

---

Installation

Prerequisites: Python 3.11+

# 1. Create and activate a virtual environment
python3.11 -m venv .venv
source .venv/bin/activate

# 2. Install dependencies
pip install -r requirements.txt

Note: On first run, the sentence-transformer model (all-MiniLM-L6-v2, ~80 MB) will be downloaded automatically and cached locally.

---

Usage

CLI

python run.py --journal <path/to/journal.jsonl> \
              --habits  <path/to/habits.csv> \
              --output  <path/to/output.json>

Arguments:

Flag	Required	Description
--journal	✅	Path to journal entries file (JSONL format)
--habits	✅	Path to habit tracking data (CSV format)
--output	✅	Path where the JSON report will be written

Example:

python run.py \
  --journal data/examples/bob/journal.jsonl \
  --habits  data/examples/bob/habits.csv \
  --output  results/bob_report.json

REST API

Start the server:

uvicorn api.main:app --reload

POST /analyze

Submit journal and habit data for analysis.

curl -X POST http://localhost:8000/analyze \
  -H "Content-Type: application/json" \
  -d '{
    "journal": [
      {"date": "2024-01-15", "domain": "fitness", "text": "Great workout today!"},
      {"date": "2024-01-16", "domain": "fitness", "text": "Hit the gym again."},
      {"date": "2024-01-17", "domain": "fitness", "text": "Feeling strong after training."}
    ],
    "habits": [
      {"date": "2024-01-15", "domain": "fitness", "metric": "workout_minutes", "value": 45},
      {"date": "2024-01-16", "domain": "fitness", "metric": "workout_minutes", "value": 0},
      {"date": "2024-01-17", "domain": "fitness", "metric": "workout_minutes", "value": 0}
    ]
  }'

Response:

{ "report_id": "a1b2c3d4-..." }

GET /report/{report_id}

Retrieve a previously generated report.

curl http://localhost:8000/report/a1b2c3d4-...

Interactive API docs are available at http://localhost:8000/docs.

---

Input Formats

Journal (journal.jsonl)

One JSON object per line. Each entry must have:

Field	Type	Description
date	string	ISO date (e.g. "2024-01-15")
domain	string	Life domain (e.g. "fitness", "work", "sleep")
text	string	Free-text journal entry

{"date": "2024-01-15", "domain": "fitness", "text": "Crushed my workout today! Feeling amazing."}
{"date": "2024-01-16", "domain": "fitness", "text": "Another great session at the gym."}
{"date": "2024-01-17", "domain": "sleep", "text": "Slept really well last night."}

Habits (habits.csv)

Standard CSV with columns:

Column	Type	Description
date	string	ISO date
domain	string	Must match domain names used in journal
metric	string	What's being measured (e.g. workout_minutes)
value	number	The measured value (use 0 for no activity)

date,domain,metric,value
2024-01-15,fitness,workout_minutes,45
2024-01-16,fitness,workout_minutes,0
2024-01-17,fitness,workout_minutes,0
2024-01-15,sleep,sleep_hours,7.5

---

How It Works

Pipeline Overview

journal.jsonl ─┐
               ├─→ Ingest ─→ Preprocess ─→ Evidence Gate ─→ Score ─→ Classify ─→ Safety Filter ─→ Report
habits.csv ────┘

Each domain (fitness, work, sleep, etc.) is scored independently. The pipeline for each domain:

1. Ingest — parse and validate input files
2. Preprocess — group texts by domain, compute narrative strength, detect aspiration language
3. Behavior Mapping — normalize habit data into per-domain signals (mean value, trend, active ratio)
4. Evidence Gate — check if there's enough data to make a claim
5. Score — embed narrative and behavior summaries, compute cosine similarity
6. Classify — assign a divergence type based on decision boundaries
7. Safety Filter — structurally block any medical/diagnostic/personality output
8. Report — assemble gap-focused JSON output

Alignment Scoring

JournalMirror uses all-MiniLM-L6-v2 (a sentence-transformer model) to compare what you write with what your data shows:

1. All journal texts for a domain are embedded and mean-pooled into a single vector
2. Behavioral data is converted into a natural-language summary (e.g. "fitness: average 8.6 workout_minutes per session, declining trend over 14 days") and embedded
3. Cosine similarity between the two embeddings gives the alignment score ∈ [0, 1]
4. Divergence = 1 − alignment

A divergence of 0.0 means perfect alignment. A divergence of 1.0 means no semantic overlap.

Divergence Classification

Each domain is classified into exactly one of four types (or marked as aligned). Rules are applied in priority order:

┌─────────────────────────────────────────────────────────────────┐
│  Rule 1: BLIND SPOT                                             │
│  narrative_strength < 0.1 AND behavior_strength ≥ 0.5           │
│  → The domain is prominent in data but absent from journaling   │
├─────────────────────────────────────────────────────────────────┤
│  Rule 2: ASPIRATION GAP                                         │
│  divergence ≥ 0.3 AND aspiration language detected              │
│  AND behavior trend ≤ 0 (flat or declining)                     │
│  → Goals are stated but progress is stagnant                    │
├─────────────────────────────────────────────────────────────────┤
│  Rule 3: OVERSTATEMENT                                          │
│  divergence ≥ 0.3 AND narrative_strength > behavior_strength    │
│  → Journal describes more activity than data reflects           │
├─────────────────────────────────────────────────────────────────┤
│  Rule 4: UNDERSTATEMENT                                         │
│  divergence ≥ 0.3 AND behavior_strength ≥ narrative_strength    │
│  → Data shows more activity than journal acknowledges           │
├─────────────────────────────────────────────────────────────────┤
│  Rule 5: ALIGNED                                                │
│  divergence < 0.3                                               │
│  → Journal and data are broadly consistent                      │
└─────────────────────────────────────────────────────────────────┘

Key signals used:

Signal	What it measures	Range
divergence	Semantic gap between narrative and behavior embeddings	[0, 1]
narrative_strength	How much the person writes about this domain (volume + verbosity)	[0, 1]
behavior_strength	How strong the behavioral signal is (metric values × consistency)	[0, 1]
has_aspiration	Whether journal text contains goal/future-intent language	boolean
behavior_trend	Slope of daily metric values over the period	ℝ

Evidence Gate

A domain must have enough data before any claim is made:

Criterion	Minimum Required
Journal entries for the domain	3
Distinct days of habit data	3

• Both met → full scoring and classification proceeds
• Enough habit data but too few entries → domain is eligible for blind_spot detection (the absence of narrative is the signal)
• Neither met → returns insufficient_evidence with an explanation

The system prefers to abstain rather than misclassify.

Safety Filter

JournalMirror structurally prevents certain outputs using a blocklist (~100 terms) and compiled regex patterns. This is enforced in code, not just by phrasing:

Blocked Category	Examples
Medical diagnoses	depression, anxiety disorder, ADHD, bipolar
DSM/ICD labels	DSM-5, ICD-10, F32.1
Personality assertions	lazy, narcissistic, undisciplined, toxic
Causal claims	"you are X because...", "this suggests a disorder"

What the system says:

"In the fitness domain, journal entries describe significantly more activity than the habit data reflects."

What the system never says:

"You are lazy about fitness." "This suggests avoidant personality traits."

---

Example Personas

Three worked examples are included in data/examples/:

Alice — Fitness Overstatement

Alice writes enthusiastically about daily workouts, but her habit data shows only ~3 sessions in 14 days.

python run.py --journal data/examples/alice/journal.jsonl \
              --habits data/examples/alice/habits.csv \
              --output results/alice_report.json

Result: overstatement — narrative_strength=1.0, behavior_strength=0.61

Bob — Work Blind Spot

Bob journals about hobbies and family but never mentions work. His habit data shows 60+ hours/week.

python run.py --journal data/examples/bob/journal.jsonl \
              --habits data/examples/bob/habits.csv \
              --output results/bob_report.json

Result: blind_spot for work domain — narratively absent, behaviourally dominant

Carol — Insufficient Evidence

Carol has only 2 journal entries and 2 days of data — below the minimum threshold.

python run.py --journal data/examples/carol/journal.jsonl \
              --habits data/examples/carol/habits.csv \
              --output results/carol_report.json

Result: insufficient_evidence — the system abstains

---

Output Format

Reports are JSON files with this structure:

{
  "report_id": "b6d936de-2063-41b0-90e1-ccf231732dc8",
  "generated_at": "2026-06-10T11:56:00.246857+00:00",
  "domains": [
    {
      "domain": "fitness",
      "evidence_status": "sufficient",
      "alignment_score": 0.5579,
      "divergence_score": 0.4421,
      "divergence_type": "overstatement",
      "narrative_strength": 1.0,
      "behavior_strength": 0.6071,
      "description": "In the fitness domain, journal entries describe significantly more activity than the habit data reflects."
    }
  ]
}

For insufficient evidence domains:

{
  "domain": "meditation",
  "evidence_status": "insufficient_evidence",
  "reason": "Only 2 journal entries (minimum 3) and 2 habit days (minimum 3)",
  "description": "There is not enough data to assess the meditation domain."
}

---

Running Tests

source .venv/bin/activate
python -m pytest tests/ -v

Test coverage (71 tests):

Test File	Tests	What It Covers
test_classifier.py	15	All 4 type boundaries, edge cases, priority ordering
test_evidence_gate.py	11	Below/at/above threshold, mixed sufficiency, blind_spot eligibility
test_safety.py	34	Medical terms, personality traits, regex patterns, clean text passthrough
test_scorer.py	11	Cosine similarity range, identical/unrelated texts, divergence formula

---

Tech Stack

Component	Technology
Language	Python 3.11
Embeddings	sentence-transformers 2.7.0 (all-MiniLM-L6-v2)
API	FastAPI + Uvicorn
Data	Pandas, NumPy
Testing	pytest
LLM calls	None — all scoring is local and deterministic

---

Design Decisions

Full design rationale is documented in decisions.md, covering:

1. Alignment method — why sentence-transformers, what it misses (sarcasm, negation, temporal context)
2. Type boundaries — exact thresholds and classification rules
3. Evidence sufficiency — why 3 entries / 3 days, and when the system abstains
4. What this system refuses to claim — the full safety policy

---

Project Structure

journalmirror/
├── README.md              ← you are here
├── decisions.md           ← design decisions document
├── requirements.txt       ← Python dependencies
├── run.py                 ← CLI entry point
├── app/
│   ├── ingest.py          ← parse journal + habit files
│   ├── preprocessor.py    ← NLP: domain extraction, aspiration detection
│   ├── behavior_mapper.py ← normalize habits to domain signals
│   ├── evidence_gate.py   ← sufficiency check, abstention logic
│   ├── scorer.py          ← embedding-based alignment scoring
│   ├── classifier.py      ← 4-type divergence classifier
│   ├── safety_filter.py   ← blocklist + regex output guard
│   └── reporter.py        ← build output cards + JSON
├── api/
│   └── main.py            ← FastAPI: /analyze, /report/{id}
├── data/examples/
│   ├── alice/             ← overstatement case
│   ├── bob/               ← blind_spot case
│   └── carol/             ← insufficient_evidence case
├── results/               ← scored output JSONs
└── tests/
    ├── test_classifier.py
    ├── test_evidence_gate.py
    ├── test_safety.py
    └── test_scorer.py

---

License

This project is for educational and personal use.