Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@ Entries that change an on-disk format or a response shape say so.
## [Unreleased]

### Added
- `taguru extract --source-id ID`, `--date WHEN`, `--tag TAG` (#466
S1, ADR 0017): bake the promotion runbook's source conventions into
the written batch — the `session:{agent}:{id}` header source (with
the `/{doc}` stem suffix across several documents, collisions
refused), and the passage line's `date` (`YYYY-MM-DD` or epoch
seconds) and `tags`. All three are manifest computation inputs
(`serde(default)` — older manifests keep matching default runs) but
deliberately not checkpoint inputs: a metadata change rewrites the
batch while reusing every cached chunk answer. `--date`/`--tag` with
`--no-passage` is a usage error (metadata rides the passage line).
- `taguru extract --coverage` / `TAGURU_EXTRACT_COVERAGE` (#496 S4,
ADR 0016): report every sentence that holds a candidate pair (two or
more deterministically segmented document names) yet is covered by
Expand Down
83 changes: 83 additions & 0 deletions adr/0017-runbook-metadata-at-extract-time.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# 0017. Promotion-runbook metadata at extract time

- **Status**: Accepted
- **Date**: 2026-08-09
- **Issue**: #466 (S1)
- **Related**: #465 (the runbook whose conventions these flags encode),
ADR 0011 (why `date` is load-bearing), ADR 0005 (the batch contract
the metadata rides), issue #167 (the passage-line metadata fields)
- **Supersedes**: — / **Superseded by**: —

Once Accepted, this document's Decision is immutable: a changed decision gets a
new `adr/000N-*.md` that names this one in *Supersedes*, never an edit here.

## 1. Scope

How `taguru extract` writes the promotion runbook's source conventions
— session source id, assertion date, topic tags — into the batches it
emits (#466 S1). Out of scope: bundling the promotion sequence itself
(a verb or MCP tool, #466's remaining splits), and the SDK producers.

## 2. Context

The 2026-08-08 runbook rehearsal (#466's gate record) found the single
most mechanical step of every promotion: extract knows only document
paths, so the operator rewrites each emitted batch's `source` to the
runbook's `session:{agent}:{id}`, and hand-adds the `date` and `tags`
the scratch conventions require — every time, purely mechanically,
with a text editor against a generated file. The import wire format
has carried all three fields since #167; extract simply never had a
way to be told them.

## 3. Decision

**`--source-id ID`, `--date WHEN`, and `--tag TAG` (repeatable) bake
the runbook's conventions into the written batch. All three are
manifest computation inputs and none is a checkpoint input.**

1. **`--source-id` replaces the header's source**: verbatim for a
single document; with several, each document gets `ID/{file stem}`
— the runbook's own `/{doc}` convention, made automatic because
import's retract-then-apply is per source id, and one id covering
two documents would silently fold them. Two documents whose stems
collide fail the second with the reason, before any model call.
The manifest stays keyed by the document PATH — the path names the
input; this names the output.
2. **`--date` and `--tag` ride the passage line**, exactly where the
wire format carries source metadata. Requiring the passage is
therefore enforced as a usage error against `--no-passage`, not a
silent drop — an associations-only source stores no metadata and is
invisible to every windowed read (docs/promotion.html's own
warning). `--date` accepts epoch seconds (the wire unit) or
`YYYY-MM-DD` (what a session note records — that day's UTC
midnight, round-tripped through the rendering direction so a
non-existent date is refused rather than normalized).
3. **Manifest inputs, not checkpoint inputs**: all three are baked
into the emitted file, so a change must rewrite the batch (the
`context`/`description` precedent — a skip would leave the old id
or date in place). But none of them reaches the prompt — the model
is still shown the document path — so cached chunk answers stay
reusable across a metadata change: the rewrite costs zero model
calls for checkpointed units. The fingerprint records the EFFECTIVE
written source (suffix included), so revising the suffix scheme
re-extracts too; `""`/`0`/`[]` are the off values, keeping pre-S1
manifest entries matching default runs.
4. **Flag-only, no env counterparts**: a session id and its date are
per-invocation values, not deployment settings — the
`--context`/`--description` precedent, so `KNOWN_KEYS` and the
config dialect are untouched.

## 4. Consequences

- The runbook's step 2 loses its hand-editing: extract now emits an
import-ready promotion batch directly, and the `#496` controls
(`--vocabulary` for the resolve-first rule, `--coverage` for the
review's mechanical floor) compose with it — the flags are
orthogonal by construction.
- Batches, manifests, and checkpoints from before this change parse
and match unchanged (`serde(default)` on the new manifest fields;
the no-flags batch is byte-for-byte identical).
- Rust-only, like every extract control; the SDK producers inherit the
whole set together in their own follow-up.
- The remaining #466 splits (an MCP promotion tool over the graph
path; a CLI text-path preset) build on this without changes here.
12 changes: 12 additions & 0 deletions docs/extract.html
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,18 @@ <h2>The CLI shape</h2>
usage, latency, parse/validation issues), one per
document written (association/alias/duplicate/dropped/
uncovered counts); off by default, ignored under --dry-run
--source-id ID write ID as the batch header's source instead of the
document path — the promotion runbook's
session:{agent}:{id} convention; several documents each
get ID/{file stem}, and a collision fails (import
retracts-then-applies per source id). Changing it
rewrites the batch but reuses cached chunk answers
(ADR 0017)
--date WHEN the session's own date, written on the passage line:
YYYY-MM-DD (UTC midnight) or positive epoch seconds;
needs the passage
--tag TAG tag the batch's source (repeatable, deduplicated),
written on the passage line; needs the passage
--context NAME the context every batch file targets
--description TEXT attach a create block (used only when the context is absent)
--schema FILE a context schema document (see below) — the same shape
Expand Down
13 changes: 10 additions & 3 deletions docs/promotion.html
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,16 @@ <h2 id="procedure">The promotion procedure</h2>
scratch has grown.</li>
<li><b>Extract the keepers</b> with <code>taguru extract</code> over the session
passages (or hand-write the batch), targeting the <b>permanent</b> context — keeping
the <code>session:{agent}:{id}</code> source ids and the <code>date</code>s. Resolve
spellings against the permanent context first (<code>resolve</code> /
<code>resolve_label</code>): reuse its vocabulary, never fork it.</li>
the <code>session:{agent}:{id}</code> source ids and the <code>date</code>s:
<code>--source-id</code>, <code>--date</code>, and <code>--tag</code> bake all three
conventions into the emitted batch directly (ADR 0017), so no hand-editing of the
generated file remains. Resolve spellings against the permanent context rather than
forking its vocabulary: export it (<code>taguru export --out DIR</code>) and hand the
export to <code>--vocabulary</code> (ADR 0015), which steers the extraction toward
the context's existing spellings and admits them through validation — the manual
<code>resolve</code>/<code>resolve_label</code> pass remains for hand-written
batches. <code>--coverage</code> (ADR 0016) reports what the extraction left behind,
sentence by sentence — the mechanical floor under the review this step owes.</li>
<li><b>Import</b> via <code>POST /import</code> / <code>taguru import</code> —
retract-then-apply per source makes re-promoting the same session idempotent.</li>
<li><b>Audit the landing zone</b>: <code>taguru consolidation --context NAME</code>
Expand Down
27 changes: 27 additions & 0 deletions src/clock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,21 @@ pub(crate) fn iso8601_utc(unix_secs: u64) -> String {
format!("{y:04}-{m:02}-{d:02}T{hh:02}:{mm:02}:{ss:02}Z")
}

/// [`civil_from_days`]'s inverse (Hinnant's `days_from_civil`, same
/// source): (year, month, day) → days since the Unix epoch. Consumed
/// by `extract --date`'s `YYYY-MM-DD` spelling (#466 S1), which
/// round-trips the result through [`iso8601_utc`] to refuse
/// non-existent dates rather than silently normalizing them.
pub(crate) fn days_from_civil(y: i64, m: u32, d: u32) -> i64 {
let y = if m <= 2 { y - 1 } else { y };
let era = if y >= 0 { y } else { y - 399 } / 400;
let yoe = (y - era * 400) as u64;
let mp = if m > 2 { m - 3 } else { m + 9 } as u64;
let doy = (153 * mp + 2) / 5 + d as u64 - 1;
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
era * 146097 + doe as i64 - 719468
}

fn civil_from_days(z: i64) -> (i64, u32, u32) {
let z = z + 719468;
let era = if z >= 0 { z } else { z - 146096 } / 146097;
Expand All @@ -55,4 +70,16 @@ mod tests {
assert_eq!(iso8601_utc(1709164800), "2024-02-29T00:00:00Z");
assert_eq!(iso8601_utc(951868800), "2000-03-01T00:00:00Z");
}

#[test]
fn days_from_civil_inverts_the_rendering_direction() {
assert_eq!(days_from_civil(1970, 1, 1), 0);
assert_eq!(days_from_civil(2026, 7, 26), 1785057262 / 86400);
assert_eq!(days_from_civil(2024, 2, 29), 1709164800 / 86400);
assert_eq!(days_from_civil(2000, 3, 1), 951868800 / 86400);
assert_eq!(days_from_civil(1969, 12, 31), -1);
// The negative-year era branch (`y - 399`): year 0 is a leap
// year (≡ 2000 mod 400), so -1-03-01 → 0-03-01 spans 366 days.
assert_eq!(days_from_civil(0, 3, 1) - days_from_civil(-1, 3, 1), 366);
}
}
23 changes: 23 additions & 0 deletions src/extract.rs
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ use vocabulary::{ContextVocabulary, context_names_block, load_vocabulary};
#[cfg(test)]
use aggregate::{cross_output_issues, schema_output_issues};
#[cfg(test)]
use args::parse_date;
#[cfg(test)]
use candidates::{CANDIDATE_CAP, CANDIDATE_MAX_BYTES};
#[cfg(test)]
use chat_client::build_chat_body;
Expand All @@ -183,6 +185,7 @@ usage: taguru extract [--dry-run] [--force] [--no-passage] [--questions N]
[--structured-output MODE] [--max-output-tokens N]
[--lossy] [--candidates] [--vocabulary PATH] [--coverage]
[--diagnostics-out FILE] [--schema FILE]
[--source-id ID] [--date WHEN] [--tag TAG]...
--context NAME [--description TEXT] --out DIR FILE|DIR...

Reads documents (.md/.txt; a directory expands to its files, sorted by
Expand Down Expand Up @@ -280,6 +283,21 @@ chat endpoint:
across runs. Default (unset): no sidecar, stdout/
stderr unchanged. Ignored under --dry-run, which
calls nothing to record.
--source-id ID write ID as the batch header's source instead of the
document path — the promotion runbook's
session:{agent}:{id} convention (docs/promotion.html).
With several documents, each gets ID/{file stem}; two
documents landing on one source id is an error (import
retracts-then-applies per source id). Changing it
rewrites the batch but reuses cached chunk answers
--date WHEN the session's own date, written on the batch's passage
line (the assertion time windowed reads and the
staleness audit run on): YYYY-MM-DD (UTC midnight) or
positive epoch seconds. Needs the passage
--tag TAG tag the batch's source (repeatable, deduplicated) —
written on the passage line; how a later session finds
its trail via passage search's tags filter. Needs the
passage
--context NAME the context every batch file targets
--description TEXT add a create block (used only if the context is absent)
--schema FILE the target context's schema document (same shape as
Expand Down Expand Up @@ -718,6 +736,11 @@ pub fn run(args: &[String]) -> i32 {
.as_ref()
.map(|vocabulary| vocabulary.labels.clone())
.unwrap_or_default(),
source_id: args.source_id,
date: args.date,
tags: args.tags,
multi_document: files.len() > 1,
claimed_source_ids: BTreeMap::new(),
claimed: BTreeMap::new(),
parallel,
lossy,
Expand Down
Loading