fix: wrap extraction transcript turns in speaker tags to stop assistant misattribution#964
Conversation
rwmjhb
left a comment
There was a problem hiding this comment.
The speaker-tag direction is sound, and both the full suite and a clean source/dist build pass. One user-facing regression blocks merge:
- For the supported two-step flow where a user states a fact and then says
remember this, the code prepends the prior message only totexts.buildConversationTurnsForExtraction()still receives the originalnewTexts, andSmartExtractorprefers any nonemptyconversationTurnsover the flatconversationText. The actual extraction prompt therefore contains onlyremember this, not the fact it references; if no candidate is produced, the fallback is skipped as well.
Please build the tagged turns from the final text sequence (or explicitly reconstruct the prepended prior turn) and add an integration assertion against the real extraction prompt for this flow.
Two follow-ups are worth covering in the same area: session compression currently changes texts after the turn list is built, so the tagged prompt can bypass the compressor selection; and a single turn longer than extractMaxChars returns a suffix with a closing tag but no opening speaker tag. Neither should be allowed to undermine the new attribution invariant.
|
All three items are addressed on the new head:
All three regressions fail on the previous head and pass on this one; tsc, the touched suites, and the full chain are green, and dist is rebuilt at the head. |
…ant misattribution
With "User:"/"Assistant:" line prefixes only the first line of a message
carries a speaker marker, so a multi-paragraph assistant reply sheds its
marker after the first paragraph and the extractor attributes
assistant-authored plans and preferences to the user. Wrap every message
wholly in <user_message>/<assistant_message> blocks built from the hook's
role-tagged message-loop order, neutralize literal speaker tags typed inside
message content, and snap extractMaxChars truncation to a tag boundary so a
sliced transcript never leads with headless text.
The extraction prompt becomes a {system, user} split so the format teaching
and grounding rules ride the system half while the transcript rides the user
half: captureAssistant=false grounds memories exclusively in user blocks
(assistant lines never enter the transcript); captureAssistant=true keeps
assistant blocks as attributable sources with explicit attribution rules.
completeJson gains an optional per-call system prompt to carry the split.
…on, and truncation re-heading into the tagged transcript
f1a8f32 to
f56499e
Compare
Problem
The extraction transcript renders turns as
User:/Assistant:line prefixes, so only the FIRST line of a message carries a speaker marker. A multi-paragraph assistant reply sheds its marker after the first paragraph, and the extractor attributes assistant-authored plans, preferences, and self-descriptions to the user and stores them as user memories. Live example that motivated this: an assistant's multi-paragraph explanation of how memory layers work came back as "User believes manual notes are only for rare big items" style rows.Change
<user_message>/<assistant_message>blocks, built from the capture hook's role-tagged message-loop order, so every line has an unambiguous owner.{system, user}split: the transcript-format teaching and grounding rules ride the system half, the tagged transcript rides the user half.completeJsongains an optional per-call system prompt to carry it (the default generic system message is unchanged for all other callers).captureAssistant: withfalse(default) assistant lines never enter the transcript and memories may only be grounded in<user_message>blocks; withtrueassistant blocks are attributable sources with explicit attribution rules ("attribute every memory to whoever actually said it").extractMaxCharstruncation snaps to a tag boundary, so a sliced transcript never opens with headless text.Tests
New
test/extraction-transcript-speaker-tags.test.mjs(16 tests): whole-message wrapping, multi-paragraph containment, chronology, spoof neutralization, boundary trimming, turn assembly (watermark tail-slice, mixed-role alignment, pair-aware skip, ingress replay fallback), and prompt teaching in both modes. Existing extraction suites updated for the new prompt shape; registered in the test chain and CI manifest.