-
Notifications
You must be signed in to change notification settings - Fork 0
feat(explain): arg fill on the token partition seam (#293 B2) #294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| /** | ||
| * B2 arg fill — claims argument names and their `=` on the residual. | ||
| * | ||
| * This is the second B2 fill of #264, added after the operator fill. It runs | ||
| * **after** the proof-only spans (`comment` + `variable-*`) and **before** the | ||
| * operator fill, so the fill order is structural: `spans` claims first, then | ||
| * `arg`, then `operator` sees only what neither wanted. That ordering is why the | ||
| * operator fill can abstain on `, / = -` outside `( )`, on bytes glued after | ||
| * an argument `=`, and on bytes glued into an argument name — those bytes | ||
| * belong to a later fill, not to a smarter operator scanner. | ||
|
mobileskyfi marked this conversation as resolved.
|
||
| * | ||
| * The source is `src/explain/args.ts`'s `Argument` — `span` and `valueSpan` | ||
| * already rebased into document analyzed-byte space by `src/explain.ts`. The | ||
| * name run is `[span.start, valueSpan.start - 1)` and the `=` is the single | ||
| * byte at `valueSpan.start - 1`; the value itself is `valueSpan` and is left | ||
| * for the next fill (see #293). A `positional` has no `name`/`valueSpan` and a | ||
| * `query` is a `?` word whose interior `args.ts` explicitly refuses to split | ||
| * (see `Argument.value`'s doc comment), so both are ignored — the `=` is | ||
| * derived from the token shape, never by scanning for the byte. | ||
| * | ||
| * `Argument.value` is not `Argument.text` and is never read: `value` is absent | ||
| * whenever the source spells a substitution or an escape this phase does not | ||
| * decode. This slice claims `span`/`valueSpan` *positions* only. | ||
| * | ||
| * A statement whose bytes were normalized is not addressable: `explain.ts` | ||
| * already refuses those (`its text was normalized`), and this fill inherits the | ||
| * refusal — it only sees tokens from `read === true` statements, which are | ||
| * exactly the addressable ones. | ||
| * | ||
| * Vocabulary is provisional until #264 B5: one `arg` class for both the name | ||
| * bytes and the `=` separator (design decision 2 of #293 — emit first, name | ||
| * later; `highlight`'s `syntax-meta` is a residual merge and never a source). | ||
| * If B5 splits the `=` into its own class, the byte coverage does not move, | ||
| * only the retag. | ||
| */ | ||
|
|
||
| import type { ExplainArgumentToken, ExplainToken } from "../explain.ts"; | ||
|
|
||
| export type ArgCandidate = ExplainArgumentToken; | ||
|
|
||
| function clipToResidual( | ||
| start: number, | ||
| end: number, | ||
| residual: readonly { start: number; end: number }[], | ||
| ): { start: number; end: number }[] { | ||
| if (start >= end) return []; | ||
| if (residual.length === 0) return []; | ||
| // Binary-search to first residual that could overlap `start` (r.end > start). | ||
| let lo = 0; | ||
| let hi = residual.length - 1; | ||
| let first = residual.length; | ||
| while (lo <= hi) { | ||
| const mid = (lo + hi) >> 1; | ||
| const r = residual[mid] as { start: number; end: number }; | ||
| if (r.end <= start) lo = mid + 1; | ||
| else { | ||
| first = mid; | ||
| hi = mid - 1; | ||
| } | ||
| } | ||
| const out: { start: number; end: number }[] = []; | ||
| for (let i = first; i < residual.length; i++) { | ||
| const r = residual[i] as { start: number; end: number }; | ||
| if (r.start >= end) break; | ||
| const oStart = Math.max(start, r.start); | ||
| const oEnd = Math.min(end, r.end); | ||
| if (oStart < oEnd) out.push({ start: oStart, end: oEnd }); | ||
| } | ||
| return out; | ||
| } | ||
|
mobileskyfi marked this conversation as resolved.
|
||
|
|
||
| /** | ||
| * Argument name + `=` spans on the residual. | ||
| * | ||
| * `analyzed` is the ASCII-normalized document text (only used for length | ||
| * checks); `residual` is the gap set left by earlier fills (sorted, no | ||
| * overlaps); `candidates` are the already-rebased `Argument` tokens from | ||
| * `read === true` statements. Every emitted span's bytes are fully inside | ||
| * `residual`, sorted by `start`, non-overlapping, and carry | ||
| * `class: "arg"` + `ev: "e11"`. | ||
| */ | ||
| export function argSpans( | ||
| analyzed: string, | ||
| residual: readonly { start: number; end: number }[], | ||
| candidates: readonly ArgCandidate[], | ||
| ): ExplainToken[] { | ||
| const len = analyzed.length; | ||
| if (len === 0 || residual.length === 0 || candidates.length === 0) return []; | ||
|
|
||
| const out: ExplainToken[] = []; | ||
|
|
||
| for (const candidate of candidates) { | ||
| if (candidate.kind !== "attribute") continue; | ||
| if (candidate.valueSpan === undefined) continue; | ||
| if (candidate.name === undefined) continue; | ||
| const nameStart = candidate.span.start; | ||
| const eqEnd = candidate.valueSpan.start; | ||
| const eqStart = eqEnd - 1; | ||
| const nameEnd = eqStart; | ||
| // Defensive: token shape must give non-empty name and single-byte `=`. | ||
| if ( | ||
| !Number.isInteger(nameStart) || | ||
| !Number.isInteger(nameEnd) || | ||
| !Number.isInteger(eqStart) || | ||
| !Number.isInteger(eqEnd) | ||
| ) | ||
| continue; | ||
| if (nameStart < 0 || eqEnd > len) continue; | ||
| if (nameStart >= nameEnd) continue; | ||
| // The whole token's `=` byte must be `=` in the analyzed text — a cheap | ||
| // shape check that catches a caller that passed un-rebased or misaligned | ||
| // spans. Not a scan for `=`; the position is derived from `valueSpan`. | ||
| if (analyzed[eqStart] !== "=") continue; | ||
| // Also ensure the candidate lies fully inside the analyzed length. | ||
| if (candidate.span.start < 0 || candidate.span.end > len) continue; | ||
| if (candidate.valueSpan.start < 0 || candidate.valueSpan.end > len) | ||
| continue; | ||
|
|
||
| // Clip each run to the residual — a `variable-*` span may already claim | ||
| // bytes that look like an arg name/value (e.g. `comment=$c` is unread, | ||
| // so it never reaches here, but a future variable inside a quoted value | ||
| // would). Offering only residual keeps `buildTokens`'s cross-fill overlap | ||
| // throw as a safety net rather than a production path. | ||
| // A `variable-*` claiming the `=` does not imply it claims the name | ||
| // bytes, so the name and the `=` are clipped separately. | ||
| const nameClipped = clipToResidual(nameStart, nameEnd, residual); | ||
| const eqClipped = clipToResidual(eqStart, eqEnd, residual); | ||
| for (const r of nameClipped) { | ||
| out.push({ | ||
| start: r.start, | ||
| end: r.end, | ||
| class: "arg" as const, | ||
| ev: "e11", | ||
| }); | ||
| } | ||
| for (const r of eqClipped) { | ||
| out.push({ | ||
| start: r.start, | ||
| end: r.end, | ||
| class: "arg" as const, | ||
| ev: "e11", | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| out.sort((a, b) => a.start - b.start || a.end - b.end); | ||
| // Coalesce adjacent `arg` runs that were split only by the name/`=` boundary | ||
| // when both were residual: `[name][=]` of the same attribute becomes one | ||
| // maximal `arg` run rather than two adjacent tokens of the same class. The | ||
| // two bytes remain distinguishable by the `=` position, but the partition | ||
| // itself need not multiply tokens for a single name+separator. | ||
| const coalesced: ExplainToken[] = []; | ||
| for (const t of out) { | ||
| const last = coalesced[coalesced.length - 1]; | ||
| if ( | ||
| last !== undefined && | ||
| last.class === t.class && | ||
| last.ev === t.ev && | ||
| last.end === t.start | ||
| ) { | ||
| last.end = t.end; | ||
| } else { | ||
| coalesced.push({ ...t }); | ||
| } | ||
| } | ||
| return coalesced; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.