-
Notifications
You must be signed in to change notification settings - Fork 0
feat(explain): arg fill on the token partition seam (#293 B2) #294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| /** | ||
| * B2 arg fill — claims argument names and their `=` on the residual. | ||
| * | ||
| * This is the second B2 fill of #264, added after the operator fill. It runs | ||
| * **after** the proof-only spans (`comment` + `variable-*`) and **before** the | ||
| * operator fill, so the fill order is structural: `spans` claims first, then | ||
| * `arg`, then `operator` sees only what neither wanted. That ordering is why the | ||
| * operator fill can abstain on `, / = -` outside `( )`, on bytes glued after | ||
| * an argument `=`, and on bytes glued into an argument name — those bytes | ||
| * belong to a later fill, not to a smarter operator scanner. | ||
|
mobileskyfi marked this conversation as resolved.
|
||
| * | ||
| * The source is `src/explain/args.ts`'s `Argument` — `span` and `valueSpan` | ||
| * already rebased into document analyzed-byte space by `src/explain.ts`. The | ||
| * name run is `[span.start, valueSpan.start - 1)` and the `=` is the single | ||
| * byte at `valueSpan.start - 1`; the value itself is `valueSpan` and is left | ||
| * for the next fill (see #293). A `positional` has no `name`/`valueSpan` and a | ||
| * `query` is a `?` word whose interior `args.ts` explicitly refuses to split | ||
| * (see `Argument.value`'s doc comment), so both are ignored — the `=` is | ||
| * derived from the token shape, never by scanning for the byte. | ||
| * | ||
| * `Argument.value` is not `Argument.text` and is never read: `value` is absent | ||
| * whenever the source spells a substitution or an escape this phase does not | ||
| * decode. This slice claims `span`/`valueSpan` *positions* only. | ||
| * | ||
| * A statement whose bytes were normalized is not addressable: `explain.ts` | ||
| * already refuses those (`its text was normalized`), and this fill inherits the | ||
| * refusal — it only sees tokens from `read === true` statements, which are | ||
| * exactly the addressable ones. | ||
| * | ||
| * Vocabulary is provisional until #264 B5: one `arg` class for both the name | ||
| * bytes and the `=` separator (design decision 2 of #293 — emit first, name | ||
| * later; `highlight`'s `syntax-meta` is a residual merge and never a source). | ||
| * If B5 splits the `=` into its own class, the byte coverage does not move, | ||
| * only the retag. | ||
| */ | ||
|
|
||
| import type { ExplainArgumentToken, ExplainToken } from "../explain.ts"; | ||
|
|
||
| export type ArgCandidate = ExplainArgumentToken; | ||
|
|
||
| function clipToResidual( | ||
| start: number, | ||
| end: number, | ||
| residual: readonly { start: number; end: number }[], | ||
| ): { start: number; end: number }[] { | ||
| if (start >= end) return []; | ||
| if (residual.length === 0) return []; | ||
| // Binary-search to first residual that could overlap `start` (r.end > start). | ||
| let lo = 0; | ||
| let hi = residual.length - 1; | ||
| let first = residual.length; | ||
| while (lo <= hi) { | ||
| const mid = (lo + hi) >> 1; | ||
| const r = residual[mid] as { start: number; end: number }; | ||
| if (r.end <= start) lo = mid + 1; | ||
| else { | ||
| first = mid; | ||
| hi = mid - 1; | ||
| } | ||
| } | ||
| const out: { start: number; end: number }[] = []; | ||
| for (let i = first; i < residual.length; i++) { | ||
| const r = residual[i] as { start: number; end: number }; | ||
| if (r.start >= end) break; | ||
| const oStart = Math.max(start, r.start); | ||
| const oEnd = Math.min(end, r.end); | ||
| if (oStart < oEnd) out.push({ start: oStart, end: oEnd }); | ||
| } | ||
| return out; | ||
| } | ||
|
|
||
| /** | ||
| * Argument name + `=` spans on the residual. | ||
| * | ||
| * `analyzed` is the ASCII-normalized document text (only used for length | ||
| * checks); `residual` is the gap set left by earlier fills (sorted, no | ||
| * overlaps); `candidates` are the already-rebased `Argument` tokens from | ||
| * `read === true` statements. Every emitted span's bytes are fully inside | ||
| * `residual`, sorted by `start`, non-overlapping, and carry | ||
| * `class: "arg"` + `ev: "e11"`. | ||
| */ | ||
| export function argSpans( | ||
| analyzed: string, | ||
| residual: readonly { start: number; end: number }[], | ||
| candidates: readonly ArgCandidate[], | ||
| ): ExplainToken[] { | ||
| const len = analyzed.length; | ||
| if (len === 0 || residual.length === 0 || candidates.length === 0) return []; | ||
|
|
||
| const out: ExplainToken[] = []; | ||
|
|
||
| for (const candidate of candidates) { | ||
| if (candidate.kind !== "attribute") continue; | ||
| if (candidate.valueSpan === undefined) continue; | ||
| if (candidate.name === undefined) continue; | ||
| const nameStart = candidate.span.start; | ||
| const eqEnd = candidate.valueSpan.start; | ||
| const eqStart = eqEnd - 1; | ||
| const nameEnd = eqStart; | ||
| // Defensive: token shape must give non-empty name and single-byte `=`. | ||
| if ( | ||
| !Number.isInteger(nameStart) || | ||
| !Number.isInteger(nameEnd) || | ||
| !Number.isInteger(eqStart) || | ||
| !Number.isInteger(eqEnd) | ||
| ) | ||
| continue; | ||
| if (nameStart < 0 || eqEnd > len) continue; | ||
| // Non-empty name. This is also what rejects `valueSpan.start === 0`, where | ||
| // `eqStart` would be -1: `nameEnd` is -1 too, so any `nameStart >= 0` trips | ||
| // it. `analyzed[-1]` on the next line would be `undefined` and continue | ||
| // anyway, so the order is safe either way — but the name check is the one | ||
| // that carries the intent. | ||
| if (nameStart >= nameEnd) continue; | ||
| // The whole token's `=` byte must be `=` in the analyzed text — a cheap | ||
| // shape check that catches a caller that passed un-rebased or misaligned | ||
| // spans. Not a scan for `=`; the position is derived from `valueSpan`. | ||
| if (analyzed[eqStart] !== "=") continue; | ||
| // Also ensure the candidate lies fully inside the analyzed length. | ||
| if (candidate.span.start < 0 || candidate.span.end > len) continue; | ||
| if (candidate.valueSpan.start < 0 || candidate.valueSpan.end > len) | ||
| continue; | ||
|
|
||
| // The name run and the `=` are contiguous (`nameEnd === eqStart`), so | ||
| // clip them as ONE range per candidate. That yields the maximal `[name=]` | ||
| // run for free — split only where the residual itself has a hole — and | ||
| // makes cross-candidate fusion structurally impossible rather than merely | ||
| // unreachable: coalescing adjacent runs globally would merge two distinct | ||
| // attributes into one token if `lexArguments` ever stopped guaranteeing a | ||
| // separator between them. A `variable-*` span claiming only the `=` still | ||
| // leaves the name run behind, because the residual hole does the splitting. | ||
| for (const r of clipToResidual(nameStart, eqEnd, residual)) { | ||
| out.push({ | ||
| start: r.start, | ||
| end: r.end, | ||
| class: "arg" as const, | ||
| ev: "e11", | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| out.sort((a, b) => a.start - b.start || a.end - b.end); | ||
| return out; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.