Skip to content

Commit already-collapsed text in bulk in TreeBuilder::push_text - #63

Open
nicoburns wants to merge 4 commits into
devin/1790425683-tree-builder-text-so-farfrom
devin/1790443049-bulk-commit-collapsed-text
Open

nicoburns wants to merge 4 commits into
devin/1790425683-tree-builder-text-so-farfrom
devin/1790443049-bulk-commit-collapsed-text

Conversation

@nicoburns

Copy link
Copy Markdown
Member

LLM Contributions: Code and this description were written by Devin (an LLM agent). The description needs rewriting by a human before this goes upstream.

Stacked on #62 (base is its branch).

In Collapse and PreserveBreaks modes, push_text stopped a text run at every collapsible character, so ordinary prose was committed one word at a time (each word going through flush_pending_whitespace, resolve_style_id, and commit_styled_text). A single space followed by content that needs no processing is already collapsed, so it can be committed as part of the run.

New collapsed_len(text, mode) replaces the find(is_collapsible || is_segment_break) that measured the text run:

scan bytes; stop at byte i if needs_processing(i) &&
    (bytes[i] != b' ' || i is last || needs_processing(i + 1))
where needs_processing = collapsible ASCII byte || CR || LF || LS/PS (E2 80 A8/A9)

So collapsing only runs at double spaces, tabs, newlines, LS/PS, and at the ends of the text. A trailing space still becomes pending whitespace, so behaviour across push_text calls and style spans is unchanged. The pending-whitespace, style and wrappability logic is untouched. #[inline(always)] is needed: without it, 1-char-per-push benchmarks were 12–30% slower.

Testing:

  • Existing resolve::tree tests pass unchanged.
  • A throwaway differential test (not included) compared against the old implementation on 300k random sequences of push_text, push_style_span, and pop_style_span. It mixed all four collapse modes, both wrap modes, and CR/LF/tab/FF/U+2028/2029/3000/lone E2 input. Text, style runs and style indices were identical. Deliberately breaking collapsed_len made the test fail.

Tango TreeBuilder push, compared paired against #62:

  • Collapse, 10 chars per push, latin/arabic 400–8000 chars: 32–52% faster (e.g. latin 8000, 8 pushes/span: 102 µs → 53 µs).
  • 20-char inputs: neutral to 19% faster.
  • 1 char per push: neutral (0–5%).
  • Japanese (no spaces, so nothing to merge): neutral to about 8% slower. For comparison, the Preserve rows, which run unchanged code, moved by ±4–12%.
  • Preserve: unchanged code path.

Standalone 11.6 KB single-push benchmark, Collapse mode:

  • ASCII prose: 39.7 µs → 11.0 µs.
  • German prose: 37.3 µs → 10.9 µs.
  • Indented HTML-like source: 36.9 µs → 14.5 µs.

Changelog: None

Link to Devin session: https://dioxus.staging.devinenterprise.com/sessions/c0f395eb38ed46d787feb525a9001029
Open in Devin Desktop: https://dioxus.staging.devinenterprise.com/desktop/session/c0f395eb38ed46d787feb525a9001029?variant=devin-insiders
Requested by: @nicoburns

White space processing no longer needs to see a span's text all at once,
so push_text processes text immediately. This makes text_so_far a
read-only getter.
@staging-devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

@staging-devin-ai-integration
staging-devin-ai-integration Bot force-pushed the devin/1790425683-tree-builder-text-so-far branch 2 times, most recently from fe83c41 to 6e57520 Compare September 28, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant