diff --git a/packages/dds/tree/src/text/README.md b/packages/dds/tree/src/text/README.md index f44318b3b009..8dd0b55cecea 100644 --- a/packages/dds/tree/src/text/README.md +++ b/packages/dds/tree/src/text/README.md @@ -5,4 +5,54 @@ A collection of text related types, schema and utilities for working with text b ## Status Very early work in progress. -See doc comment on `TextAsTree` for more details. \ No newline at end of file +See doc comment on `TextAsTree` for more details. + +## Extrinsic Ranges + +### Challenges: + +Editing semantics: + +Assuming we want the behavior where Extrinsic Ranges have lifetime of tied to the container, not the content, +then deletions need to shrink the range they apply to, but never remove it fully. +This complications deletes (which contain all or part of the range) and moves. +Presumably a delete should just shorten the range, as should detaches, but if a portion is detached containing the entire range, and moved elsewhere in the same range, maybe it should move? +Maybe optionally detect such moves, and if some constraints pass, move the ranges with the main content? +Relations to anchors, cursors, presense? + +Designs: + +parallel array: + - Anchor marker at location in placeholder array: doesn't handle deletes well. + - before and after tokens in parallel array: how do inserts produce the correct token type? How about moves? + + +index field kind: +- Reuse code from sequence field kind (identical changeset logic, including rebase and ops) +- Child is just a number representing an index + +Can we do better anchoring for better semantics? Does this require knowing both start and end, and maybe extra config inside the field, not just a single number? + +### Bulk editing: + +How does this work in our setup? Allow all nodes (or maybe fields) in changesets to be roots for structural bulk edits? +Do we need to make it more limited, like can only do edits which don't change the tree shape? (Like replace leaf nodes) + + +Extrinsic Ranges Update: + + + +After some initial design exploration, I think its possible to implement: + +An extrinsic range MVP leveraging the "NoChangeConstraint" which can maintain its data invariants. I think we can do this with no currently unreleased features (just needs sufficient min version for collab for the constraint). This will not be robust to AI editing of the data generally, but we can expose methods which are safe for it to use. (Won't work for agents without view schema, but I don't think that's a priority). We could either make the implementation minimal (just store index numbers) or a parallel array approach. Items below will assume the array version, but I'll provide a comparison of the approaches in more detail later. +Optimized encoding for the extrinsic ranges (Brennan's current work for optimizing text's codec should actually be sufficient) +Optimized in memory format and traversal for the parallel arrays similar to the encoded format for chunked forest. +Add a "no shallow change" constraint (also desired for tables): when using the parallel array approach, this can allow concurrent editing of the main range/string as well as comments: you just can't concurrently add and/or remove ranges. +Bulk editing op, to allow expressing an update to all ranges at once with a wild card. When this is supported we can disable the shallow change constraint and get full concurrency. I think there might still be some issues with edge cases around un-removal (like delete undo): that needs some further investigation. Part of this (which could be delivered first and not avoid the need for the constraint) could be done separately and makes the Ops more efficient and (which is important as each character typed produces one) and not scale with the number of ranges. + +All of these can be done in parallel. + + + +The 1-4 subset has relatively few unknowns (mostly around the now shallow change constraint, and possibly some complications with undo/redo) does not need any new kinds of things we don't know how to incrementally add/maintain/version (Just an optimized codec, and a new constraint gated by min version for collab), and should be sufficient as long as the number of ranges isn't particularly large. \ No newline at end of file diff --git a/packages/dds/tree/src/text/arrayWithExtrinsicRangesDomain.ts b/packages/dds/tree/src/text/arrayWithExtrinsicRangesDomain.ts new file mode 100644 index 000000000000..ecea9dc92634 --- /dev/null +++ b/packages/dds/tree/src/text/arrayWithExtrinsicRangesDomain.ts @@ -0,0 +1,348 @@ +/*! + * Copyright (c) Microsoft Corporation and contributors. All rights reserved. + * Licensed under the MIT License. + */ + +import { compareArrays, debugAssert } from "@fluidframework/core-utils/internal"; +import { + buildFunc, + exposeMethodsSymbol, + type ExposedMethods, + type IExposedMethods, + // eslint-disable-next-line import-x/no-internal-modules +} from "@fluidframework/type-factory/alpha"; +import { typeFactory as tf } from "@fluidframework/type-factory/internal"; + +import { EmptyKey, mapCursorField, type ITreeCursorSynchronous } from "../core/index.js"; +import { TreeAlpha } from "../shared-tree/index.js"; +import { + eraseSchemaDetails, + getInnerNode, + SchemaFactory, + SchemaFactoryAlpha, + TreeArrayNode, +} from "../simple-tree/index.js"; +import type { + ArrayNodeDeltaOp, + ArrayNodeTreeChangedDeltaOp, + TreeNode, + WithType, + // eslint-disable-next-line import-x/no-duplicates +} from "../simple-tree/index.js"; + +// Add some unused imports which show up in the generated d.ts file. +// This prevents them from getting inline imports generated, cleaning up the d.ts file and API reports. +// eslint-disable-next-line @typescript-eslint/no-unused-vars, unused-imports/no-unused-imports, import-x/no-duplicates +import type { NodeKind, TreeNodeSchema } from "../simple-tree/index.js"; + +const sf = new SchemaFactoryAlpha("com.fluidframework.extrinsicRanges"); + +class ExtrinsicRangeManager extends sf.object("Manager", { + content: SchemaFactory.required([() => ExtrinsicRangeManagerInner], { key: EmptyKey }), +}) { + public insertAt(index: number, additionalCharacters: string): void { + this.content.insertAt( + index, + TreeArrayNode.spread(charactersFromString(additionalCharacters)), + ); + } + public removeRange(index: number | undefined, end: number | undefined): void { + this.content.removeRange(index, end); + } + + public static fromString(value: string): ExtrinsicRangeManager { + // Constructing an ArrayNode from an iterator is supported, so creating an array from the iterable of characters seems like its not necessary here, + // but to reduce the risk of incorrect data interpretation, we actually ban this in the special case where the iterable is a string directly, which is the case here. + // Thus the array construction here is necessary to avoid a runtime error. + return new ExtrinsicRangeManager({ content: [...charactersFromString(value)] }); + } +} + +class ExtrinsicRangeManagerInner extends sf.object("ManagerInner", { + content: SchemaFactory.required([() => MainArray], { key: EmptyKey }), + ranges: ExtrinsicRanges, +}) { + public insertAt(index: number, additionalCharacters: string): void { + this.content.insertAt( + index, + TreeArrayNode.spread(charactersFromString(additionalCharacters)), + ); + } + public removeRange(index: number | undefined, end: number | undefined): void { + this.content.removeRange(index, end); + } + + public static fromString(value: string): ExtrinsicRangeManager { + // Constructing an ArrayNode from an iterator is supported, so creating an array from the iterable of characters seems like its not necessary here, + // but to reduce the risk of incorrect data interpretation, we actually ban this in the special case where the iterable is a string directly, which is the case here. + // Thus the array construction here is necessary to avoid a runtime error. + return new ExtrinsicRangeManager({ content: [...charactersFromString(value)] }); + } +} + +class MainArray extends sf.array("StringArray", SchemaFactory.string) { + public withBorrowedSequenceCursor(f: (cursor: ITreeCursorSynchronous) => T): T { + const cursor = getInnerNode(this).borrowCursor(); + cursor.enterField(EmptyKey); + const result = f(cursor); + cursor.exitField(); + return result; + } + + public charactersCopy(): string[] { + return this.withBorrowedSequenceCursor((cursor) => + mapCursorField(cursor, () => cursor.value as string), + ); + } + + public fullString(): string { + return this.charactersCopy().join(""); + } +} + +/** + * To atomically edit all of these in parallel we can either: + * 1. Break collab using constraints: ideally we could add a shallow now change constraint on this array so concurrent edits on the main content are allowed, just not current edits of which ranges exist. + * 2. Use bulk editing + */ +class ExtrinsicRanges extends sf.array("ExtrinsicRanges", [() => ExtrinsicRange]) {} + +class ExtrinsicRange extends sf.object("ExtrinsicRange", { + data: [() => Data], + + // Atomically maintaining these can be done without constraints if we add a field kind which allows some math operations which match index behavior: + // Mainly for: + // insert: add a specified constant if current value is > a specified value. + // remove: subtract a specified constant if current value is > a specified value. + // while the specified values are also updated be rebase like array indexes. + start: SchemaFactory.number, + end: SchemaFactory.number, +}) {} + +class Empty extends sf.object("Empty", {}) {} + +class Marker extends sf.object("Marker", {}) {} + +/** + * Tracks a location in a array by making a parallel array. + * This could be compressed to just a length and index of the marker in the common case (not corrupted). + * Even simple runs of matching shapes compression we have for persisted format should compress this to O(log(Length)) by compressing it to three runs each with a shape and length. + */ +class ArrayAnchor extends sf.array("ArrayAnchor", [Marker, Empty]) {} + +export class ExtrinsicRange2 extends sf.object("ExtrinsicRange", { + data: [() => Data], + start: ArrayAnchor, + end: ArrayAnchor, +}) {} + +/** + * Replace with generic parameter + */ +const Data = SchemaFactory.string; + +/** + * A collection of text related types, schema and utilities for working with text beyond the basic {@link SchemaStatics.string}. + * @privateRemarks + * Currently this API only supports a really minimal feature set, and has no support for more advanced features like: + * - Alternative character boundaries (e.g. grapheme clusters, paragraphs, tokens, etc.). + * We may want to provide either ways to create strings with application controlled character boundaries since there is not a clear single answer on how to break a string into atomic units. + * - Character attributes (e.g. bold, italic, etc): + * Properties that can be set on any character independently with optimizations for runs of characters with the same attributes. + * - Inline objects (e.g. images, embedded components, etc): + * These would be logically part of the text, generalizing characters to allow inline objects in character ranges. + * How character attributes apply to inline objects is an open question + * (there could be a kind of object which gets them, and one that doesn't for example). + * - Annotations (e.g. comments, suggestions, etc). + * Objects which can be associated with a range of characters but are not logically part of the text. + * These would need to have the logical range they apply to updated by edits. + * How edits which overlap annotation boundaries are handled may require hints from the application for optimal behavior (mainly inserts at the boundaries). + * These get a lifetime tied to the text node, not any of the characters the annotation covers, + * however it might be desirable to have a way for a range edit to (optionally) also remove any annotations which are fully covered by the edit. + * Annotations over an empty range should also be supported and behave well (for example not end up with characters inside the range after edits unless specifically structured so that makes sense). + * - Anchors (e.g. positions in the text which survive edits). + * These would be useful for ephemeral state like cursor positions, but should match the behaviors with respect to edits exhibited by the ends of Annotations. + * + * How these features will be represented in the schema and API should be determined before any of this is stabilized so the simple more limited version can neatly fit into the larger design. + * + * There are various optimizations that should be implemented to make this performant for large texts and common usage patterns. + * These include: + * - Optimized persisted format. + * - Optimized in memory representation (via chunked forest). + * - Optimized edit persisted format (e.g. combining adjacent inserts/removes into single operations as well as support for efficient attribute editing of ranges). + * - Optimized edit application (e.g. applying the above noted optimizable edit cases efficiently). + * This applies to the revision manager, the forest, and any flex or simple-tree nodes and user events. + * + * There are also additional features required for ensuring the invariants of collaborative text editing are maintained through concurrent edits. + * The main challenges here are related to annotations, but some policies for what to do in the case of corrupt/invalid text should also be included. + * There are quite a few ways invariants could break, including: + * - concurrent edits without proper constraints. + * - collaboration with clients using compatible schema with different constraints. + * - opening documents which contain invalid content (e.g. from older versions of the software, manual edits to the persisted format, or simply an existing corrupt case which was saved). + * - a user inserting/constructing/importing invalid content. + * + * These cases could break constraints causing issues like invalided characters (empty, a utf-16 surrogate pair alone, etc) or + * annotations which reference out of bounds character ranges. + * Addressing these issues mainly falls into these categories: + * - Handling of invalid content on import/construction of unhydrated nodes and/or insertion into the document (hydration). + * - Handling of invalid content which is already part of the document (live). This should ideally include both detection and repair. + * - Constraints on edits to prevent invalid content from being created by merges. + * - Optimization of the constraints to reduce cases in which edits are rejected due to conflicts. + * + * Note that these cases of invariant violations are the same cases any component should handle, so ideally there would be a general framework or pattern for documenting and enforcing such constraints. + * + * Another area for future work is improved APIs for import, export and non-schema-aware use. This includes a variety of cases, including but not limited to: + * - Insertable content format (taken by the constructor and import APIs). + * - Customizable export formats (like a way to make exportVerbose and exportConcise flatten the text nodes to strings automatically). + * - Customizable toJson behavior (e.g. flattening to strings, possibly via tools or patterns for custom tree stringifier). + * - Ensure JS object APIs (like iteration, own vs inherited properties, etc) provide a good and consistent experience with respect to other nodes. + * - Support in generateSchemaFromSimpleSchema for recognizing text nodes and generating the appropriate schema. + * - Ensure above features work well enough to support important scenarios like AI assisted editing and document indexing for search. + * + * Part of that work will be establishing and documenting those patterns so other components with complex encodings can follow them, + * in addition to implementing them for text. + * @alpha + */ +export namespace TextAsTree { + /** + * A retain op in a character-level delta — a span of unchanged characters that the consumer should skip over. + * @sealed + * @alpha + */ + export interface TextRetainOp { + /** + * Discriminator identifying this op as a retain. + */ + readonly type: "retain"; + /** + * The number of Unicode code points to retain. + */ + readonly count: number; + /** + * Whether at least one character in the retained range had a deep change. + * @remarks + * Present only on retain ops delivered by {@link @fluidframework/tree#FormattedTextAsTree.Members.onContentChanged}; + * always absent on retain ops delivered by {@link TextAsTree.Members.onCharactersChanged}. + * When present, `true` indicates the retained range contained a formatting property update + * or an atom content edit; `false` indicates no deep change. + */ + readonly formattingChanged?: boolean; + } + + /** + * A single operation in a character-level delta describing an insert, remove, or retain of text. + * @alpha + */ + export type TextOp = TextRetainOp | TextInsertOp | TextRemoveOp; + + /** + * Statics for text nodes. + * @alpha + */ + export interface Statics { + /** + * Construct a {@link TextAsTree.(Tree:type)} from a string, where each character (as defined by iterating over the string) becomes a single character in the text node. + * This combines pairs of utf-16 surrogate code units into single characters as appropriate. + */ + fromString(value: string): Tree; + } + + /** + * Interface for a text node. + * @remarks + * The string is broken up into substrings which are referred to as 'characters'. + * Unlike with JavaScript strings, all indexes are by character, not UTF-16 code unit. + * This avoids the problem JavaScript where it can split UTF-16 surrogate pairs producing invalid strings, + * and avoids the issue where indexing a string and iterating it segment the string differently. + * This does NOT mean the characters correspond to user perceived characters (like grapheme clusters try to do): + * applications will likely want to include higher level segmentation logic + * which might differ between operations like delete + * (which often operates on something in between unicode code points and grapheme clusters) + * and navigation/selection (which typically uses grapheme clusters). + * + * @see {@link TextAsTree.Statics.fromString} for construction. + * @see {@link TextAsTree.(Tree:type)} for schema. + * @sealed + * @alpha + */ + export interface Members { + /** + * Gets an iterable over the characters currently in the text. + * @remarks + * This iterator matches the behavior of {@link (TreeArrayNode:interface)} with respect to edits during iteration. + */ + characters(): Iterable; + + /** + * Optimized way to get a copy of the {@link TextAsTree.Members.characters} in an array. + */ + charactersCopy(): string[]; + + /** + * Gets the number of characters currently in the text. + * @remarks + * The length of {@link TextAsTree.Members.characters}. + * This is not the length of the string returned by {@link TextAsTree.Members.fullString}, + * as that string may contain characters which are made up of multiple UTF-16 code units. + */ + characterCount(): number; + + /** + * Copy the content of this node into a string. + */ + fullString(): string; + + /** + * Insert a range of characters into the string based on character index. + * @remarks + * See {@link (TreeArrayNode:interface).insertAt} for more details on the behavior. + * See {@link TextAsTree.Statics.fromString} for how the `additionalCharacters` string is broken into characters. + * @privateRemarks + * If we provide ways to customize character boundaries, that could be handled here by taking in an Iterable instead of a string. + * Doing this currently would enable insertion of text with different character boundaries than the existing text, + * which would violate the currently documented character boundary invariants. + * + * Another option would be to take an approach like Table, + * where the user of the API uses a factory function to generate the schema, and can inject custom logic, like a string character iterator. + */ + insertAt(index: number, additionalCharacters: string): void; + + /** + * Remove a range from a string based on character index. + * See {@link (TreeArrayNode:interface).removeRange} for more details on the behavior. + */ + removeRange(startIndex: number | undefined, endIndex: number | undefined): void; + + /** + * Subscribe to shallow character-level changes on this text node — inserts and removes only. + * @param callback - Called after each change with a sequence of {@link TextAsTree.TextOp}s describing what changed, + * or `undefined` when a delta could not be computed (e.g. during a schema upgrade). + * @returns A cleanup function that unsubscribes the callback when called. + * @remarks + * Only fires on shallow changes — inserts and removes. + * It does not fire on deep changes such as formatting property updates on existing characters. + * For formatted text, use {@link @fluidframework/tree#FormattedTextAsTree.Members.onContentChanged} to also receive deep changes. + * + * All counts in the delivered ops are in Unicode code points, not UTF-16 code units. + * For characters outside the Basic Multilingual Plane (e.g. emoji), one code point + * corresponds to two UTF-16 code units — convert before using the counts as string indices. + */ + onCharactersChanged(callback: (ops: readonly TextOp[] | undefined) => void): () => void; + } + + /** + * Schema for a {@link TextAsTree.(Tree:variable)} node. + * @remarks + * See {@link TextAsTree.Statics} for static APIs on this schema, including construction. + * @alpha + */ + export const Tree = eraseSchemaDetails()(TextNode); + + /** + * Node for the {@link TextAsTree.(Tree:type)} schema exposing the {@link TextAsTree.Members} API. + * @remarks + * Create using {@link TextAsTree.Statics.fromString}. + * @alpha + */ + export type Tree = Members & TreeNode & WithType<"com.fluidframework.text.Text">; +}