Skip to content

tsp: prototype filesystem-backed sessions for WASI - #5599

Draft
yeonwooson wants to merge 1 commit into
meta:mainfrom
yeonwooson:wasi/filesystem-tsp-session
Draft

yeonwooson wants to merge 1 commit into
meta:mainfrom
yeonwooson:wasi/filesystem-tsp-session

Conversation

@yeonwooson

Copy link
Copy Markdown
Contributor

Summary

Experimental architecture draft. Not for landing. This is a user-directed, AI-assisted prototype for feedback on the filesystem-backed WASI session boundary, not a finished Pylance backend. The human-understanding checklist remains unchecked pending the contributor's code review.

Contributor-provided intent

The contributor requested this draft, verbatim:

ok can you post a draft PR, so we casn ask if that's what he's looking for

Project context supplied by the contributor:

For problem 1, we're hoping we can make a WASI call to Pyrefly in-proc and that will be faster than an out of proc RPC call.

AI-assisted proposal

Following the host-written filesystem discussion in #5565, keep unsaved contents in the host filesystem rather than adding source overrides to Pyrefly State. This draft does not claim that in-process calls are faster; it establishes a small query/snapshot boundary to review before adding transport and packaging.

  • Add a serialized FilesystemSession with configuration, checked-file selection, filesystem invalidation, getSnapshot and one getComputedType endpoint.
  • Use ordinary filesystem module identities. The host writes editor contents at their resolved paths, reports created/modified/removed events, and retains saved contents for discard/save handling.
  • Reuse existing GetTypeParams, protocol Type output, snapshotChanged shape and ServerCancelled/InvalidParams errors. Queries convert types against the same warm transaction and reject stale snapshots before and after answering.
  • Share the existing transaction-aware converter/resolver with native code and WASIp1 without bringing native queues/transports into the Wasm build.
  • Add six native session regressions and document the host contract.

Relationship to existing work: the minimal shared-conversion extraction is included so the prototype can compile and be reviewed independently. It overlaps #5436 and must be reconciled or split into that prerequisite before landing. This is not a second shared-conversion PR. Neither that PR nor the source-overlay PR is changed or closed by this draft. The PR5565 filesystem implementation and the local Wasm validation adapter are not included here.

Architectural feedback requested

  1. Is a small host-owned-filesystem session the right boundary for the WASI backend?
  2. Is using committed generation as the session snapshot, with one authoritative filesystem event path, appropriate before adding a transport?
  3. Should shared conversion remain the separate prerequisite in PR5436, with the session built on top?

Not implemented or claimed

  • Production WASI ABI, transport, worker lifecycle or distribution packaging.
  • Pylance integration, request scheduling/retry behavior, cancellation or concurrent callers.
  • Remaining TSP methods, dynamic configuration or notebook-cell projection.
  • Actual save-to-disk integration, complete bundled-stub declaration navigation or comparative performance.

The caller must serialize filesystem writes, events and queries. Empty change batches are no-ops; nonempty duplicate notifications may still advance generation, so the host should avoid redundant events. Search paths are import roots, not filesystem-access boundaries.

No issue closure is requested.

  • I am a human, and I fully understand the code I am submitting.

Test Plan

AI-assisted validation report: Windows ARM64 using public Rust 1.99.1, WASI SDK 34 and Node 24.15.0. The source is based on cf67d00701e5b4c12756e8dac15033f566aad2b6; these local results do not claim validation of subsequent upstream changes or merge readiness.

  • PASS, 209 tests; 0 failed or ignored, rerun for this draft: cargo test -p pyrefly --lib --locked --offline -- test::tsp:: test::state:: tsp::type_conversion::tests:: tsp::validation::tests:: --test-threads=2
  • PASS, all native targets typechecked/linted: cargo clippy -p pyrefly --all-targets --locked --offline --message-format=json
  • PASS, required formatting/Clippy rerun with no changes to the tested source: python test.py --mode cargo --no-test --no-tensor-shapes --no-conformance --no-jsonschema --no-extension
  • PASS: cargo check -p pyrefly --lib --target wasm32-wasip1 --locked --offline
  • PASS, existing sandbox compilation/linking from pyrefly_wasm: cargo build -p pyrefly_wasm --lib --target wasm32-wasip1 --locked --offline

The six session tests cover unsaved imported edits (int -> str -> restored int), old-snapshot rejection, stable repeated queries, new/deleted imports, function declaration ranges/call results, invalid URI/path/range inputs, UTF-16 boundaries and explicit configuration errors. Existing native TSP/converter/State tests remain in the selected run; the moved converter/resolver algorithms are unchanged.

A separate local validation adapter was built with cargo build --manifest-path <adapter>/Cargo.toml --target wasm32-wasip1 --locked --offline and executed with node <harness>/run-node.mjs <artifact.wasm> <result.json>: 29 checks passed, including a fresh draft rerun. The same Wasm artifact previously passed 29 checks in a real Chromium/Electron browser. The checks exercise actual session queries, protocol-shaped outputs and stale errors through Wasm, not merely a generation-counter mock. They also cover invalid requests and disposed adapter handles.

The adapter, host filesystem from PR5565, and runtime harness are out-of-tree experimental validation, not a shipped interface or part of this diff. Native/WASI behavior differs for materialized bundled-stub paths; no full navigation parity or Pylance-to-WASI test is claimed.

Existing unrelated warnings remain; no added-line diagnostics were introduced. The full native suite, local Linux/macOS execution, a cross-browser matrix, production packaging and performance comparisons were not run. Remote CI results are separate from these local checks. The draft must remain not merge-ready until human code review and the architectural questions are resolved.

Let a host-owned filesystem present editor contents through ordinary
module identities, then reuse the existing type converter to answer
computed-type requests against a committed snapshot.

Expose a small serialized session for architectural feedback. Keep native
transports separate and verify real type-query and stale-snapshot behavior
without introducing engine source overlays or a production WASI ABI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@meta-codesync

meta-codesync Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D130755466. (Because this pull request was imported automatically, there will not be any future comments.)

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant