Skip to content

Windows CI hang: in-box Windows PowerShell attach E2E test wedges after the 20260614 runner image #2323

Description

Summary

The Windows leg of CI (CI Tests workflow) hangs on the E2E debugger-attach
test CanAttachScriptWithPathMappings and rides GitHub Actions' default
6-hour job timeout. The root cause is not a PSES code change — it is a
windows-latest runner-image refresh that broke in-box Windows PowerShell
5.1
's cross-process attach. This issue tracks the real fix; PR #2318 is a
stopgap (skip the test on in-box WinPS + a 30-minute timeout-minutes
backstop).

Root cause: runner-image refresh 20260608 → 20260614

Comparing the last-green and first-red main runs:

Same image family, same runner agent (2.335.1), same -Preview PowerShell.
The only thing that moved at the boundary is the weekly OS-servicing patch in
the image. That refresh broke in-box Windows PowerShell 5.1's cross-process
Debug-Runspace / Enter-PSHostProcess attach — exactly the path this test
exercises. The precise servicing delta (likely a .NET Framework or Windows
named-pipe/IPC update) is still unidentified and is the main open question
here.

Symptoms

  • The hang is specifically the in-box Windows PowerShell E2E suite
    (TestE2EPowerShell). PowerShell Core (TestE2EPwsh) and the preview pass
    the same attach test; macOS and Linux pass in ~5 minutes.
  • The test wedges on the attach handshake and the run sits on that single
    test until the job is killed.
  • Its xUnit [SkippableFact(Timeout = 15000)] never fires: the wedge blocks
    in a way xUnit's cooperative timeout cannot abort, so it is reported as a
    "Long Running Test" climbing past 20+ minutes.

What we ruled out

Mitigation (PR #2318)

  • Skip CanAttachScriptWithPathMappings on in-box Windows PowerShell
    (IsWindowsPowerShell, covering the WinPS and WinPS-CLM suites) so the
    windows-latest leg completes; Core / preview / macOS / Linux keep full
    coverage of the attach path.
  • timeout-minutes: 30 on the matrix test job as a permanent backstop.

Next steps toward a real fix

  • Identify the precise 20260614 servicing delta (.NET Framework / Windows
    IPC) that changed in-box WinPS 5.1 attach behavior.
  • Capture a hung Windows run's process/runspace state or DAP transcript to see
    where the attach handshake stalls under the new image.
  • Investigate the harness's readiness logic (RunWithAttachableProcess keys
    off RunspaceBase.AvailabilityChanged via reflection) and the
    Debug-Runspace handshake for an OS-version-sensitive race.
  • Once fixed, remove the Skip.If from Skip attach E2E test on in-box Windows PowerShell (20260614 image regression); cap CI job #2318; the timeout-minutes backstop
    can stay.

References

Activity

  1. changed the title [-]Intermittent Windows CI hang: debugger-attach E2E test wedges and rides the job timeout[/-] [+]Windows CI hang: in-box Windows PowerShell attach E2E test wedges after the 20260614 runner image[/+] on Jun 19, 2026
  2. andyleejordan commented on Jun 19, 2026

    @andyleejordan
    MemberAuthor

    Update: the stopgap is green on main's CI.

    The expanded skip landed in e6ca2dfa6, and the CI Tests run on that commit passed — the dotnet (windows-latest) leg finished in ~9.5 min (21:50→22:00) instead of riding the timeout to cancellation.

    A few corrections/expansions now that we understand the failure better:

    • It's not attach-specific — it's WinPS-hosted server startup. Skipping only CanAttachScriptWithPathMappings just relocated the wedge to the next test that starts the in-box Windows PowerShell server. The hang is in spawning Start-EditorServices.ps1 under Windows PowerShell 5.1, which both E2E suites do. So Skip attach E2E test on in-box Windows PowerShell (20260614 image regression); cap CI job #2318 now discovery-time skips the entire WinPS-hosted E2E surface:
      • the whole debug-adapter class (DebugAdapterProtocolMessageTests, 13 tests) — c667d3906
      • the whole language-server class (LanguageServerProtocolMessageTests) — e6ca2dfa6
    • xUnit gotcha worth recording: a discovery-time Skip stops per-test IAsyncLifetime setup (the DAP class), but xUnit still creates an IClassFixture<> even when every method in the class is skipped. The LSP server start lives in LSPTestsFixture.InitializeAsync, so the discovery skip alone didn't help there — LSPTestsFixture now also guards against starting the server under Windows PowerShell.
    • Strongest evidence yet that it's purely the image: I re-ran an old main commit that predates Match strong-name identity when resolving PSES dependencies #2303 and all of our recent PRs and previously passed — 6ad4f46657 ("Correct example PowerShell -Uri argument name (Correct example PowerShell -Uri argument name #2304)"). On today's image it now hangs the same way (run 27572333402), while macOS/Linux stay green. That fully exonerates our code, including Match strong-name identity when resolving PSES dependencies #2303.
    • Windows PowerShell unit coverage (TestPS51) is unaffected and still runs; only the WinPS-hosted E2E server tests are skipped.

    The job timeout-minutes is currently 60 (raised from 30 as a backstop while the suite was still wedging). The "Next steps toward a real fix" above still stand — identify the 20260614 servicing delta and, once fixed, remove the discovery-time skips.

    Drafted by Copilot (Claude Opus 4.8) on Andy's behalf.

  3. andyleejordan commented on Jun 22, 2026

    @andyleejordan
    MemberAuthor

    Okay so no idea how the GitHub Actions image update managed to break the PowerShell 5.1 end-to-end tests but I'm guessing something to do with permissions. Thing is, that is what happened, since if you go back and re-run old passing CI runs they'll start to fail the same way. Hence disabled for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions