Skip to content

tests: OperationCoordinator fixtures hold the persist gate across blocking waits #82

Closed as not planned

Description

@AresConnor

Found during the flake project (PR #80) and deliberately deferred: several OperationCoordinatorTests fixtures block inside coordinator-owned synchronous seams while production code holds _persistGate — e.g. persistence Action<byte[]> delegates and IServiceScopeFactory.CreateScope() fakes waiting on release events. Two consequences, both measured:

  1. It made an otherwise-correct production change hazardous: adding a context-free yield to OperationCoordinator.RunOperationAsync (so the runner resumes on the ThreadPool) produced a real hang under 2-core stress — 1-in-30 with the change, 0-in-30 without, live stack showing the runner parked in Monitor.Enter on _persistGate while the gate holder waited for a pool thread that the runner occupied. The change was reverted (revert(backend): keep the operation runner on the scheduling caller's context), but the underlying hazard — a lock held across a blocking wait in test fixtures — remains, and constrains future scheduling changes in the coordinator.
  2. OperationCoordinatorTests.Start_ReturnsPersistedHandleBeforeBlockedExecutor flakes under load on unmodified master (observed in local 2-core stress), same neighborhood.

Acceptance criteria:

  • Restructure the blocking fixtures so no test-side wait happens while _persistGate (or any coordinator lock) is held on the waiting thread's dependency chain — the established patterns are the shared OffPool/DedicatedWorker helpers and LongRunning starts (see PR fix(backend,tests): unpin production workers from the xunit scheduler and stop starving it #80).
  • After the restructuring, re-attempt the reverted RunOperationAsync context-free yield and keep it if 30×2-core stress stays clean both ways (the flake it originally fixed is real).
  • Start_ReturnsPersistedHandleBeforeBlockedExecutor stable across a 30-run pinned stress.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions