Add artifact merge engines and Kind metadata flow (RFC 018 phase 1)#9805
Merged
Conversation
Implements the first testfx-side pieces of RFC 018 (artifact post-processing for dotnet test / MTP): - Report merge engines (pure, invocation-agnostic, no I/O in the core): - TrxReportEngine.Merge / MergeToFileAsync (XML-level TRX merge). - JUnitReportMerger (unions <testsuite>, counters derived per-suite). - CtrfReportMerger (JSON merge; summary derived from merged tests[]). - Kind metadata flow so the dotnet/sdk orchestrator can group artifacts by format: SessionFileArtifact gains an optional kind (experimental), the FileArtifactMessage IPC record + serializer carry Kind (field id 7), the dotnet-test consumer propagates it, and report producers tag their kind (trx/junit/ctrf/html). Unit tests cover all three merge engines and the IPC round-trip. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds report-merging foundations and propagates artifact format metadata through MTP’s artifact and IPC layers.
Changes:
- Adds TRX, JUnit, and CTRF merge engines with unit tests.
- Adds experimental
SessionFileArtifact.Kindmetadata and IPC serialization. - Tags TRX, JUnit, CTRF, and HTML report artifacts by format.
Show a summary per file
| File | Description |
|---|---|
test/UnitTests/Microsoft.Testing.Platform.UnitTests/IPC/ProtocolTests.cs |
Covers IPC kind serialization. |
test/UnitTests/Microsoft.Testing.Platform.UnitTests/IPC/ProtocolEdgeCaseTests.cs |
Tests nullable kind round trips. |
test/UnitTests/Microsoft.Testing.Platform.DotnetTestProtocolContract.UnitTests/DotnetTestProtocolSerializerTests.cs |
Verifies protocol-contract serialization. |
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/TrxReportEngineMergeTests.cs |
Tests TRX merging. |
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/JUnitReportMergerTests.cs |
Tests JUnit merging. |
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/CtrfReportMergerTests.cs |
Tests CTRF merging. |
src/Platform/SharedExtensionHelpers/ReportGeneratorBase.cs |
Publishes report artifact kinds. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Serializers/FileArtifactMessagesSerializer.cs |
Serializes the new kind field. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/ObjectFieldIds.cs |
Assigns IPC field ID 7. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Models/FileArtifactMessages.cs |
Adds kind to the wire model. |
src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/DotnetTestDataConsumer.cs |
Propagates session artifact kinds. |
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt |
Tracks the experimental public API. |
src/Platform/Microsoft.Testing.Platform/Messages/FileArtifacts.cs |
Adds the kind-aware constructor and property. |
src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt |
Tracks IPC internal API changes. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxReportEngine.Merge.cs |
Implements TRX merging. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxDataConsumer.cs |
Tags in-process TRX artifacts. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks TRX internal APIs. |
src/Platform/Microsoft.Testing.Extensions.Retry/InternalAPI/InternalAPI.Unshipped.txt |
Tracks linked IPC field API. |
src/Platform/Microsoft.Testing.Extensions.MSBuild/InternalAPI.Unshipped.txt |
Tracks linked IPC field API. |
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportMerger.cs |
Implements JUnit merging. |
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs |
Tags JUnit artifacts. |
src/Platform/Microsoft.Testing.Extensions.JUnitReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks JUnit internal APIs. |
src/Platform/Microsoft.Testing.Extensions.HtmlReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks shared generator API. |
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportGenerator.cs |
Tags HTML artifacts. |
src/Platform/Microsoft.Testing.Extensions.HangDump/InternalAPI/InternalAPI.Unshipped.txt |
Tracks linked IPC field API. |
src/Platform/Microsoft.Testing.Extensions.CtrfReport/InternalAPI/InternalAPI.Unshipped.txt |
Tracks CTRF internal APIs. |
src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportMerger.cs |
Implements CTRF merging. |
src/Platform/Microsoft.Testing.Extensions.CtrfReport/CtrfReportGenerator.cs |
Tags CTRF artifacts. |
Review details
- Files reviewed: 28/28 changed files
- Comments generated: 4
- Review effort level: Medium
…ure-broccoli # Conflicts: # src/Platform/Microsoft.Testing.Extensions.CtrfReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.HangDump/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.HtmlReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.JUnitReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt
- Carry microsoft.testing.trx through the out-of-process TRX path by adding an (experimental) Kind to FileArtifact and propagating it in DotnetTestDataConsumer and TrxProcessLifetimeHandler; centralize the kind in TrxReportEngine.TrxArtifactKind. - Deduplicate <UnitTest> definitions by id when merging TRX (ids are deterministic from the test UID and the schema forbids duplicates). - Preserve <RunInfos> (crash/exit diagnostics) and <CollectorDataEntries> from every input's <ResultSummary> instead of discarding them; omit them when empty. - Correct the inaccurate 'attachment paths are absolute' remark and relocate each input's attachment deployment tree into the merged deployment root in MergeToFileAsync (best-effort, never fails the merge). - Add unit tests for dedup and diagnostics preservation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
0101
approved these changes
Jul 13, 2026
…ure-broccoli # Conflicts: # src/Platform/Microsoft.Testing.Extensions.CtrfReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.HangDump/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.HtmlReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.JUnitReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.MSBuild/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.Retry/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt # src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt
0101
approved these changes
Jul 13, 2026
- Skip relocation when the merged 'In' root overlaps the source 'In' root (output written beside the input with a matching runName), preventing the copy from recursing into its own destination; the original hrefs already resolve there. - Skip file reparse points (symlinks) before File.Copy so a link inside the confined tree can't pull in content from outside it. - Overwrite copied attachment files so a reused output directory can't leave stale bytes behind while the merged XML references the fresh per-input destination. - Add a unit test for the overlapping-root skip case. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
0101
approved these changes
Jul 13, 2026
This comment has been minimized.
This comment has been minimized.
Regenerated via UpdateXlf. The last localized check-in added translations for 'GlobalTestFixtureShouldBeValidClassLayout' (fr) and 'SlowTestStillRunning' (ru) that XliffTasks rejects (placeholder mismatch), breaking the build with "xlf is out-of-date with resx". UpdateXlf resets those two units to state="new" so the build passes; they will be re-translated on the next localization pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4350f662-0e5d-4e05-b156-1ef619da3b9e
This comment has been minimized.
This comment has been minimized.
0101
approved these changes
Jul 15, 2026
…ias check (review) - TRX Merge now carries run-level <ResultSummary>/<Output> across (StdOut/StdErr/ DebugTrace concatenated, TextMessages/Message unioned), emitted in schema order right after <Counters>, so VSTest's run-level skipped/informational messages are no longer silently dropped. - MergeToFileAsync rejects an empty input list before any filesystem work (output directory creation / attachment relocation), so an invalid call no longer mutates the disk before Merge throws. Applied to TRX/JUnit/CTRF. - Output-alias check is now filesystem-aware: it compares canonicalized paths with a cached case-sensitivity probe (case-sensitive on case-sensitive volumes, insensitive otherwise), so on Linux a legitimate separate output like 'A.trx' is no longer rejected as aliasing input 'a.trx', while Windows/macOS case aliases are still caught. Applied to TRX/JUnit/CTRF. Tests: run-level Output merge + schema order; empty-input rejection without touching the filesystem (all three mergers). 74 merge tests pass; full solution build clean. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
0101
approved these changes
Jul 16, 2026
This comment has been minimized.
This comment has been minimized.
…view) The alias-equality check derived its case sensitivity from a one-time probe in Path.GetTempPath(), but case sensitivity can differ per volume and (on Windows) per directory. Replace the cached temp probe with a per-call probe of the OUTPUT's own location (nearest existing ancestor) in all three mergers. An alias can only occur when input and output share a directory, so that shared location's sensitivity is the correct one — a different-directory input never compares equal. Falls back to case-insensitive (rejects more, never less) if the probe fails. This closes the gap where 'A.trx' could alias input 'a.trx' on a case-insensitive output volume while the temp-based probe said Ordinal, letting ReplaceFile delete the input. Applied to TRX/JUnit/CTRF. 74 merge tests pass; full solution build clean. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
0101
approved these changes
Jul 16, 2026
…er (review) The read-only-input alias check (canonicalization, per-output-directory case-sensitivity probe, netstandard reparse-ancestor fail-closed) and the atomic temporary-sibling write/ replace were duplicated in the TRX, JUnit and CTRF mergers. Because this is security- and data-loss-sensitive, extract it into a single shared SharedExtensionHelpers/ MergeOutputFileHelper.cs (linked into all three, namespace Microsoft.Testing.Extensions), exposing EnsureOutputDoesNotAliasInput and WriteViaTemporarySiblingAsync so the logic can no longer diverge across formats. Each merger now delegates to it; the per-format writer (XDocument stream vs string) is passed as a callback. Registered the shared internal type/ methods in each project's InternalAPI.Unshipped.txt. No behavior change. 74 merge tests pass; full solution build clean (0 warnings). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
0101
approved these changes
Jul 16, 2026
…ments CI (macOS): GetCanonicalPath now re-resolves ancestor symlinks after every link hop, so a link target whose prefix is itself a symlink (macOS /var -> /private/var) canonicalizes consistently and the read-only-input alias check no longer misses a symlinked-parent output. Fixes the 3 WhenOutputAliasesInputViaSymlinkedParent merge tests that failed on macOS. Review comments addressed: - MergeOutputFileHelper.ReplaceFile: atomic File.Move(overwrite:true) on netcore so an interruption can never leave the previous report missing; delete-then-move retained only on .NET Framework (no overwrite overload). - TRX relocation: deployment root is now unique per run (run id suffix), so relocation always writes a fresh tree and can never corrupt a previously committed report's attachments if a later merge fails. - TRX Times: earliest creation/queuing/start tracked independently and omitted when no input supplies them, instead of fabricating all three from the earliest start. - CTRF reportId: derived only from accepted CTRF payloads, so an ignored non-CTRF input no longer changes the merged report identity. Adds regression tests for each and updates deployment-root-coupled TRX tests to read the emitted runDeploymentRoot instead of assuming a literal name. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 34f1a49b-b1e0-4f38-b8a1-3870dd0842d2
0101
approved these changes
Jul 16, 2026
Contributor
🧪 Test quality grade — PR #9805
This advisory comment was generated automatically. Grades are heuristic Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the first testfx-side pieces of RFC 018 — Artifact post-processing for
dotnet test(MTP). Two independent, self-contained foundations that every later phase (dispatcher tool, SDK election) reuses. No SDK dependency; the SDK orchestration lands separately once this merges.1. Report merge engines (pure, invocation-agnostic — no I/O in the core)
Validated the RFC's "collect same-kind inputs → re-aggregate → emit one" shape across both data models:
TrxReportEngine.Merge/MergeToFileAsyncResults/TestDefinitions/TestEntriesTimesearliest/latest,TestListsdeduped by id, outcomeFailedif any input failedJUnitReportMerger<testsuite><testsuite>rootsCtrfReportMergerJsonNode)results.tests[]summaryderived from the mergedtests[]sosummary.testsalways matches the array;start/stopmin/maxHTML is intentionally not file-merged (it's a rendered template with an embedded JSON blob) — this validates the design's opt-in / return-null semantics rather than breaking it.
2.
Kindmetadata flow (producer → IPC wire)So the
dotnet testorchestrator can group artifacts by format without inspecting file contents:SessionFileArtifactgains an optionalkind(experimental-gated,[TPEXP]); the shipped 4-arg constructor is preserved for binary compatibility.FileArtifactMessage.Kind+FileArtifactMessageFieldsId.Kind = 7+ serializer read/write/field-count (forward/backward compatible — old readers skip the unknown field).DotnetTestDataConsumerpropagatesKindonto the wire message.microsoft.testing.trx, JUnitmicrosoft.testing.junit, CTRFmicrosoft.testing.ctrf, HTMLmicrosoft.testing.html.Tests
Kindsurvives the pipe.WasiPlaygroundsample fails on a missing workload); Platform IPC/protocol, contract-package, and Extensions unit tests all green.Not in this PR (future phases, per the RFC)
IArtifactPostProcessortyped contract wrapping the engines.ITool+ tool-host-over-pipe composition.merge-trx, …) and the dotnet/sdk orchestration.cc @Evangelink
Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com