Skip to content

fix(mtp): send run request test selection as tests not testCases - #3755

Open
Evangelink wants to merge 2 commits into
stryker-mutator:masterfrom
Evangelink:dev/amauryleve/cautious-memory
Open

fix(mtp): send run request test selection as tests not testCases#3755
Evangelink wants to merge 2 commits into
stryker-mutator:masterfrom
Evangelink:dev/amauryleve/cautious-memory

Conversation

@Evangelink

Copy link
Copy Markdown

Fixes #3754

Problem

The MTP runner serialized the testing/runTests test selection under the JSON name testCases, but Microsoft.Testing.Platform reads it as tests:

  • Protocol spec: interface RunTestsParams { tests?: TestNode[], runId: GUID }
  • Wire constant: JsonRpcMethods.cs -> public const string Tests = "tests";
  • Deserializer: GetOptionalPropertyFromJson(properties, JsonRpcStrings.Tests)

tests is optional server-side, so an unrecognised testCases is dropped without any error or warning and the server falls through to running the entire test suite.

The likely origin is the testfx samples/Playground/ServerMode sample this client was ported from, where the C# member is TestCases but the wire name is tests:

public sealed record RunRequest(
    [property:JsonProperty("tests")]
    TestNode[]? TestCases,

Present since the MTP runner was introduced in #3404.

Second, related defect

TestNode's optional location.* members were written as explicit nulls. The platform probes them with TryGetValue - which succeeds on an explicit null - and then asserts non-null:

if (properties.TryGetValue("location.file", out object? location_file))
{
    ApplicationStateGuard.Ensure(location_file is not null);   // throws InvalidOperationException

This was latent while TestNodes only flowed server -> client, and would have become live the moment the rename made selection actually work: every test the framework reported without location info would fail the run request. Both parts have to land together, so they are in one commit.

The same applies to the selection itself, which is now omitted rather than sent as "tests": null when running every test.

Changes

  • RunTestsRequest.TestCases serializes as tests, omitted when null.
  • TestNode location properties omitted when null.
  • New RpcWireFormatTests asserting the raw JSON payload.

Why the existing tests did not catch this

Two independent reasons, both worth noting because they explain the blind spot:

  1. SingleMicrosoftTestPlatformRunner.RunAssemblyTestsAsync currently always passes testUidFilter: null, so no selection is ever sent in practice and the wrong name was never exercised against a real server.
  2. TestingPlatformClientTests.FakeTestServer deserializes using Stryker's own RunTestsRequest record, so the name matches on both ends and the test passes for any name.

The new tests therefore assert the raw JSON rather than round-tripping through Stryker's records, so they are anchored to the platform's contract instead of to our own types.

Verification

  • Full Stryker.TestRunner.MicrosoftTestPlatform.UnitTest suite: 213/213 pass (3 consecutive runs).
  • Reverting only the source fix while keeping the new tests: 8 tests fail, covering both defects. Confirms they are real regression guards rather than tests written to match current behaviour.
  • Full Stryker.slnx builds with 0 errors.

Scope

Deliberately narrow. Actually using a selection (passing a real testUidFilter) only pays off once MTP coverage is per-test rather than cumulative, so that belongs in a follow-up. This change is a prerequisite for it and is correct on its own: it makes the payload match the protocol.

Note: I could not apply the 🐛 Bug label to the linked issue (requires admin rights) - a maintainer may want to add it.

The MTP runner serialized the `testing/runTests` test selection under
`testCases`, but Microsoft.Testing.Platform reads it as `tests`
(`JsonRpcStrings.Tests`). The property is optional server-side, so the
misnamed selection was dropped silently and the server ran the entire
suite instead of the selection.

The likely origin is the testfx ServerMode sample this client was ported
from, where the C# member is named `TestCases` but the wire name is
`tests`.

Also omit absent optional properties instead of writing explicit nulls.
The platform probes `location.*` with TryGetValue and then asserts the
value is not null, so a `"location.file": null` on a node echoed back
reads as present but invalid and fails the request. This was latent while
nodes only flowed server to client, and would have become live as soon as
the rename made selection work.

Add wire-format tests that assert the raw JSON. The existing round-trip
test deserializes with Stryker's own record, so it passes for any property
name and could not detect the mismatch.

Fixes stryker-mutator#3754
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 3, 2026 12:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the Microsoft Testing Platform (MTP) runner’s testing/runTests request payload to match the server-mode JSON-RPC contract, ensuring test selection is honored (instead of being silently ignored and running the entire suite) and preventing failures caused by sending explicit null location fields.

Changes:

  • Serialize the run test selection under JSON property tests (not testCases) and omit it when null to represent “run all tests”.
  • Omit TestNode optional location.* properties when they are null to avoid server-side assertions triggered by explicit JSON nulls.
  • Add wire-format unit tests that validate the raw JSON payload shape against the platform’s expectations.
Show a summary per file
File Description
src/Stryker.TestRunner.MicrosoftTestPlatform/Models/TestNode.cs Omits nullable location.* fields during serialization to avoid invalid explicit-null properties on the wire.
src/Stryker.TestRunner.MicrosoftTestPlatform/Models/RunTestsRequest.cs Renames the serialized selection property to tests and omits it when null, matching MTP protocol behavior.
src/Stryker.TestRunner.MicrosoftTestPlatform.UnitTest/RpcWireFormatTests.cs Adds raw-JSON assertions to guard the wire contract (selection name + null-vs-absent behavior).

Review details

  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/Stryker.TestRunner.MicrosoftTestPlatform.UnitTest/RpcWireFormatTests.cs Outdated
`Serialize<T>` returned `JsonDocument.Parse(...).RootElement` from an
undisposed document. `JsonDocument` owns the pooled memory backing its
elements, so the buffer was never returned to the pool and the element
lifetime was tied to an object nothing held.

Dispose the document and return a cloned element, which is detached from
the document and remains valid after it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MTP test runner sends run request test selection as testCases instead of tests, so selection is silently ignored

2 participants