fix: report baseline compile failures instead of an internal error - #3715
Open
anagnorisis2peripeteia wants to merge 1 commit into
Open
Conversation
anagnorisis2peripeteia
force-pushed
the
fix/report-baseline-compile-failure
branch
10 times, most recently
from
July 16, 2026 17:07
1208a5b to
bed7629
Compare
When rollback cannot attribute a compile error to a mutant and the erroring file contains no mutants, the failure is not a mutation problem: the project does not compile independently of Stryker (e.g. a source generator produced no output). Report that as a baseline failure naming the file and diagnostic codes, rather than the misleading "internal error for Stryker.NET". Before concluding a mutant-free failure is baseline, prove it: rebuild the user's original source only - drop generated trees and Stryker's injected helper trees, reverse every mutation AND the structural rewrites Stryker applies to host them (e.g. expression-body to block-body), then re-run the source generators on that clean source and recompile. If it builds, the failure was mutation-induced (rollback just could not attribute it) and stays an internal error; only a failure that persists on the fully-uninstrumented baseline is reported as baseline. This baseline capability is an internal interface (no new public API); content that cannot provide it keeps the conservative internal-error path. Stryker's own injected helper files (MutantControl etc.) are also mutant-free, but a compile failure there is an internal Stryker problem, so those keep the internal-error path. Helper detection is an internal, immutable check. Refs stryker-mutator#3698
anagnorisis2peripeteia
force-pushed
the
fix/report-baseline-compile-failure
branch
from
July 16, 2026 18:01
bed7629 to
26a6f46
Compare
This was referenced Jul 16, 2026
Open
anagnorisis2peripeteia
added a commit
to anagnorisis2peripeteia/marmorkrebs
that referenced
this pull request
Jul 17, 2026
Recognise the concrete, reliable equivalent-mutant classes from the Stryker.NET PR (stryker-mutator/stryker-net#3715) so they stop surfacing as false survivors / reviewer noise: - logging-only: a string/removal mutation on a bare, result-discarded ILogger/ILog-style call (message not asserted -> no observable change) - attribute context: a mutation on an attribute-only line (no runtime effect) - manual: an in-source `// marmorkrebs-ok[: reason]` reviewer directive Classification is report/source-driven (reads the mutant's own source span from the mutation-testing-elements `source`), not a Roslyn AST pass — that belongs in the Stryker.NET fork. This keeps it self-contained, lane-extensible and unit-testable. New `--classify-equivalent <off|annotate|suppress>` (default annotate): - annotate: flag survivors via SurvivingMutant.likelyEquivalent, score unchanged - suppress: drop heuristic matches from the threshold denominator - a `// marmorkrebs-ok` directive is authoritative and suppresses in any mode but off Fail-closed: heuristic guesses never silently inflate a score (annotate is the default; only explicit suppress or a human directive removes a survivor), and an all-equivalent-suppressed run still trips the vacuous-run guard rather than passing 100%. Distinct from the existing stryker-cxx `--equivalent-suppression` binary passthrough.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a source generator (e.g.
Microsoft.Windows.CsWin32) produces no output during Stryker's mutation compilation, the unmutated file that consumes the generated symbols fails to compile. Rollback can't attribute those errors to any mutant, so Stryker aborts — but the message it prints is"Stryker.NET could not compile the project after mutation. This is probably an error for Stryker.NET and not your project.", which points people at the wrong thing. That's the reporting gap noted in #3698.The cause: in
CSharpRollbackProcess.RollbackMutationsInError, once rollback can't recover a compile error it always throws"Internal error due to compile error."and logs the "probably a Stryker.NET error" critical — even when the erroring file contains no mutants at all, i.e. the project simply doesn't build independently of mutation.The change: when an erroring syntax tree carries no mutants (
MutantPlacer.GetAllMutations(...)is empty), report the failure naming the file and diagnostic codes, and point at the likely cause (a generator that emitted nothing), without asserting the user's project is broken — keeping a "this may be a Stryker.NET issue worth reporting" fallback.Proving it's actually mutation-independent: before concluding a mutant-free failure is a baseline problem, Stryker now rebuilds a genuinely uninstrumented baseline and recompiles it (
ICompilationContent.CompilesWithoutMutations()). "Uninstrumented" means reversing not just the mutations but also the structural rewrites Stryker applies to host them (e.g. converting an expression-bodied member to a block body), then re-running the source generators on that clean source so a syntax-sensitive generator sees the original code. If that baseline builds, the failure was mutation-induced (rollback just couldn't attribute it) and stays an internal error; only a failure that persists with no instrumentation at all is reported as a baseline failure. With today's mutators this always confirms "baseline" here — Stryker only places runtime-wrapped mutations and never mutates the compile-time constructs that could break a different file (const/enum/attribute/parameter-default) — but the recompile keeps the classification sound if that ever changes.Injected helpers: Stryker's own injected helper files (
MutantControletc.) are also mutant-free, but a compile failure there really is an internal Stryker problem, so those keep the internal-error message (an internal, immutable check, not public API). Trees that still carry mutants keep the existing internal-error path unchanged.Real-behaviour proof (the CsWin32 source-generator repro from #3698). A minimal project that uses
Microsoft.Windows.CsWin32to generate a P/Invoke consumed byWin32Anchor.cs. It builds fine withdotnet build, but under Stryker's mutation compilation CsWin32 emits nothing, soWin32Anchor.cs(which has no mutants) fails to compile. PackingStryker.CLIfrommasterand from this branch, then running each against the repro test project with the identical command:Base (
433fde6a) — misdiagnoses it as Stryker's fault, preceded by contradictory warnings:This branch — reverses all instrumentation, re-runs the generators on the clean source, sees Win32Anchor.cs still fails (mutation-independent), and reports it plainly with no contradictory warnings:
Both runs were captured to files; the exact before/after transcripts (and their sha256) are recorded in the branch review-history gist below, alongside the local CI, mutation, and generator-baseline test results.
Testing: unit tests in
CSharpRollbackProcessTestscover the branch — an unmutated file that fails to compile gets the baseline message (with joined diagnostic codes, and a generic label when it has noFilePath); a mutant-carrying tree still gets the internal-error message; an injected-helper file gets the internal-error message; a mutant-free tree co-erroring with a mutant-carrying tree is deferred rather than misreported; andCompilesWithoutMutationsis covered both ways (true when removing mutations fixes the build, false when the failure is mutation-independent) plus the branch where a baseline that compiles keeps the internal error. Verified end-to-end against the CsWin32 repro (the before/after above).Refs #3698. AI-assisted, but the before/after is reproducible so you don't have to take my word for it.
Review history: https://gist.github.com/387d71b3928dfead781013182e7286ec