Lower the test-results floor to ERRORS on a passing run too

The #1251 fix corrected a failing run's already-fine ERRORS floor but only
raised a passing run's floor from NORMAL to COMPLAIN -- still one step short.
A build's own test results are core information a user silencing routine
build chatter still wants to see, whether the run passed or failed; ERRORS
is the true floor for both, not a pass/fail distinction.

PluginReportinator#test_results_floor_verbosity centralizes the answer so
the three plugins that print post-build test results in this shape here
(report_tests_stdout_plugin, gcov, valgrind) share one source of truth
instead of each carrying its own copy of the same conditional. Bullseye is
untouched -- its own post_build test-results call is a different, disabled
shape on this branch, unrelated to this fix.

Also: fixed a pre-existing bug this work's own system-test run surfaced --
ceedling_build_exec/ceedling_appcmd_exec sanitized captured subprocess output
for @raw_output but not before compose_failure_report's own scan of it,
crashing on non-ASCII compiler diagnostic text under a non-UTF-8 default
external encoding. Sanitize once, upfront, and reuse the same cleaned
strings for both.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: mkarlesky

docs/Changelog.md

@@ -14,7 +14,7 @@ This changelog is complemented by three other documents:
 
 ## 💪 Fixed
 
-- [#1251](https://github.com/ThrowTheSwitch/Ceedling/issues/1251) Fixed _Overall Test Summary_ not printing at logging verbosity of warning on an otherwise all-passing test run.
+- [#1251](https://github.com/ThrowTheSwitch/Ceedling/issues/1251) Fixed _Overall Test Summary_ not printing at logging verbosity of warning on an otherwise all-passing test run. A follow-up closes the remaining gap: an all-passing run's test summary now also prints at the stricter `--verbosity=errors`, matching a failing run, rather than requiring `warnings` or higher.
 - [#1252](https://github.com/ThrowTheSwitch/Ceedling/issues/1252) Fixed the Gcov plugin requiring `gcovr` to be installed for any build merely because the plugin was enabled, rather than only when a `gcov:` build actually runs.
 - Fixed a Unity `TEST_IGNORE_MESSAGE()` test case being misreported as crash evidence during crash-diagnosis retries when it shares a test file with a genuine crash.
 

lib/ceedling/plugins/plugin_reportinator.rb

@@ -95,6 +95,15 @@ def assemble_test_results(results_list, options={:boom => false})
     return aggregated_results
   end
 
+  # A build's own test results -- pass or fail -- are core information a user
+  # silencing routine build chatter still wants to see; ERRORS is the floor
+  # for both, not just for a failing run. Centralized here so every plugin
+  # printing post-build test results (report_tests_stdout_plugin, gcov,
+  # valgrind) agrees on the same answer instead of each hardcoding its own.
+  def test_results_floor_verbosity
+    Verbosity::ERRORS
+  end
+
   def run_test_results_report(hash, verbosity=Verbosity::NORMAL)
     if @test_results_template.nil?
       raise CeedlingException.new( "No test results report template has been set." )

lib/ceedling/plugins/report_tests_stdout_plugin.rb

@@ -50,10 +50,7 @@ def post_build(_timestamp_s)
 
     results = @plugin_reportinator.assemble_test_results( @result_list )
     hash = { :context => TEST_SYM, :results => results }
-    # #1251 -- `:warnings` on the CLI maps to Verbosity::COMPLAIN (see VERBOSITY_OPTIONS
-    # in constants.rb), not NORMAL, so gating the all-clean case at NORMAL silently
-    # swallowed the Overall Test Summary at --verbosity=warnings even on a passing run.
-    verbosity = (results[:counts][:failed] > 0) ? Verbosity::ERRORS : Verbosity::COMPLAIN
+    verbosity = @plugin_reportinator.test_results_floor_verbosity
 
     # Print unit test suite results
     @plugin_reportinator.run_test_results_report( hash, verbosity )

plugins/gcov/lib/gcov.rb

@@ -222,10 +222,7 @@ def post_build(_timestamp_s)
         results: results
       }
 
-      # #1251 -- `:warnings` on the CLI maps to Verbosity::COMPLAIN (see VERBOSITY_OPTIONS
-      # in constants.rb), not NORMAL, so gating the all-clean case at NORMAL silently
-      # swallowed the Overall Test Summary at --verbosity=warnings even on a passing run.
-      verbosity = (results[:counts][:failed] > 0) ? Verbosity::ERRORS : Verbosity::COMPLAIN
+      verbosity = @plugin_reportinator.test_results_floor_verbosity
 
       # Print unit test suite results
       @plugin_reportinator.run_test_results_report( hash, verbosity )

plugins/valgrind/lib/valgrind.rb

@@ -101,10 +101,7 @@ def post_build(_timestamp_s)
         results: results
       }
 
-      # #1251 -- `:warnings` on the CLI maps to Verbosity::COMPLAIN (see VERBOSITY_OPTIONS
-      # in constants.rb), not NORMAL, so gating the all-clean case at NORMAL silently
-      # swallowed the Overall Test Summary at --verbosity=warnings even on a passing run.
-      verbosity = (results[:counts][:failed] > 0) ? Verbosity::ERRORS : Verbosity::COMPLAIN
+      verbosity = @plugin_reportinator.test_results_floor_verbosity
 
       # Print unit test suite results
       @plugin_reportinator.run_test_results_report( hash, verbosity )

spec/support/system/system_context.rb

@@ -235,8 +235,11 @@ def ceedling_build_exec(*args)
     # encoding regardless of their actual content, so any project source content the
     # subprocess echoes (verbose logging, compiler output, etc.) can make later regex
     # matching against this string raise. Sanitize once here, the same way Ceedling itself
-    # sanitizes file content it scans.
-    @raw_output       = (stdout + stderr).clean_encoding
+    # sanitizes file content it scans -- reused for both the raw output and the failure
+    # report below, since a real compiler's own diagnostic text (e.g. clang's curly-quoted
+    # identifiers) is exactly the kind of non-ASCII content this needs to survive too.
+    stdout, stderr    = stdout.clean_encoding, stderr.clean_encoding
+    @raw_output       = stdout + stderr
     @console_summary  = compose_failure_report(stdout, stderr)
 
     SystemTestOutput.new(@raw_output)
@@ -250,7 +253,8 @@ def ceedling_appcmd_exec(*args)
 
     @last_cmd         = cmd
     @last_exit_status = status.exitstatus
-    @raw_output       = (stdout + stderr).clean_encoding
+    stdout, stderr    = stdout.clean_encoding, stderr.clean_encoding
+    @raw_output       = stdout + stderr
     @console_summary  = compose_failure_report(stdout, stderr)
 
     SystemTestOutput.new(@raw_output)

spec/system/deployment_as_gem_spec.rb

@@ -88,6 +88,7 @@
     describe "Verbosity and output" do
       test_case :test_project_with_named_verbosity
       test_case :test_project_with_warnings_verbosity_prints_overall_summary
+      test_case :test_project_with_errors_verbosity_prints_overall_summary_on_passing_run
       test_case :test_project_with_numerical_verbosity
       test_case :report_tests_raw_output_log_plugin
     end

spec/system/deployment_as_vendor_spec.rb

@@ -91,6 +91,7 @@
     describe "Verbosity and output" do
       test_case :test_project_with_named_verbosity
       test_case :test_project_with_warnings_verbosity_prints_overall_summary
+      test_case :test_project_with_errors_verbosity_prints_overall_summary_on_passing_run
       test_case :test_project_with_numerical_verbosity
       test_case :report_tests_raw_output_log_plugin
     end

spec/system/support/common_test_cases.rb

@@ -201,11 +201,9 @@ def test_project_with_named_verbosity
     end
   end
 
-  # #1251 -- the Overall Test Summary must still print at :warnings verbosity
+  # The Overall Test Summary must still print at :warnings verbosity
   # (`--verbosity=warnings`, Verbosity::COMPLAIN) on an all-passing run, not just
-  # when verbosity is :normal or higher, or when a test fails. Before this fix, the
-  # summary's own gate mistakenly required :normal, silently swallowing the summary
-  # on the (default) all-clean case at :warnings.
+  # when verbosity is :normal or higher, or when a test fails.
   def test_project_with_warnings_verbosity_prints_overall_summary
     @c.with_context do
       Dir.chdir @proj_name do
@@ -224,6 +222,27 @@ def test_project_with_warnings_verbosity_prints_overall_summary
     end
   end
 
+  # ERRORS is the true floor for the Overall Test Summary -- a build's own
+  # test results are core information, visible even at the most restrictive
+  # verbosity short of :silent, whether the run passed or failed.
+  def test_project_with_errors_verbosity_prints_overall_summary_on_passing_run
+    @c.with_context do
+      Dir.chdir @proj_name do
+        FileUtils.cp test_asset_path("example_file.h"), 'src/'
+        FileUtils.cp test_asset_path("example_file.c"), 'src/'
+        FileUtils.cp test_asset_path("test_example_file_success.c"), 'test/'
+
+        output = @c.ceedling_build_exec("--verbosity=errors")
+        expect(@c.last_exit_status).to eq(0)
+        expect(output).to match(/OVERALL TEST SUMMARY/)
+        expect(output).to match(/TESTED:\s+\d/)
+        expect(output).to match(/PASSED:\s+\d/)
+        expect(output).to match(/FAILED:\s+\d/)
+        expect(output).to match(/IGNORED:\s+\d/)
+      end
+    end
+  end
+
   def test_project_with_numerical_verbosity
     @c.with_context do
       Dir.chdir @proj_name do

spec/units/plugin_reportinator_spec.rb

@@ -0,0 +1,38 @@
+# =========================================================================
+#   Ceedling - Test-Centered Build System for C
+#   ThrowTheSwitch.org
+#   Copyright (c) 2010-26 Mike Karlesky, Mark VanderVoord, & Greg Williams
+#   SPDX-License-Identifier: MIT
+# =========================================================================
+
+require 'spec_helper'
+require 'ceedling/constants'
+require 'ceedling/plugins/plugin_reportinator'
+
+describe PluginReportinator do
+  before(:each) do
+    @plugin_reportinator_helper = double('plugin_reportinator_helper')
+    @plugin_manager             = double('plugin_manager')
+    @reportinator                = double('reportinator')
+    @loginator                      = double('loginator')
+
+    @reportinator_plugin = described_class.new(
+      {
+        :plugin_reportinator_helper => @plugin_reportinator_helper,
+        :plugin_manager             => @plugin_manager,
+        :reportinator               => @reportinator,
+        :loginator                  => @loginator
+      }
+    )
+  end
+
+  # A build's own test results -- pass or fail -- are core information a user
+  # silencing routine build chatter still wants to see, so every plugin that
+  # prints post-build test results (report_tests_stdout_plugin, gcov, valgrind)
+  # shares this one answer instead of each deciding independently.
+  describe '#test_results_floor_verbosity' do
+    it 'is ERRORS' do
+      expect( @reportinator_plugin.test_results_floor_verbosity ).to eq( Verbosity::ERRORS )
+    end
+  end
+end