Skip to content

chore(test): fix test flakiness due to tchannel span assertion - #1077

Closed
shijiesheng wants to merge 2 commits into
cadence-workflow:v3.13.xfrom
shijiesheng:flaky-test-v3
Closed

chore(test): fix test flakiness due to tchannel span assertion#1077
shijiesheng wants to merge 2 commits into
cadence-workflow:v3.13.xfrom
shijiesheng:flaky-test-v3

Conversation

@shijiesheng

@shijiesheng shijiesheng commented Aug 13, 2026

Copy link
Copy Markdown
Member

What changed?

  • Disabled two tests that will 100% fail the docker test because of the thrift header is now invalid to new grpc-go server. It still works today because internally we use a forked grpc library.
  • Add awaitSpan helper method
  • fixed StartWorkflowTest and WorkflowClientInternalTest

Why?

TChannel can activate span but has race conditions on the callback. This would cause span assertions assuming it's synchronos.

How did you test it?

Unit Test

Potential risks
no, it's pure test changes

Release notes

Documentation Changes

Signed-off-by: Shijie Sheng <liouvetren@gmail.com>
…gher grpc

Signed-off-by: Shijie Sheng <liouvetren@gmail.com>
@gitar-bot

gitar-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown
CI failed: 4 test failures occurred during the unit test suite execution under both sticky-on and sticky-off Docker service configurations.

Overview

Unit test execution failed in both the unit-test-docker-sticky-off and unit-test-docker-sticky-on jobs, with 4 tests failing out of 2182 completed tests in each run.

Failures

Unit Test Failures (confidence: high)

  • Type: test
  • Affected jobs: 94609211715, 94898069212
  • Related to change: yes
  • Root cause: Test failures occurred during the Gradle test task execution (Execution failed for task ':test'), specifically 4 failing tests out of 2182 completed.
  • Suggested fix: Inspect the test report at build/reports/tests/test/index.html or run ./gradlew test --info locally to identify the specific failing tests and adjust assertions or code accordingly.

Summary

  • Change-related failures: 2 job failures due to failing unit tests (sticky-on and sticky-off configurations)
  • Infrastructure/flaky failures: 0 infrastructure failures
  • Recommended action: Review the local test execution report to pinpoint the 4 failing tests and fix the underlying assertion or code issues related to the PR changes.
Code Review 👍 Approved with suggestions 0 resolved / 1 findings

Adds awaitSpans helper methods to resolve tchannel span assertion flakiness in StartWorkflowTest and WorkflowClientInternalTest. Consider removing the unused awaitFinishedSpansCount helper.

💡 Quality: Unused helper awaitFinishedSpansCount added but never called

📄 src/test/java/com/uber/cadence/internal/sync/WorkflowClientInternalTest.java:547-558

The newly added private static helper awaitFinishedSpansCount(int, Duration) is never invoked anywhere in the file — the tracing tests instead call fakeService.awaitSpan(...). This is dead code that will likely trigger unused-method warnings and adds noise. Either wire it into the span-count assertions it was meant to fix, or remove it.

🤖 Prompt for agents
Code Review: Adds awaitSpans helper methods to resolve tchannel span assertion flakiness in StartWorkflowTest and WorkflowClientInternalTest. Consider removing the unused awaitFinishedSpansCount helper.

1. 💡 Quality: Unused helper awaitFinishedSpansCount added but never called
   Files: src/test/java/com/uber/cadence/internal/sync/WorkflowClientInternalTest.java:547-558

   The newly added private static helper `awaitFinishedSpansCount(int, Duration)` is never invoked anywhere in the file — the tracing tests instead call `fakeService.awaitSpan(...)`. This is dead code that will likely trigger unused-method warnings and adds noise. Either wire it into the span-count assertions it was meant to fix, or remove it.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@shijiesheng

Copy link
Copy Markdown
Member Author

merged into this #1079

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant