Restore 55 Pulsar tests and skip only the four behaviours that are missing (GH-3763) - #3798
Merged
Merged
Conversation
…ssing (GH-3763) Commit 02f5318 on 2026-06-08 -- message "Disable Pulsar compliance tests", no other rationale recorded -- tagged all three Pulsar compliance classes Category=Flaky. That is 66 of the project's 224 tests; CIPulsar has been running 158. Untagged, the exclusion turns out to be far too broad: 213 pass and 11 fail, and the 11 are the SAME FOUR behaviours repeated across the three fixtures. will_requeue_and_increment_attempts Expected ending activity was not detected can_schedule_retry Expected ending activity was not detected will_move_to_dead_letter_queue_without_any_exception_match No ending activity detected will_move_to_dead_letter_queue_with_exception_match No ending activity detected They are not flaky. They fail deterministically, every run, alone or in a suite, and they look like genuinely unimplemented transport behaviour: native requeue with attempt tracking, scheduled retry, and dead-letter routing. Filed as #3797. So all three classes come off the ledger and only those four are skipped, each carrying the reason and the issue number. can_schedule_retry had to become virtual in TransportCompliance to be overridable; the other three already were. with_cloud_events already opted out of will_move_to_dead_letter_queue_with_ exception_match by overriding it with a bare `return Task.CompletedTask`, which reports as a PASS -- the suite was counting a test that ran nothing. That is now a real Skip carrying its original explanation. Net: CIPulsar goes from 158 executing tests to 213, plus 11 named skips. Two things worth knowing that came out of this: - CIPulsar runs Pulsar in Testcontainers, not docker-compose (#3467). A local run hangs silently at "224 batched" if Docker is short on memory -- nine stale containers were holding 4.7 of 7.7 GiB here and no Pulsar container could start. Stopping them fixed it; nothing in the output said so. - The supervisor's summary counts SKIPPED tests inside its "passed" number. Run 2 reports "224 passed" for 213 executed + 11 skipped. The per-assembly runner reports them correctly (18 succeeded / 4 skipped for one fixture). Verified: CIPulsar green twice -- 223 passed + 1 retry, then 224 passed with 0 retries. wolverine.slnx builds clean in Release. Repo flaky tags 16 -> 13. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0116vfBcKwcjWn8msM4ZjkuA
This was referenced Aug 3, 2026
Member
Author
|
Follow-ups filed out of this work, so the PR itself stays scoped to the tag burn-down:
|
This was referenced Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Pulsar pass of #3763, after #3791 (AWS), #3794 (Kafka) and #3795 (RabbitMQ).
An exclusion with no recorded reason, and far too broad
Commit
02f531842on 2026-06-08 — message "Disable Pulsar compliance tests", no other rationale — tagged all three Pulsar compliance classesCategory=Flaky. That is 66 of the project's 224 tests;CIPulsarhas been running 158.Untagged: 213 pass, 11 fail — and the 11 are the same four behaviours repeated across the three fixtures.
will_requeue_and_increment_attemptsExpected ending activity was not detectedcan_schedule_retryExpected ending activity was not detectedwill_move_to_dead_letter_queue_without_any_exception_matchNo ending activity detectedwill_move_to_dead_letter_queue_with_exception_matchNo ending activity detectedThey are not flaky — they fail deterministically, every run, alone or in a suite, and they look like genuinely unimplemented transport behaviour: native requeue with attempt tracking, scheduled retry, and dead-letter routing. Filed as #3797.
So: all three classes come off the ledger, only those four are skipped
Each skip carries the reason and the issue number, so what's missing is greppable instead of buried under three blanket exclusions.
can_schedule_retryhad to becomevirtualinTransportComplianceto be overridable; the other three already were.with_cloud_eventsalready opted out ofwill_move_to_dead_letter_queue_with_exception_matchby overriding it with a barereturn Task.CompletedTask— which reports as a PASS, so the suite was counting a test that ran nothing. That's now a realSkipcarrying its original explanation.Net:
CIPulsargoes from 158 executing tests to 213, plus 11 named skips.Two things worth knowing that came out of this
CIPulsarruns Pulsar in Testcontainers, not docker-compose (Global partitioning: dedicated end-to-end tests for Azure Service Bus, GCP Pub/Sub, NATS, Redis, and Pulsar #3467). A local run hangs silently at224 batchedif Docker is short on memory — nine stale containers were holding 4.7 of 7.7 GiB here and no Pulsar container could start. Stopping them fixed it; nothing in the output said so.18 succeeded / 4 skippedfor one fixture). Worth a look given CI retry counts are invisible: a job burning 22 of 25 retries is indistinguishable from a clean green #3787's honest-reporting goal — not changed here.Verification
Repo flaky tags 16 → 13.
wolverine.slnxbuilds clean in Release.🤖 Generated with Claude Code
https://claude.ai/code/session_0116vfBcKwcjWn8msM4ZjkuA