Skip to content

Point the AWS tests at LocalStack, and fix the SQS name limit they were hiding (GH-3763) - #3791

Merged
jeremydmiller merged 1 commit into
mainfrom
gh-3763/aws-flaky-tag-burndown
Aug 3, 2026
Merged

Point the AWS tests at LocalStack, and fix the SQS name limit they were hiding (GH-3763)#3791
jeremydmiller merged 1 commit into
mainfrom
gh-3763/aws-flaky-tag-burndown

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Continues the flaky-tag burn-down in #3763. Azure Service Bus is deliberately out of scope here — this is the AWS half.

The tags were wrong

Eleven AWS test files were tagged [Trait("Category", "Flaky")] in a bulk sweep on 2026-03-20/21 — the same sweep #3789 showed had mis-diagnosed the ASB set. Ten of them were never flaky. They call UseAmazonSqsTransport(), which resolves the ambient AWS credential chain, so on CI (no credentials) they fail deterministically:

Amazon.Runtime.AmazonClientException: No RegionEndpoint or ServiceURL configured
Error trying to start message broker sqs on Attempt 1..20 of 20

…and on a developer machine that is logged in, they quietly provision queues in a real AWS account. The tag wasn't hiding instability, it was hiding a wiring mistake. All of them now use UseAmazonSqsTransportLocally() like the rest of the suite.

Two real defects fell out once they ran

1. SanitizeSqsName ignored SQS's name rules. It only replaced . with -. SQS accepts "alphanumeric characters, hyphens, or underscores. 1 to 80 in length", and

shazaam-Wolverine-AmazonSqs-Tests-ConventionalRouting-SqsHandlerTypeNamingMessage

is 81 characters. Broker initialization provisions every queue together, so one overlong or illegally-named queue — Handle(Item[]), generics, nested types — takes down startup for every conventionally-routed host in the assembly:

BrokerInitializationException: Unable to initialize the Broker sqs in time
 ---- WolverineSqsTransportException: Error while trying to initialize Amazon SQS queue 'shazaam-…-SqsHandlerTypeNamingMessage'
 -------- AmazonSQSException: Can only include alphanumeric characters, hyphens, or underscores. 1 to 80 in length

This is the same defect #3789 fixed on Azure Service Bus, and the exception chaining added there is exactly what made this readable in minutes rather than hours. Illegal characters are now substituted rather than stripped, so Item[] stays separable from Item; an overlong name is truncated with a SHA-256 digest appended — deterministic across processes and machines, which rules out GetHashCode(). No name that works today changes: a name SQS rejects could never have been provisioned in the first place. 13 unit tests cover it, including the no-op cases.

2. The fixtures leaked their hosts. IHost.Dispose() tears down the container without ever running IHostedService.StopAsync, so the SQS listeners kept polling — 12 listeners started on the shared sqs://routed queue and only 4 stopped. The eight zombies then stole messages from whichever class ran next. ConventionalRoutingContext now owns disposal, and derived classes override InitializeAsync instead of re-declaring IAsyncLifetime — re-declaring it is what let a no-op DisposeAsync shadow the real one, here and in the ASB fixtures (#3758).

end_to_end_with_conventional_routing also gets its own message type. The shard runs this project across worker processes partitioned by class, so CollectionPerAssembly only serializes within one process; a sibling class holding a sqs://routed listener in another process received the message and the tracked session timed out waiting for a delivery that had already happened elsewhere.

Two that stay excluded — labelled honestly, not called flaky

  • Samples/Bootstrapping.customize_mappers was the file's only [Fact], inside a doc region that must keep showing the real UseAmazonSqsTransport() a reader would write. Now private, compile-checked like every other sample in that file.
  • SNS send_to_topic_and_receive_in_queue_in_aws is a line-for-line duplicate of send_to_topic_and_receive_in_queue except that it points at a real AWS account. Now [Fact(Skip = …)] with the reason, so it reports as skipped rather than being silently filtered.

Verification

Run with no AWS credentials at all (HOME pointed at an empty directory) against a freshly recreated LocalStack:

Shard Before After Retries
CIAWSSqs 137 passed 172 passed 0
CIAWSSqsCompliance 93 passed 0
CIAWSSns 119 passed 0

Repo flaky tags 32 → 21. wolverine.slnx builds clean in Release.

🤖 Generated with Claude Code

https://claude.ai/code/session_0116vfBcKwcjWn8msM4ZjkuA

…re hiding (GH-3763)

Eleven AWS test files were tagged [Trait("Category", "Flaky")] in a bulk sweep on
2026-03-20/21. Ten of them were never flaky: they call UseAmazonSqsTransport(),
which resolves the ambient AWS credential chain. CI has no credentials, so they
fail deterministically:

  Amazon.Runtime.AmazonClientException: No RegionEndpoint or ServiceURL configured
  Error trying to start message broker sqs on Attempt 1..20 of 20

and on a developer machine that IS logged in, they quietly provision queues in a
real AWS account. The tag was not hiding instability, it was hiding a wiring
mistake. All of them now use UseAmazonSqsTransportLocally(), like the rest of the
suite.

Two real defects fell out once they ran:

- AmazonSqsTransport.SanitizeSqsName only replaced '.' with '-'. SQS accepts
  "alphanumeric characters, hyphens, or underscores. 1 to 80 in length", and
  "shazaam-Wolverine-AmazonSqs-Tests-ConventionalRouting-SqsHandlerTypeNamingMessage"
  is 81. Broker initialization provisions every queue together, so one overlong
  or illegally-named queue -- Handle(Item[]), generics, nested types -- takes down
  startup for every conventionally-routed host in the assembly. Same defect
  GH-3786 just fixed on Azure Service Bus, and the exception chaining added there
  is what made this readable in minutes instead of hours. Illegal characters are
  now substituted rather than stripped, so Item[] stays separable from Item, and
  an overlong name is truncated with a SHA-256 digest appended -- deterministic
  across processes, which rules out GetHashCode(). No name that works today
  changes: a name SQS rejects could never have been provisioned.

- The fixtures leaked their hosts. IHost.Dispose() tears down the container
  without running IHostedService.StopAsync, so the SQS listeners kept polling: 12
  listeners started on the shared sqs://routed queue and 4 stopped. Eight zombies
  then stole messages from whichever class ran next. ConventionalRoutingContext
  now owns disposal, and derived classes override InitializeAsync instead of
  re-declaring IAsyncLifetime -- re-declaring it is what let a no-op
  DisposeAsync shadow the real one, here and in the ASB fixtures (GH-3758).

end_to_end_with_conventional_routing also gets its own message type. The shard
runs this project across worker PROCESSES partitioned by class, so
CollectionPerAssembly only serializes within one process; a sibling class holding
a sqs://routed listener in another process received the message and the tracked
session timed out waiting for a delivery that had already happened elsewhere.

Two that stay excluded, honestly labelled rather than called flaky:

- Samples/Bootstrapping.customize_mappers was the file's only [Fact], inside a
  doc region that must keep showing the real UseAmazonSqsTransport() a reader
  would write. Now private, compile-checked like every other sample there.
- SNS send_to_topic_and_receive_in_queue_in_aws is a line-for-line duplicate of
  send_to_topic_and_receive_in_queue except that it points at a real AWS account.
  Now [Fact(Skip = ...)] with the reason, so it reports as skipped instead of
  being silently filtered.

Verified with no AWS credentials at all (HOME pointed at an empty directory) on a
freshly recreated LocalStack: CIAWSSqs 137 -> 172 passed, CIAWSSqsCompliance 93,
CIAWSSns 119, all with 0 retries. Repo flaky tags 32 -> 21. wolverine.slnx builds
clean in Release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0116vfBcKwcjWn8msM4ZjkuA
@jeremydmiller
jeremydmiller merged commit f32bb42 into main Aug 3, 2026
34 checks passed
This was referenced Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant