Skip to content

test(resilience): characterize Brainarr circuit breaker behavior#371

Merged
RicherTunes merged 1 commit intomainfrom
test/breaker-characterization
Jan 16, 2026
Merged

test(resilience): characterize Brainarr circuit breaker behavior#371
RicherTunes merged 1 commit intomainfrom
test/breaker-characterization

Conversation

@RicherTunes
Copy link
Owner

Summary

Add 26 characterization tests documenting current circuit breaker semantics for WS4.2 migration decision-making.

Test Coverage by Category

Keying Scheme (2 tests)

  • Documents "ai:{provider}:{modelId}" format from BreakerRegistry

Failure Classification (4 tests)

  • TaskCanceledException → failure ⚠️ CRITICAL: may trip during shutdown
  • TimeoutException → failure
  • HttpRequestException → failure
  • 4xx client errors → excluded via string matching ⚠️ IMPORTANT: brittle detection

Dual-Trip Logic (2 tests)

  • Consecutive failures threshold: ≥5 opens circuit
  • Failure rate threshold: ≥50% with ≥10 minimum throughput

Half-Open Transitions (3 tests)

  • 3 successes to close
  • Any failure immediately re-opens
  • Concurrency cap during half-open

Minimum Throughput Guard (1 test)

  • Rate-based opening disabled below 10 operations

Windowing (2 tests)

  • CircularBuffer sliding window behavior
  • Statistics API

Configuration Constants (3 tests)

  • Default, Aggressive, Lenient presets

Key Findings for WS4.2

  1. CRITICAL: TaskCanceledException treated as failure - can trip breaker during host shutdown, request aborts, or client timeouts
  2. IMPORTANT: 4xx detection via ex.Message.Contains("4") && ex.Message.Contains("Bad Request") is brittle and locale-dependent
  3. IMPORTANT: Dual-trip logic (consecutive OR rate) + minimum throughput + half-open concurrency cap must map to Common breaker

WS4.2 Migration Path

These tests become the contract that WS4.2 must satisfy:

  1. Adapter layer in Brainarr (no behavior change) + run characterization tests against it
  2. If Common lacks semantics, extend Common then delete Brainarr implementation

Test Plan

  • All 26 tests pass locally
  • No sleeps/polling - uses immediate assertions
  • Brittle behaviors documented in test comments

🤖 Generated with Claude Code

@RicherTunes RicherTunes force-pushed the test/breaker-characterization branch from e70f250 to c3bbd26 Compare January 16, 2026 20:29
Add 25 characterization tests documenting current circuit breaker semantics
for WS4.2 migration decision-making:

- Keying scheme: "ai:{provider}:{modelId}" format
- Failure classification: TaskCanceledException/TimeoutException/HttpRequestException
  as failures; 4xx client errors excluded via brittle string matching
- Dual-trip logic: consecutive failures (≥5) OR failure rate (≥50% with ≥10 ops)
- Half-open transitions: 3 successes to close, any failure re-opens
- Minimum throughput guard: rate-based opening requires ≥10 operations
- Windowing: CircularBuffer sliding window behavior
- Configuration constants: documents BrainarrConstants defaults

Key findings for WS4.2:
- CRITICAL: TaskCanceledException treated as failure (may trip during shutdown)
- IMPORTANT: 4xx detection via string matching ("4" + "Bad Request") is brittle
- Dual-trip + minimum throughput must map to Common

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@RicherTunes RicherTunes force-pushed the test/breaker-characterization branch from c3bbd26 to 99d226b Compare January 16, 2026 20:30
@RicherTunes RicherTunes merged commit 02f5420 into main Jan 16, 2026
20 checks passed
@RicherTunes RicherTunes deleted the test/breaker-characterization branch January 16, 2026 23:20
RicherTunes added a commit that referenced this pull request Feb 17, 2026
Picks up:
- #383: fix(ci): Windows credential issue in change detection step
- #367: deps: coverlet.collector 6.0.4 → 8.0.0
- #370: deps: publicapigenerator.tool 11.4.6 → 11.5.4
- #371: deps: Spectre.Console 0.50.0 → 0.54.0
- #375: deps: xunit 2.9.2 → 2.9.3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RicherTunes added a commit that referenced this pull request Feb 17, 2026
Picks up:
- #383: fix(ci): Windows credential issue in change detection step
- #367: deps: coverlet.collector 6.0.4 → 8.0.0
- #370: deps: publicapigenerator.tool 11.4.6 → 11.5.4
- #371: deps: Spectre.Console 0.50.0 → 0.54.0
- #375: deps: xunit 2.9.2 → 2.9.3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RicherTunes added a commit that referenced this pull request Feb 17, 2026
Picks up:
- #383: fix(ci): Windows credential issue in change detection step
- #367: deps: coverlet.collector 6.0.4 → 8.0.0
- #370: deps: publicapigenerator.tool 11.4.6 → 11.5.4
- #371: deps: Spectre.Console 0.50.0 → 0.54.0
- #375: deps: xunit 2.9.2 → 2.9.3

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant