Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
b2656b5
feat(orchestrator): implement registration orchestrator node [C1]
jonahgabriel Dec 22, 2025
6f5c590
chore(validation): increase union threshold for orchestrator models [C1]
jonahgabriel Dec 22, 2025
3b50f8f
merge: resolve main conflicts, update union threshold to 555
jonahgabriel Dec 23, 2025
08c5568
fix(orchestrator): address PR #79 review - time injection and model c…
jonahgabriel Dec 23, 2025
87b2f41
fix(orchestrator): address PR #79 review - complete fixes [C1]
jonahgabriel Dec 23, 2025
f2c0b00
merge: resolve main branch conflicts (union threshold docs)
jonahgabriel Dec 23, 2025
4885416
fix(pr-79): address remaining review comments - type safety and docs
jonahgabriel Dec 23, 2025
4d9ab79
fix(orchestrator): add warning log for unexpected None ack_deadline
jonahgabriel Dec 23, 2025
6563e3c
docs(event-bus): add Phase 9 documentation and tests [OMN-57]
jonahgabriel Dec 23, 2025
e1fbe4c
fix(tests): add missing timestamp field to ModelNodeIntrospectionEven…
jonahgabriel Dec 23, 2025
a686025
fix(tests): add missing timestamp to ModelNodeIntrospectionEvent fixt…
jonahgabriel Dec 24, 2025
aacf1cb
merge: resolve main branch conflicts (union threshold)
jonahgabriel Dec 24, 2025
ce74af5
refactor(registration): code quality enhancements and CI fix
jonahgabriel Dec 24, 2025
42f840b
fix(tests): add missing timestamp to ModelNodeIntrospectionEvent fixt…
jonahgabriel Dec 24, 2025
1900067
fix(tests): add missing timestamp to ModelNodeIntrospectionEvent fixt…
jonahgabriel Dec 24, 2025
e2b4c1f
merge: resolve main branch conflicts (registration __init__)
jonahgabriel Dec 24, 2025
1a715eb
fix(pr-review): address PR #79 review feedback
jonahgabriel Dec 25, 2025
40ded30
fix(ci): fix lint and ONEX validator failures for PR #79
jonahgabriel Dec 25, 2025
4fdd9d1
fix(pr-review): address final PR #79 feedback - docs and time injection
jonahgabriel Dec 25, 2025
6cd8e41
fix(pr-review): address remaining PR #79 feedback - thread safety and…
jonahgabriel Dec 25, 2025
e61df2c
merge: resolve conflicts with main branch
jonahgabriel Dec 25, 2025
03c5d8b
fix(pr-review): address all remaining PR #79 feedback [OMN-952]
jonahgabriel Dec 25, 2025
dcab5b0
merge: resolve conflicts with main branch (accept protocol_event_bus_…
jonahgabriel Dec 25, 2025
1c0c867
fix(mixins): restore ProtocolEventBusLike protocol [OMN-952]
jonahgabriel Dec 25, 2025
79bbd52
fix(pr-review): address final PR #79 feedback - CI fix and docs [OMN-…
jonahgabriel Dec 25, 2025
cba7dae
merge: resolve main branch conflicts (import cleanup, union threshold)
jonahgabriel Dec 25, 2025
97915c3
fix(pr-review): address PR #79 review feedback and CI failure [OMN-952]
jonahgabriel Dec 25, 2025
3340e0d
fix(pr-review): address remaining PR #79 feedback - release ready [OM…
jonahgabriel Dec 25, 2025
b9aa0de
merge(main): resolve conflicts and fix PR #79 review issues [OMN-952]
jonahgabriel Dec 25, 2025
6a0ef9e
refactor(orchestrator): use declarative pattern with container inject…
jonahgabriel Dec 25, 2025
d350679
docs(CLAUDE.md): condense to <40K and add declarative node requiremen…
jonahgabriel Dec 25, 2025
a16839b
merge(main): resolve conflicts after OMN-816 VaultHandler rename
jonahgabriel Dec 25, 2025
c9d747e
fix(tests): update expected node description for ack handler [OMN-952]
jonahgabriel Dec 25, 2025
7a9e5fe
fix(pr-review): address final PR #79 nitpicks and polish [OMN-952]
jonahgabriel Dec 25, 2025
8bc48c1
merge(main): integrate OMN-949 DLQ features with OMN-952 orchestrator…
jonahgabriel Dec 25, 2025
469d53c
fix(tests): add missing timestamp field to DLQ routing test fixtures …
jonahgabriel Dec 25, 2025
9cdbf57
merge(main): integrate OMN-1006 heartbeat features with OMN-952 orche…
jonahgabriel Dec 25, 2025
f8fd2da
fix(pr-review): address PR #79 review feedback and fix integration te…
jonahgabriel Dec 26, 2025
634e1e5
fix(pr-review): address PR #79 release-ready review feedback [OMN-952]
jonahgabriel Dec 26, 2025
d38534f
fix(pr-review): address final PR #79 release-ready nitpicks [OMN-952]
jonahgabriel Dec 26, 2025
70ef5ca
merge(main): integrate OMN-955 chaos testing with orchestrator branch…
jonahgabriel Dec 26, 2025
9fbbc24
fix(pr-review): address PR release-ready feedback [OMN-952]
jonahgabriel Dec 26, 2025
8ce1c8c
fix(pr-review): address remaining PR #79 release-ready feedback [OMN-…
jonahgabriel Dec 26, 2025
b5957ad
merge(main): resolve conflicts with main branch [OMN-952]
jonahgabriel Dec 26, 2025
841ebba
fix(pr-review): address PR #79 release-ready issues and CI failures […
jonahgabriel Dec 26, 2025
c53b70b
refactor(architecture): consolidate orchestrator handlers into node d…
jonahgabriel Dec 26, 2025
021deac
merge(main): resolve mixins __init__.py conflict
jonahgabriel Dec 26, 2025
16a5790
merge(main): resolve conflict in operations README
jonahgabriel Dec 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,017 changes: 158 additions & 859 deletions CLAUDE.md

Large diffs are not rendered by default.

18 changes: 12 additions & 6 deletions docs/architecture/CIRCUIT_BREAKER_THREAD_SAFETY.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
# Circuit Breaker Thread Safety Implementation
# Circuit Breaker Concurrency Safety Implementation

## Overview

The ONEX infrastructure layer uses `MixinAsyncCircuitBreaker` to provide production-grade fault tolerance for infrastructure components. This mixin uses `asyncio.Lock` with a caller-held locking pattern to ensure thread-safe concurrent access.
The ONEX infrastructure layer uses `MixinAsyncCircuitBreaker` to provide production-grade fault tolerance for infrastructure components. This mixin uses `asyncio.Lock` with a caller-held locking pattern to ensure **coroutine-safe** concurrent access.

## Thread Safety Implementation
**Important Terminology Clarification**:
- **Coroutine safety** (asyncio.Lock): Protects against concurrent access from multiple asyncio coroutines within a single event loop
- **Thread safety** (threading.Lock): Protects against concurrent access from multiple OS threads

This implementation provides **coroutine safety**, not full thread safety. For multi-threaded asyncio usage, additional synchronization (e.g., `threading.Lock`) would be required. Most ONEX infrastructure uses single-threaded asyncio event loops, making coroutine safety sufficient.

## Coroutine Safety Implementation

### Lock Type: `asyncio.Lock` (Async Lock)

Expand All @@ -25,7 +31,7 @@ self._circuit_breaker_lock = asyncio.Lock()
The `MixinAsyncCircuitBreaker` uses a **caller-held locking pattern** where the caller must acquire the lock before calling circuit breaker methods. This pattern is documented in each method's docstring with:

```
Thread Safety:
Concurrency Safety:
REQUIRES: self._circuit_breaker_lock must be held by caller.
```

Expand Down Expand Up @@ -309,11 +315,11 @@ circuit_breaker_reset_timeout_seconds=30.0 # Auto-reset timeout

## Conclusion

The circuit breaker implementation provides production-grade thread safety with:
The circuit breaker implementation provides production-grade coroutine safety with:
- **Async-native design** using asyncio.Lock
- **Caller-held locking** for flexibility and clear responsibility
- **Comprehensive state machine** (CLOSED -> OPEN -> HALF_OPEN -> CLOSED)
- **Proper error context** with correlation ID propagation
- **Minimal overhead** < 10us per operation

Thread safety is guaranteed for all production workloads when callers follow the documented pattern of acquiring `_circuit_breaker_lock` before calling circuit breaker methods.
Coroutine safety is guaranteed for all single-threaded asyncio workloads when callers follow the documented pattern of acquiring `_circuit_breaker_lock` before calling circuit breaker methods. For multi-threaded usage, additional synchronization would be required.
Loading