Repository navigation
feat(replay): add configuration override injection for A/B testing (OMN-1205) - #331
Conversation
…MN-1205) Add ServiceConfigOverrideInjector to inject configuration overrides during replay, enabling A/B testing of different models, parameters, or endpoints while keeping inputs identical. Key features: - ModelConfigOverride and ModelConfigOverrideSet for override specifications - ModelReplayInput[T] to bundle inputs with optional overrides - Three-phase API: validate() -> preview() -> apply() - Environment overlay support (without mutating os.environ) - Nested dot-path traversal (e.g., "llm.openai.temperature") - Thread-safe, stateless service with deep-copy immutability Non-negotiable invariants enforced: - Original inputs are NEVER modified - Unspecified fields retain original values - No cross-contamination between concurrent replays 85 unit tests covering all core behaviors.
Move RecorderEffect, InjectorTime, and InjectorRNG from pipeline/replay/ to services/replay/ to fix structural misplacement. These utilities are passive observers and injection mechanisms - NOT nodes. Nodes are addressable graph vertices with contracts and dispatch semantics. Recorders and injectors are lifecycle-bound utilities for side-effect capture. The pipeline/ directory is governed by Node naming rules, which correctly flagged RecorderEffect as a violation. The fix is relocation, not renaming. Moved files: - pipeline/replay/recorder_effect.py → services/replay/recorder_effect.py - pipeline/replay/injector_time.py → services/replay/injector_time.py - pipeline/replay/injector_rng.py → services/replay/injector_rng.py Backwards compatibility maintained via re-exports from pipeline/replay/__init__.py.
Add RecorderEffect, InjectorTime, InjectorRNG, and ReplaySession to the naming convention exemptions for services/replay/. These classes are deterministic replay utilities (OMN-1116), NOT nodes. The heuristic incorrectly flagged "RecorderEffect" because it contains "effect" - but RecorderEffect RECORDS effects, it doesn't PERFORM them as a node would. Nodes are addressable graph vertices with contracts and dispatch semantics. Recorders and injectors are lifecycle-bound utilities for side-effect capture.
📝 WalkthroughWalkthroughReorganizes replay surface: moves recorder/injector types into a new services/replay package; adds EnumOverrideInjectionPoint and several immutable Pydantic models for config overrides; introduces ServiceConfigOverrideInjector and MISSING sentinel; updates pipeline re-exports and tests to the new service paths. (49 words) Changes
Sequence Diagram(s)sequenceDiagram
participant Caller
participant Injector as ServiceConfigOverrideInjector
participant Config as ConfigStore
participant Models as Validation/Preview/Result
rect rgb(235,245,255)
note right of Injector: validate(overrides, config, injection_point)
Caller->>Injector: validate(overrides, config, injection_point)
Injector->>Injector: validate path syntax & depth
Injector->>Config: read values (or MISSING)
Injector->>Injector: type checks, accumulate violations/warnings
Injector-->>Caller: ModelConfigOverrideValidation
end
rect rgb(245,255,240)
note right of Injector: preview(overrides, config, injection_point)
Caller->>Injector: preview(overrides, config, injection_point)
Injector->>Config: read original values
Injector->>Injector: build field previews, detect missing/type mismatches
Injector-->>Caller: ModelConfigOverridePreview
end
rect rgb(255,250,230)
note right of Injector: apply(overrides, config, injection_point)
Caller->>Injector: apply(overrides, config, injection_point)
Injector->>Injector: deep_copy(config)
Injector->>Config: operate on copy, set values/create intermediates
Injector->>Models: assemble ModelConfigOverrideResult
Injector-->>Caller: ModelConfigOverrideResult
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
Comment |
Combine replay infrastructure exemptions from both branches: - services/replay/: RecorderEffect, InjectorTime, InjectorRNG (relocated) - pipeline/replay/: ExecutorReplay, SessionReplay (remaining in pipeline)
Code Review - PR #331: Configuration Override Injection for A/B TestingExecutive SummaryThis PR introduces a comprehensive configuration override system for replay-based A/B testing. The implementation is well-architected, thoroughly tested, and follows ONEX conventions. I have a few minor suggestions but overall this is production-ready code. ✅ Strengths1. Excellent Architecture & Design
2. Robust Error Handling
3. Comprehensive Testing
4. Code Quality
5. Structural Improvements (OMN-1116)
🔍 Minor Issues & Suggestions1. Potential Issue: Frozen Pydantic ModelsLocation: The code attempts # Line 322-328
try:
setattr(current, final_part, value)
return (True, "")
except (AttributeError, TypeError, ValueError) as e:
return (False, f"Cannot set field '{final_part}' on Pydantic model: {e}")Issue: Frozen Pydantic models raise Recommendation: from pydantic import ValidationError
except (AttributeError, TypeError, ValueError, ValidationError) as e:
# catch-all-ok: boundary handler for all Pydantic/type errorsOr use from omnibase_core.errors.exception_groups import PYDANTIC_MODEL_ERRORS
except (AttributeError, TypeError, ValueError, *PYDANTIC_MODEL_ERRORS) as e:2. Documentation:
|
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/omnibase_core/services/replay/injector_rng.py (1)
1-10: Rename file to follow naming convention:injector_rng.py→service_injector_rng.pyPer the naming convention enforcer, all files in
src/omnibase_core/services/must use theservice_prefix. The file should be renamed frominjector_rng.pytoservice_injector_rng.py. Ensure all imports throughout the codebase are updated to reflect this change.Note:
injector_time.py,recorder_effect.py, andsentinel_missing.pyin the same directory also violate this convention and should be similarly renamed toservice_injector_time.py,service_recorder_effect.py, andservice_sentinel_missing.pyrespectively.
🧹 Nitpick comments (2)
src/omnibase_core/models/replay/model_config_override.py (1)
1-54: Consider adding__all__export for explicit public API.The module does not define
__all__. While not strictly required if the model is re-exported from__init__.py, adding__all__ = ["ModelConfigOverride"]would make the public API explicit and help with tooling/IDE support.🔎 Suggested addition
from omnibase_core.enums.replay.enum_override_injection_point import ( EnumOverrideInjectionPoint, ) + +__all__ = ["ModelConfigOverride"] class ModelConfigOverride(BaseModel):src/omnibase_core/services/replay/service_config_override_injector.py (1)
127-131: Unused class attributePATH_SYNTAX_PATTERN.The compiled regex pattern is defined but never used in the validation logic. The
_is_valid_path_syntaxmethod performs manual validation instead, which provides better error messages and flexibility for numeric indices.Consider removing this unused attribute to avoid confusion.
🔎 Proposed fix
- # Valid path segment pattern: starts with letter or underscore, - # followed by letters, digits, or underscores - PATH_SYNTAX_PATTERN: re.Pattern[str] = re.compile( - r"^[a-zA-Z_][a-zA-Z0-9_]*(\.[a-zA-Z_][a-zA-Z0-9_]*|\.[0-9]+)*$" - ) - # Maximum path depth to prevent DoS via deeply nested paths MAX_PATH_DEPTH: int = 20
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (31)
scripts/validation/validate_naming.pysrc/omnibase_core/enums/replay/__init__.pysrc/omnibase_core/enums/replay/enum_override_injection_point.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_result.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override_validation.pysrc/omnibase_core/models/replay/model_replay_input.pysrc/omnibase_core/pipeline/__init__.pysrc/omnibase_core/pipeline/replay/__init__.pysrc/omnibase_core/pipeline/replay/executor_replay.pysrc/omnibase_core/pipeline/replay/session_replay.pysrc/omnibase_core/protocols/replay/protocol_effect_recorder.pysrc/omnibase_core/protocols/replay/protocol_rng_service.pysrc/omnibase_core/protocols/replay/protocol_time_service.pysrc/omnibase_core/services/replay/__init__.pysrc/omnibase_core/services/replay/injector_rng.pysrc/omnibase_core/services/replay/injector_time.pysrc/omnibase_core/services/replay/recorder_effect.pysrc/omnibase_core/services/replay/sentinel_missing.pysrc/omnibase_core/services/replay/service_config_override_injector.pytests/unit/models/replay/test_model_config_overrides.pytests/unit/models/replay/test_model_replay_input.pytests/unit/pipeline/replay/test_injector_rng.pytests/unit/pipeline/replay/test_injector_time.pytests/unit/pipeline/replay/test_recorder_effect.pytests/unit/services/replay/__init__.pytests/unit/services/replay/test_service_config_override_injector.py
🧰 Additional context used
📓 Path-based instructions (4)
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: Always use Poetry for Python package management and task execution (poetry run pytest, poetry run mypy, poetry add, poetry install). Never use pip directly or python -m for execution.
Implement ONEX Four-Node Architecture following the unidirectional workflow: EFFECT → COMPUTE → REDUCER → ORCHESTRATOR. Each node must inherit from appropriate base class (NodeEffect, NodeCompute, NodeReducer, NodeOrchestrator).
Always call super().init(container) in node constructors, where container is of type ModelONEXContainer. This eliminates boilerplate initialization code.
Use ModelONEXContainer for dependency injection in node constructors, never use ModelContainer[T]. Resolve services using protocol names only (e.g., container.get_service('ProtocolEventBus')), never by concrete class names.
Use ModelOnexError with EnumCoreErrorCode for all exception handling. Never raise generic Exception. Use the @standard_error_handling decorator to eliminate try/catch boilerplate.
Use PEP 604 union syntax (str | None) instead of Optional[str] and Union types. This is enforced by ruff UP007.
Prefix exception tuples with standardized markers (# fallback-ok:, # catch-all-ok:, # cleanup-resilience-ok:, # boundary-ok:, # init-errors-ok:, # tool-resilience-ok:) to document intent of broad exception handlers.
Order exception tuples alphabetically by exception class name (AttributeError, TypeError, ValidationError, ValueError).
Use centralized exception groups from omnibase_core.errors.exception_groups (PYDANTIC_MODEL_ERRORS, VALIDATION_ERRORS, FILE_IO_ERRORS, etc.) instead of manual exception tuple construction.
Never share node instances across threads. Use thread-local instances or separate instances per thread. Node instances (NodeCompute, NodeEffect) are not thread-safe by default.
Use duck typing with protocols - never use isinstance() checks. Rely on protocol interfaces for type checking and service resolution.
Load runtime contracts using FileRegistry from omnibase_core.runti...
Files:
src/omnibase_core/models/replay/model_config_override_validation.pytests/unit/pipeline/replay/test_injector_rng.pysrc/omnibase_core/protocols/replay/protocol_rng_service.pysrc/omnibase_core/services/replay/__init__.pysrc/omnibase_core/services/replay/recorder_effect.pysrc/omnibase_core/pipeline/__init__.pysrc/omnibase_core/services/replay/sentinel_missing.pysrc/omnibase_core/enums/replay/enum_override_injection_point.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/services/replay/injector_time.pytests/unit/models/replay/test_model_replay_input.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/enums/replay/__init__.pysrc/omnibase_core/pipeline/replay/session_replay.pysrc/omnibase_core/pipeline/replay/executor_replay.pysrc/omnibase_core/models/replay/model_replay_input.pytests/unit/models/replay/test_model_config_overrides.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/protocols/replay/protocol_effect_recorder.pytests/unit/services/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override_set.pyscripts/validation/validate_naming.pysrc/omnibase_core/protocols/replay/protocol_time_service.pysrc/omnibase_core/models/replay/model_config_override.pytests/unit/pipeline/replay/test_recorder_effect.pytests/unit/services/replay/test_service_config_override_injector.pysrc/omnibase_core/services/replay/service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_result.pytests/unit/pipeline/replay/test_injector_time.pysrc/omnibase_core/pipeline/replay/__init__.pysrc/omnibase_core/services/replay/injector_rng.py
src/omnibase_core/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/omnibase_core/**/*.py: All Python files in src/omnibase_core/ must follow directory-specific naming prefixes enforced by checker_naming_convention.py. Examples: cli_, container_, decorator_, enum_, error_, exception_, factory_, model_, node_, mixin_, protocol_, service_, util_, validator_, checker_, resolver_, etc.
Maintain 100% mypy strict mode compliance and ensure pyright passes in CI. Type annotations must be complete and correct.
Files:
src/omnibase_core/models/replay/model_config_override_validation.pysrc/omnibase_core/protocols/replay/protocol_rng_service.pysrc/omnibase_core/services/replay/__init__.pysrc/omnibase_core/services/replay/recorder_effect.pysrc/omnibase_core/pipeline/__init__.pysrc/omnibase_core/services/replay/sentinel_missing.pysrc/omnibase_core/enums/replay/enum_override_injection_point.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/services/replay/injector_time.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/enums/replay/__init__.pysrc/omnibase_core/pipeline/replay/session_replay.pysrc/omnibase_core/pipeline/replay/executor_replay.pysrc/omnibase_core/models/replay/model_replay_input.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/protocols/replay/protocol_effect_recorder.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/protocols/replay/protocol_time_service.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/services/replay/service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_result.pysrc/omnibase_core/pipeline/replay/__init__.pysrc/omnibase_core/services/replay/injector_rng.py
src/omnibase_core/models/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/omnibase_core/models/**/*.py: Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.
Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.
Files:
src/omnibase_core/models/replay/model_config_override_validation.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_replay_input.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_result.py
tests/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
Test with: poetry run pytest tests/ for all tests, poetry run pytest tests/unit/ for unit tests only, poetry run pytest tests/ -n 0 -xvs for debug mode. Use markers: @pytest.mark.unit, @pytest.mark.integration, @pytest.mark.slow, @pytest.mark.smoke, @pytest.mark.performance.
Files:
tests/unit/pipeline/replay/test_injector_rng.pytests/unit/models/replay/test_model_replay_input.pytests/unit/models/replay/test_model_config_overrides.pytests/unit/services/replay/__init__.pytests/unit/pipeline/replay/test_recorder_effect.pytests/unit/services/replay/test_service_config_override_injector.pytests/unit/pipeline/replay/test_injector_time.py
🧠 Learnings (2)
📚 Learning: 2026-01-04T19:09:33.269Z
Learnt from: CR
Repo: OmniNode-ai/omnibase_core PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-04T19:09:33.269Z
Learning: Applies to src/omnibase_core/models/**/*.py : Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.
Applied to files:
src/omnibase_core/models/replay/model_config_override_validation.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_replay_input.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_result.py
📚 Learning: 2026-01-04T19:09:33.269Z
Learnt from: CR
Repo: OmniNode-ai/omnibase_core PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-04T19:09:33.269Z
Learning: Applies to src/omnibase_core/models/**/*.py : Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.
Applied to files:
src/omnibase_core/models/replay/model_config_override_validation.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_replay_input.pytests/unit/models/replay/test_model_config_overrides.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override.pytests/unit/services/replay/test_service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_result.py
🧬 Code graph analysis (14)
src/omnibase_core/models/replay/model_config_override_validation.py (1)
src/omnibase_core/protocols/validation/protocol_constraint_validation_result.py (1)
warnings(85-87)
tests/unit/pipeline/replay/test_injector_rng.py (1)
src/omnibase_core/services/replay/injector_rng.py (1)
InjectorRNG(64-185)
src/omnibase_core/pipeline/__init__.py (1)
src/omnibase_core/services/replay/injector_rng.py (1)
InjectorRNG(64-185)
src/omnibase_core/models/replay/model_config_override_field_preview.py (1)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)
src/omnibase_core/models/replay/model_config_override_preview.py (2)
src/omnibase_core/models/replay/model_config_override_field_preview.py (2)
ModelConfigOverrideFieldPreview(20-98)to_markdown_row(68-80)src/omnibase_core/services/replay/service_config_override_injector.py (1)
preview(475-563)
src/omnibase_core/models/replay/model_replay_input.py (1)
src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(21-50)
tests/unit/models/replay/test_model_config_overrides.py (3)
src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(20-54)src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override_set.py (3)
ModelConfigOverrideSet(21-50)by_injection_point(39-46)with_override(48-50)
src/omnibase_core/models/replay/__init__.py (8)
src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(20-54)src/omnibase_core/models/replay/model_config_override_field_preview.py (1)
ModelConfigOverrideFieldPreview(20-98)src/omnibase_core/models/replay/model_config_override_preview.py (1)
ModelConfigOverridePreview(18-98)src/omnibase_core/models/replay/model_config_override_result.py (1)
ModelConfigOverrideResult(16-43)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(21-50)src/omnibase_core/models/replay/model_effect_record.py (1)
ModelEffectRecord(75-158)src/omnibase_core/models/replay/model_replay_context.py (1)
ModelReplayContext(87-272)src/omnibase_core/models/replay/model_replay_input.py (1)
ModelReplayInput(24-80)
src/omnibase_core/models/replay/model_config_override_set.py (2)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(20-54)
src/omnibase_core/models/replay/model_config_override.py (1)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)
tests/unit/pipeline/replay/test_recorder_effect.py (4)
src/omnibase_core/services/replay/recorder_effect.py (1)
RecorderEffect(103-483)src/omnibase_core/services/replay/injector_time.py (1)
InjectorTime(62-172)src/omnibase_core/enums/replay/enum_recorder_mode.py (1)
EnumRecorderMode(45-67)src/omnibase_core/models/replay/model_effect_record.py (1)
ModelEffectRecord(75-158)
tests/unit/services/replay/test_service_config_override_injector.py (6)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(20-54)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(21-50)src/omnibase_core/services/replay/service_config_override_injector.py (3)
preview(475-563)apply(565-638)apply_environment_overlay(640-692)src/omnibase_core/models/replay/model_config_override_preview.py (1)
to_markdown(68-98)src/omnibase_core/services/replay/sentinel_missing.py (1)
_Missing(15-41)
tests/unit/pipeline/replay/test_injector_time.py (1)
src/omnibase_core/services/replay/injector_time.py (1)
InjectorTime(62-172)
src/omnibase_core/pipeline/replay/__init__.py (3)
src/omnibase_core/services/replay/injector_rng.py (1)
InjectorRNG(64-185)src/omnibase_core/services/replay/injector_time.py (1)
InjectorTime(62-172)src/omnibase_core/services/replay/recorder_effect.py (1)
RecorderEffect(103-483)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Tests (Split 12/20)
- GitHub Check: claude-review
🔇 Additional comments (50)
src/omnibase_core/services/replay/injector_time.py (1)
18-18: LGTM - Documentation correctly updated to reflect new module location.The import path in the usage example has been properly updated to match the relocated module structure.
src/omnibase_core/protocols/replay/protocol_time_service.py (1)
23-23: LGTM - Example import path updated correctly.The documentation example now imports from the new
services.replaylocation, consistent with the structural reorganization.tests/unit/services/replay/__init__.py (1)
1-15: LGTM - Standard test namespace initialization.The file follows best practices with proper SPDX headers, descriptive docstring, and explicitly typed empty
__all__declaration for the test namespace.src/omnibase_core/services/replay/__init__.py (1)
1-29: LGTM - Package initializer correctly implements OMN-1071 policy.The module documentation clearly states that services are not exported at the package level, with helpful examples showing the correct import patterns. The empty
__all__correctly enforces this policy.scripts/validation/validate_naming.py (1)
214-235: LGTM - Architectural exemptions correctly updated for replay reorganization.The exemptions are now properly split between
services/replay/(for injectors and recorders) andpipeline/replay/(for executors and sessions), with clear rationale explaining why these utilities don't follow standard node naming conventions.src/omnibase_core/services/replay/injector_rng.py (2)
18-18: Verify the docstring example import path aligns with actual module location.Line 18 shows the import as
from omnibase_core.services.replay.injector_rng import InjectorRNG. Confirm:
- This matches the actual module location (appears to be correct based on file path)
- The example is consistent with other documentation in the codebase
- If re-exports exist in
omnibase_core.pipeline.replayfor backward compatibility, consider documenting both import paths
64-110: Module-level protocol compliance check is good practice.The pattern at lines 188–189 (
_rng_check: ProtocolRNGService = InjectorRNG(seed=0)) provides compile-time verification thatInjectorRNGcorrectly implementsProtocolRNGService. This is excellent defensive programming.src/omnibase_core/protocols/replay/protocol_effect_recorder.py (1)
24-24: Import path update is correct and properly supported.The docstring example update importing
RecorderEffectfromomnibase_core.services.replay.recorder_effectis valid. Both the new path and the legacyomnibase_core.pipeline.replaypath work correctly—RecorderEffectis defined in services/replay and re-exported in pipeline/replay for backward compatibility. The example usage will execute without issue.src/omnibase_core/protocols/replay/protocol_rng_service.py (1)
93-93: Import path update is correct and properly re-exported.The new import path
omnibase_core.services.replay.injector_rngexists and properly exposesInjectorRNG. Backward compatibility re-exports are in place inomnibase_core.pipeline.replay.__init__.py, making the class accessible via both the direct import (recommended per OMN-1071 policy) and the re-export path. All consumers of the example code can import successfully.src/omnibase_core/services/replay/recorder_effect.py (1)
25-25: LGTM! Documentation examples updated correctly.The import paths in the docstring examples have been properly updated to reflect the new module location. This ensures users follow the correct import pattern.
Also applies to: 124-124
tests/unit/pipeline/replay/test_injector_rng.py (1)
338-338: LGTM! Test updated to verify new import path.The test correctly validates that
InjectorRNGcan be imported from the newservices.replaymodule location.src/omnibase_core/pipeline/replay/executor_replay.py (1)
95-97: LGTM! Imports updated to reflect architectural reorganization.The three replay infrastructure components (
InjectorRNG,InjectorTime,RecorderEffect) are now correctly imported from their new canonical location inservices.replay. This aligns with the architectural decision to treat these as passive utility services rather than pipeline nodes.tests/unit/pipeline/replay/test_recorder_effect.py (1)
34-34: LGTM! Test imports comprehensively updated.All test fixtures and test cases have been updated to import
RecorderEffectandInjectorTimefrom the newservices.replaymodule location. The updates are consistent throughout the file and maintain test functionality.Also applies to: 46-46, 74-74, 85-85, 120-120, 390-390, 453-455, 490-492, 532-532
src/omnibase_core/enums/replay/__init__.py (1)
6-10: LGTM! New enum properly integrated.The
EnumOverrideInjectionPointenum is correctly added to the module exports with updated documentation. This supports the new configuration override injection feature for A/B testing (OMN-1205).Also applies to: 15-17, 22-22
tests/unit/pipeline/replay/test_injector_time.py (1)
29-29: LGTM! Import paths consistently updated.All imports of
InjectorTimehave been correctly migrated fromomnibase_core.pipeline.replay.injector_timetoomnibase_core.services.replay.injector_time, aligning with the structural relocation of replay components to the services module.Also applies to: 35-35, 49-49, 128-128, 159-159, 183-183, 238-238, 248-248, 258-258
src/omnibase_core/pipeline/__init__.py (1)
94-94: LGTM! Import path updated while preserving public API.The
InjectorRNGimport source has been correctly updated to the newservices.replaylocation. The re-export in__all__maintains backwards compatibility for consumers importing fromomnibase_core.pipeline.src/omnibase_core/pipeline/replay/session_replay.py (1)
66-68: LGTM! Import paths consistently migrated.All imports for
InjectorRNG,InjectorTime, andRecorderEffecthave been updated to the newomnibase_core.services.replaymodule locations, both inTYPE_CHECKINGblocks and in the deferred factory functions. The deferred import pattern is correctly maintained to avoid circular dependencies.Also applies to: 134-134, 141-141, 148-148
src/omnibase_core/services/replay/sentinel_missing.py (1)
15-46: LGTM! Well-designed sentinel for distinguishing missing paths from None.The singleton pattern is correctly implemented. While the
__new__method has a theoretical race condition, this is mitigated by creating the singleton at module load time (line 45), which is protected by Python's import lock. The__bool__returningFalseis a sensible choice for intuitive truthiness checks.tests/unit/models/replay/test_model_config_overrides.py (2)
80-519: LGTM! Comprehensive test coverage for config override models.Excellent test structure covering:
- Creation scenarios (defaults, explicit values, complex/None values)
- Immutability verification via
ValidationErroron frozen models- Field validation (path min_length, required fields, extra fields forbidden)
- Serialization roundtrips (dict and JSON)
ModelConfigOverrideSetgrouping (by_injection_point) and immutable updates (with_override)All test classes correctly use
@pytest.mark.unitmarker as per guidelines.
26-29: Fix incorrect import path: use singular form or package-level import.The test imports from
omnibase_core.models.replay.model_config_overrides(plural), but no such module exists. All actual files use singular form (model_config_override.py,model_config_override_set.py). The__init__.pyre-exports these models, so either import directly from the package or use the singular file path:
from omnibase_core.models.replay import ModelConfigOverride(preferred via init.py), orfrom omnibase_core.models.replay.model_config_override import ModelConfigOverride(direct from singular file)Also applies to: 35-35
⛔ Skipped due to learnings
Learnt from: CR Repo: OmniNode-ai/omnibase_core PR: 0 File: CLAUDE.md:0-0 Timestamp: 2026-01-04T19:09:33.269Z Learning: Applies to src/omnibase_core/models/**/*.py : Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.Learnt from: CR Repo: OmniNode-ai/omnibase_core PR: 0 File: CLAUDE.md:0-0 Timestamp: 2026-01-04T19:09:33.269Z Learning: Applies to src/omnibase_core/**/*.py : All Python files in src/omnibase_core/ must follow directory-specific naming prefixes enforced by checker_naming_convention.py. Examples: cli_*, container_*, decorator_*, enum_*, error_*, exception_*, factory_*, model_*, node_*, mixin_*, protocol_*, service_*, util_*, validator_*, checker_*, resolver_*, etc.Learnt from: CR Repo: OmniNode-ai/omnibase_core PR: 0 File: CLAUDE.md:0-0 Timestamp: 2026-01-04T19:09:33.269Z Learning: Applies to src/omnibase_core/models/**/*.py : Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
16-55: LGTM! Well-designed enum with proper documentation.The
EnumOverrideInjectionPointfollows best practices:
- Inherits from
str, Enumfor seamless JSON serialization- Lowercase string values align with Python/JSON conventions
- Comprehensive docstring with thread safety notes and usage examples
- Correct
__all__exportsrc/omnibase_core/models/replay/model_config_override.py (1)
40-54: LGTM! Model follows Pydantic best practices and coding guidelines.The
ModelConfigOverrideis correctly configured:
ConfigDict(frozen=True, extra="forbid", from_attributes=True)— immutable, strict, and compatible with pytest-xdist parallel execution (per learnings)pathfield withmin_length=1prevents empty path errorsinjection_pointdefaults toHANDLER_CONFIGfor the common casesrc/omnibase_core/models/replay/model_config_override_validation.py (1)
75-137: LGTM! Well-designed immutable validation result model.The model correctly follows the violations/warnings/suggestions pattern consistent with other validation components in the codebase. Proper use of
ConfigDict(frozen=True, extra="forbid", from_attributes=True)ensures immutability and pytest-xdist compatibility. Usingtupletypes for collections enforces immutability at the field level.tests/unit/models/replay/test_model_replay_input.py (2)
27-33: Good use of TYPE_CHECKING for type hints.Using
TYPE_CHECKINGguard with lazy imports inside fixtures avoids circular import issues while maintaining type safety. This is a clean pattern for test files.
85-401: Comprehensive test coverage for ModelReplayInput.The tests thoroughly cover model creation, immutability enforcement, validation, the
has_overridesproperty edge cases (None, empty, populated), immutable update viawith_overrides, generic type preservation, and serialization roundtrips. Good coverage of the model's invariants.src/omnibase_core/models/replay/model_replay_input.py (1)
24-81: LGTM! Well-implemented generic replay input model.The generic
ModelReplayInput[T_Data]correctly uses Pydantic v2's pattern for generic models. Thehas_overridesproperty properly handles bothNoneand empty override sets. Thewith_overridesmethod correctly implements the immutable update pattern by preserving theinput_idanddatawhile replacing overrides.src/omnibase_core/pipeline/replay/__init__.py (1)
54-66: Good backwards-compatible re-export structure.The re-exports from
omnibase_core.services.replaymaintain backwards compatibility while the documentation clearly indicates the new canonical import paths. This allows existing code to continue working while guiding users to the preferred imports.src/omnibase_core/models/replay/model_config_override_set.py (3)
31-31: LGTM on model configuration!The
ConfigDictproperly includesfrom_attributes=Truefor pytest-xdist parallel execution compatibility, along withfrozen=Truefor immutability andextra="forbid"for strict validation. Based on learnings, this follows the project guidelines.
38-46: LGTM!The
by_injection_pointproperty correctly creates a new dict with new lists on each call, preserving the model's immutability guarantees. The grouping logic usingsetdefaultis concise and correct.
48-50: LGTM!The
with_overridemethod correctly implements the immutable update pattern, returning a new instance with the additional override appended.tests/unit/services/replay/test_service_config_override_injector.py (9)
98-124: Test Pydantic models are appropriately configured for test scope.These test models are local fixtures and not shared across pytest-xdist workers as value objects, so the absence of
from_attributes=Trueis acceptable here. The frozen model (FrozenPydanticConfig) correctly tests the injector's behavior with immutable Pydantic models.
131-238: LGTM!The
TestValidateclass provides good coverage for validation scenarios including path existence, type checking, strict mode behavior, and path syntax validation. The assertions verify both positive and negative cases correctly.
245-332: LGTM!The
TestPreviewclass adequately tests the preview functionality including before/after state tracking, new path identification, markdown table generation, and type mismatch detection.
339-437: LGTM on critical immutability tests!The
TestApplyclass properly verifies the key invariants:
- Original configuration is never modified (verified with deep copy comparison)
- Patched config is a new object with deep-copied nested structures
- Unspecified fields are preserved
- Path creation is tracked correctly
These tests are essential for ensuring the injector's safety guarantees.
444-534: LGTM!The
TestEnvironmentOverlayclass correctly verifies that:
os.environis never mutated- Only
ENVIRONMENTinjection point overrides are included- Values are properly converted to strings (including
None→"",bool→"true"/"false")
541-619: LGTM!The
TestPathTraversalclass provides comprehensive coverage for nested path handling including deeply nested dicts, list index access, out-of-bounds error handling, and automatic intermediate dict creation.
626-709: LGTM!The
TestPydanticModelConfigclass adequately tests Pydantic model support including mutable models, frozen model error handling, path validation, and nested model traversal.
716-831: LGTM on thread-safety tests!The
TestConcurrentAccessclass properly verifies the service's thread-safety guarantees:
- Concurrent
apply()calls don't cause race conditions- Original config remains unchanged across all concurrent operations
- Single injector instance can be safely reused across threads
- Each result has independent patched config
The use of
ThreadPoolExecutorwith proper synchronization for result collection is appropriate.
838-945: LGTM!The
TestEdgeCasesclass covers important boundary conditions:
- Empty override set handling
- Setting values to
NoneMISSINGsentinel singleton behavior- Injection point filtering
- Path depth limit enforcement (DoS prevention)
Good coverage for robustness.
src/omnibase_core/models/replay/__init__.py (2)
46-62: LGTM!The new model imports are properly organized and follow the existing pattern. All seven new override-related models and
ModelReplayInputare correctly imported from their respective modules.
64-74: LGTM!The
__all__list is properly updated with all new exports in alphabetical order, maintaining consistency with the module's public API surface.src/omnibase_core/services/replay/service_config_override_injector.py (9)
104-144: LGTM on class design!The service is correctly designed as stateless (only
strict_modeconfiguration), making it inherently thread-safe. The class attributesPATH_SYNTAX_PATTERNandMAX_PATH_DEPTHare appropriate constants for validation.
200-246: LGTM!The
_get_value_at_pathmethod correctly traverses nested structures (dict, Pydantic model, list/tuple) and returns theMISSINGsentinel when paths don't exist. The fallback togetattrfor other object types provides reasonable extensibility.
248-344: LGTM!The
_set_value_at_pathmethod correctly handles in-place modification with proper error handling:
- Creates intermediate dicts when needed
- Handles Pydantic models with appropriate exception catching
- Returns informative error messages for various failure cases
- Exception tuple
(AttributeError, TypeError, ValueError)is alphabetically ordered per coding guidelines
346-364: LGTM!The
_deep_copy_configmethod correctly uses Pydantic'smodel_copy(deep=True)for BaseModel instances andcopy.deepcopyfor other types, ensuring proper deep copying semantics for both cases.
366-473: LGTM!The
validatemethod provides comprehensive validation:
- Path syntax validation
- Path existence checking (warning vs. violation based on injection point)
- Type compatibility checking with numeric conversion allowance
- Strict mode support for failing on warnings
The logic correctly distinguishes between violations (hard errors) and warnings (soft issues).
475-563: LGTM!The
previewmethod correctly generates a preview of configuration changes:
- Skips invalid paths gracefully
- Tracks paths that don't exist
- Identifies type mismatches (excluding numeric conversions)
- Creates detailed field previews with before/after values
565-638: LGTM on the criticalapplymethod!The implementation correctly maintains the key invariant that the original config is never modified:
- Deep copies the config before any modification
- Tracks which paths were newly created
- Collects errors without throwing exceptions
- Returns a comprehensive result object
This is the core functionality for safe A/B testing configuration injection.
640-692: LGTM!The
apply_environment_overlaymethod correctly creates an environment overlay dict:
- Only includes
ENVIRONMENTinjection point overrides- Uses the last path segment as the env var name (uppercased by convention)
- Properly converts values to strings (
None→"",bool→"true"/"false")- Never mutates
os.environ
695-695: LGTM!The
__all__export correctly exposesServiceConfigOverrideInjectorandMISSINGas the public API of this module.
- Update test imports from model_config_overrides to model_config_override and model_config_override_set (file was renamed/split in prior commit) - Add __all__ exports to all replay model files for consistency - Remove unused PATH_SYNTAX_PATTERN from ServiceConfigOverrideInjector - Fix _Missing import in test_service_config_override_injector.py Fixes CI failures in Test Suite (Splits 16/20 and 19/20). Addresses PR review nitpicks about missing exports.
Code Review: Configuration Override Injection for A/B Testing (OMN-1205)SummaryThis is an excellent PR that adds sophisticated configuration override capabilities to the replay infrastructure. The implementation is well-architected, thoroughly tested, and follows ONEX best practices. The code demonstrates strong software engineering principles with a focus on immutability, thread-safety, and clear API design. Strengths 🌟1. Exemplary Architecture & Design
2. Code Quality Excellence
3. Test Coverage Outstanding
4. Documentation Quality
5. Structural Refactoring (OMN-1116)
Issues & Recommendations 🔍1. Property Returns Mutable Dict (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
tests/unit/services/replay/test_service_config_override_injector.py (2)
98-124: Consider addingfrom_attributes=Trueto test Pydantic models for pytest-xdist compatibility.The test Pydantic models (
SamplePydanticConfig,FrozenPydanticConfig,NestedPydanticConfig) don't includefrom_attributes=Truein theirConfigDict. While these are test fixtures rather than production models, adding this setting would prevent potential class identity issues during parallel test execution with pytest-xdist.🔎 Suggested update
class SamplePydanticConfig(BaseModel): """Sample Pydantic model for testing override application.""" - model_config = ConfigDict(extra="forbid") + model_config = ConfigDict(extra="forbid", from_attributes=True) timeout: int = 30 debug: bool = False name: str = "default" class FrozenPydanticConfig(BaseModel): """Frozen Pydantic model for testing immutability.""" - model_config = ConfigDict(frozen=True, extra="forbid") + model_config = ConfigDict(frozen=True, extra="forbid", from_attributes=True) timeout: int = 30 debug: bool = False class NestedPydanticConfig(BaseModel): """Nested Pydantic model for testing deep paths.""" - model_config = ConfigDict(extra="forbid") + model_config = ConfigDict(extra="forbid", from_attributes=True) inner: SamplePydanticConfig = SamplePydanticConfig() level: int = 1Based on learnings, this prevents class identity mismatches across pytest-xdist worker processes.
448-477: Tighten environment variable name assertions.Lines 476-477 check for
"TEST_VAR" in overlay or "test_var" in overlay, which is imprecise. Based on the service implementation (lines 670-673), the environment overlay usesparts[-1].upper(), so "test_var" should always become "TEST_VAR", not "test_var".🔎 More precise assertions
# Overlay should contain the environment variables - assert "TEST_VAR" in overlay or "test_var" in overlay + assert "TEST_VAR" in overlay assert "KEY" in overlay # Last segment, uppercasedsrc/omnibase_core/services/replay/service_config_override_injector.py (1)
315-322: Document exception tuple intent with standardized marker.The exception handler at line 318 catches multiple exception types. Per coding guidelines, broad exception handlers should be prefixed with standardized markers like
# fallback-ok:,# boundary-ok:, etc., to document the intent.🔎 Add exception tuple marker
try: setattr(current, final_part, value) return (True, "") + # boundary-ok: Pydantic field assignment validation except (AttributeError, TypeError, ValueError) as e: return ( False, f"Cannot set field '{final_part}' on Pydantic model: {e}", )As per coding guidelines, this documents the boundary validation intent of the exception handler.
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
src/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_result.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/services/replay/service_config_override_injector.pytests/unit/models/replay/test_model_config_overrides.pytests/unit/models/replay/test_model_replay_input.pytests/unit/services/replay/test_service_config_override_injector.py
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/unit/models/replay/test_model_config_overrides.py
- src/omnibase_core/models/replay/model_config_override_result.py
🧰 Additional context used
📓 Path-based instructions (4)
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: Always use Poetry for Python package management and task execution (poetry run pytest, poetry run mypy, poetry add, poetry install). Never use pip directly or python -m for execution.
Implement ONEX Four-Node Architecture following the unidirectional workflow: EFFECT → COMPUTE → REDUCER → ORCHESTRATOR. Each node must inherit from appropriate base class (NodeEffect, NodeCompute, NodeReducer, NodeOrchestrator).
Always call super().init(container) in node constructors, where container is of type ModelONEXContainer. This eliminates boilerplate initialization code.
Use ModelONEXContainer for dependency injection in node constructors, never use ModelContainer[T]. Resolve services using protocol names only (e.g., container.get_service('ProtocolEventBus')), never by concrete class names.
Use ModelOnexError with EnumCoreErrorCode for all exception handling. Never raise generic Exception. Use the @standard_error_handling decorator to eliminate try/catch boilerplate.
Use PEP 604 union syntax (str | None) instead of Optional[str] and Union types. This is enforced by ruff UP007.
Prefix exception tuples with standardized markers (# fallback-ok:, # catch-all-ok:, # cleanup-resilience-ok:, # boundary-ok:, # init-errors-ok:, # tool-resilience-ok:) to document intent of broad exception handlers.
Order exception tuples alphabetically by exception class name (AttributeError, TypeError, ValidationError, ValueError).
Use centralized exception groups from omnibase_core.errors.exception_groups (PYDANTIC_MODEL_ERRORS, VALIDATION_ERRORS, FILE_IO_ERRORS, etc.) instead of manual exception tuple construction.
Never share node instances across threads. Use thread-local instances or separate instances per thread. Node instances (NodeCompute, NodeEffect) are not thread-safe by default.
Use duck typing with protocols - never use isinstance() checks. Rely on protocol interfaces for type checking and service resolution.
Load runtime contracts using FileRegistry from omnibase_core.runti...
Files:
src/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/services/replay/service_config_override_injector.pytests/unit/models/replay/test_model_replay_input.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pytests/unit/services/replay/test_service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override.py
src/omnibase_core/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/omnibase_core/**/*.py: All Python files in src/omnibase_core/ must follow directory-specific naming prefixes enforced by checker_naming_convention.py. Examples: cli_, container_, decorator_, enum_, error_, exception_, factory_, model_, node_, mixin_, protocol_, service_, util_, validator_, checker_, resolver_, etc.
Maintain 100% mypy strict mode compliance and ensure pyright passes in CI. Type annotations must be complete and correct.
Files:
src/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/services/replay/service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override.py
src/omnibase_core/models/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/omnibase_core/models/**/*.py: Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.
Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.
Files:
src/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override.py
tests/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
Test with: poetry run pytest tests/ for all tests, poetry run pytest tests/unit/ for unit tests only, poetry run pytest tests/ -n 0 -xvs for debug mode. Use markers: @pytest.mark.unit, @pytest.mark.integration, @pytest.mark.slow, @pytest.mark.smoke, @pytest.mark.performance.
Files:
tests/unit/models/replay/test_model_replay_input.pytests/unit/services/replay/test_service_config_override_injector.py
🧠 Learnings (2)
📚 Learning: 2026-01-04T19:09:33.280Z
Learnt from: CR
Repo: OmniNode-ai/omnibase_core PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-04T19:09:33.280Z
Learning: Applies to src/omnibase_core/models/**/*.py : Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.
Applied to files:
src/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pytests/unit/services/replay/test_service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override.py
📚 Learning: 2026-01-04T19:09:33.280Z
Learnt from: CR
Repo: OmniNode-ai/omnibase_core PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-04T19:09:33.280Z
Learning: Applies to src/omnibase_core/models/**/*.py : Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.
Applied to files:
src/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override.py
🧬 Code graph analysis (7)
src/omnibase_core/models/replay/model_config_override_set.py (2)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(22-56)
src/omnibase_core/services/replay/service_config_override_injector.py (6)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override_field_preview.py (1)
ModelConfigOverrideFieldPreview(22-100)src/omnibase_core/models/replay/model_config_override_preview.py (1)
ModelConfigOverridePreview(20-100)src/omnibase_core/models/replay/model_config_override_result.py (1)
ModelConfigOverrideResult(18-45)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(23-52)src/omnibase_core/models/replay/model_config_override_validation.py (1)
ModelConfigOverrideValidation(75-134)
tests/unit/models/replay/test_model_replay_input.py (3)
src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(22-56)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(23-52)src/omnibase_core/models/replay/model_replay_input.py (3)
ModelReplayInput(24-80)has_overrides(65-70)with_overrides(72-80)
src/omnibase_core/models/replay/model_config_override_preview.py (2)
src/omnibase_core/models/replay/model_config_override_field_preview.py (2)
ModelConfigOverrideFieldPreview(22-100)to_markdown_row(70-82)src/omnibase_core/services/replay/service_config_override_injector.py (1)
preview(468-556)
src/omnibase_core/models/replay/model_config_override_field_preview.py (3)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/core/model_extension_data.py (1)
value_type(89-105)src/omnibase_core/protocols/event_bus/protocol_event_message.py (1)
value(33-35)
tests/unit/services/replay/test_service_config_override_injector.py (5)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(22-56)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(23-52)src/omnibase_core/services/replay/service_config_override_injector.py (5)
ServiceConfigOverrideInjector(104-685)validate(359-466)preview(468-556)apply(558-631)apply_environment_overlay(633-685)src/omnibase_core/services/replay/sentinel_missing.py (1)
_Missing(15-41)
src/omnibase_core/models/replay/model_config_override.py (1)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (21)
- GitHub Check: Tests (Split 1/20)
- GitHub Check: Tests (Split 9/20)
- GitHub Check: Tests (Split 8/20)
- GitHub Check: Tests (Split 12/20)
- GitHub Check: Tests (Split 16/20)
- GitHub Check: Tests (Split 15/20)
- GitHub Check: Tests (Split 14/20)
- GitHub Check: Tests (Split 19/20)
- GitHub Check: Tests (Split 10/20)
- GitHub Check: Tests (Split 7/20)
- GitHub Check: Tests (Split 11/20)
- GitHub Check: Tests (Split 13/20)
- GitHub Check: Tests (Split 17/20)
- GitHub Check: Tests (Split 18/20)
- GitHub Check: Tests (Split 5/20)
- GitHub Check: Tests (Split 20/20)
- GitHub Check: Tests (Split 2/20)
- GitHub Check: Tests (Split 6/20)
- GitHub Check: Tests (Split 3/20)
- GitHub Check: Tests (Split 4/20)
- GitHub Check: claude-review
🔇 Additional comments (21)
tests/unit/models/replay/test_model_replay_input.py (9)
19-29: Excellent use of TYPE_CHECKING for forward references.The conditional import pattern correctly avoids circular dependencies while maintaining type hints for IDEs and type checkers.
32-77: Well-structured fixtures provide good test isolation.The fixture hierarchy (sample_data → simple_override → override_set → replay_input variants) creates reusable test data with clear dependencies. The lazy imports within fixtures ensure models are loaded only when needed.
79-138: Comprehensive creation tests validate all construction paths.Tests cover:
- Data-only creation (default overrides)
- Auto-generated UUIDs with uniqueness
- Explicit UUID assignment
- Override inclusion
This ensures the model's factory defaults and explicit parameters work correctly.
140-168: Immutability tests properly validate frozen model behavior.The ValidationError assertions confirm that Pydantic's
frozen=Trueprevents field reassignment on all fields (data, input_id, config_overrides), which is critical for the thread-safety guarantee documented in the PR.
170-196: Validation tests enforce required fields and extra field rejection.The tests correctly verify:
datais required (line 179)- Extra fields trigger ValidationError with clear messages (lines 183-195)
This aligns with the
extra="forbid"ConfigDict setting in the model.
198-225: has_overrides property logic thoroughly tested.All three branches are covered:
- None overrides → False
- Empty override set → False
- Non-empty overrides → True
This validates the short-circuit logic in the property implementation.
227-282: with_overrides immutable update pattern properly validated.Tests confirm:
- New instance returned (not mutation)
- input_id preserved across updates
- Original data preserved
- Existing overrides replaced (not merged)
This validates the immutable update contract documented in ModelReplayInput.
284-326: Generic type preservation validated across diverse data types.Tests exercise the Generic[T_Data] with dict, list, string, and nested structures, confirming that Pydantic preserves the original data types without coercion.
328-392: Serialization roundtrips validate persistence contract.Tests confirm:
- model_dump/model_validate roundtrip preserves all fields
- JSON serialization via model_dump_json/model_validate_json works correctly
- UUID serializes as string (line 386-392), which is the expected JSON representation
This ensures ModelReplayInput can be safely persisted and restored.
src/omnibase_core/models/replay/model_config_override_field_preview.py (3)
19-19: Proper module export added.The
__all__export ensures explicit public API surface, addressing the past review comment. This aligns with package conventions.
22-68: Well-designed immutable preview model.The model correctly:
- Uses
ConfigDict(frozen=True, extra="forbid", from_attributes=True)per guidelines- Provides clear field descriptions
- Documents thread-safety guarantees
- Includes helpful docstring example
The field defaults (
value_type="unknown",path_exists=True) are sensible for typical use cases.Based on learnings and coding guidelines.
70-100: Clean markdown rendering with appropriate value formatting.The
to_markdown_row()and_format_value()methods produce readable table rows. The value formatter correctly:
- Renders
Noneas`null`- Wraps strings in quotes:
`"value"`- Backtick-wraps other types
Note: The docstring mentions "MISSING sentinel" for
old_value(line 32), but_format_valuehandles it asNone. This is correct—the service layer (service_config_override_injector.py lines 520-524) converts MISSING to None for display.src/omnibase_core/models/replay/model_config_override_set.py (3)
20-20: Explicit module export present.The
__all__declaration correctly exposes the public API.
23-38: Immutable collection model properly configured.The model follows all guidelines:
ConfigDict(frozen=True, extra="forbid", from_attributes=True)for pytest-xdist compatibility- Uses
tuplefor the overrides collection, maintaining immutability- Default factory prevents shared mutable defaults
Based on learnings and coding guidelines.
40-52: Efficient grouping and immutable update patterns.The
by_injection_pointproperty computes a fresh dict on each access, which is acceptable since:
- The model is frozen (no internal caching possible)
- The returned dict is not shared state
- Typical usage involves a single call for batch processing
The
with_overridemethod correctly implements the immutable update pattern using tuple unpacking(*self.overrides, override).src/omnibase_core/models/replay/model_config_override.py (2)
19-19: Module export properly declared.
22-56: Well-designed single override specification.The model implementation is excellent:
ConfigDict(frozen=True, extra="forbid", from_attributes=True)per guidelinespathfield enforcesmin_length=1to prevent empty pathsvalueusesAnytype, which is correct for configuration overrides that can be any JSON-serializable valueinjection_pointdefaults toHANDLER_CONFIG, the most common use case- Clear documentation with example usage
Based on learnings and coding guidelines.
src/omnibase_core/models/replay/model_config_override_preview.py (3)
17-17: Module export added per previous review.The
__all__declaration addresses the past review comment, ensuring consistent public API exposure.
20-68: Comprehensive aggregate preview model.The model design is excellent:
ConfigDict(frozen=True, extra="forbid", from_attributes=True)per guidelines- All collections use
tupleto maintain immutability- Clear separation of concerns: field previews, missing paths, type mismatches
- Rich docstring with example showing expected usage
Based on learnings and coding guidelines.
70-100: Clean markdown diff generation with conditional sections.The
to_markdown()method produces a well-structured diff report:
- Standard markdown table with headers
- Conditional rendering of "New Paths" and "Type Mismatches" sections (only when populated)
- Each section is clearly labeled and formatted
This aligns with the preview/validation/apply workflow documented in the PR objectives.
src/omnibase_core/services/replay/service_config_override_injector.py (1)
104-685: Excellent implementation with strong immutability and thread-safety guarantees.The service implementation demonstrates several strengths:
- Thread safety: Correctly implemented through stateless design and deep-copy semantics. The
apply()method's deep copy on line 589 ensures the original config is never mutated.- Path traversal: Comprehensive handling of dicts, Pydantic models, and lists with proper bounds checking and the MISSING sentinel pattern.
- Error handling: Good coverage of edge cases including frozen Pydantic models, out-of-bounds indices, and invalid path syntax.
- DoS prevention: MAX_PATH_DEPTH limit (line 127) protects against deeply nested path attacks.
- Environment overlay safety: Correctly avoids mutating
os.environand returns an isolated dict.The implementation aligns well with the PR objectives for A/B testing support while maintaining the stated invariants.
Merged OMN-1202 (execution corpus) with OMN-1205 (config overrides). Combined all replay model imports and exports from both branches.
MAJOR fixes: - Add robust error handling with None input validation - Return immutable MappingProxyType from apply_environment_overlay - Add frozen Pydantic model detection and dict conversion MINOR fixes: - Add from_attributes=True rationale comments to all models - Add complete docstrings to ModelConfigOverrideSet methods - Document environment variable naming derivation logic - Add test_preview_filters_by_injection_point test NITPICK fixes: - Add from_attributes=True to test Pydantic models - Tighten environment variable assertions in tests - Add exception handling with alphabetical ordering and markers - Add test_env_overlay_is_immutable test - Add test_is_frozen_pydantic_model_detection test OMN-1205
PR Review: Configuration Override Injection for A/B Testing (OMN-1205)Overall AssessmentVerdict: ✅ APPROVE with minor suggestions This is a well-architected, thoroughly tested feature that adds configuration override injection capabilities to the replay infrastructure. The implementation demonstrates excellent code quality, comprehensive testing, and careful attention to thread safety and immutability guarantees. Code Quality ✅Strengths
Architecture & Design ✅Well-Designed Features
Security Considerations ✅Good Security Practices
No Critical Security Issues FoundTesting ✅Test Coverage
Test Quality
Test Coverage Appears ExcellentThe test file structure shows thorough coverage of all major code paths. Potential Issues & SuggestionsMINOR: Performance ConsiderationLocation: The Suggestion: Consider adding a comment about potential performance implications for large frozen models, or benchmark if this becomes a bottleneck. # Note: Frozen models require dict conversion for mutability.
# For large configs (>1000 fields), this may impact performance.
return copy.deepcopy(config.model_dump())MINOR: Type Mismatch Warning LogicLocation: The numeric type conversion exemption ( Current Behavior: # Type mismatch str → int produces WARNING (not VIOLATION)
override = ModelConfigOverride(path="timeout", value="30") # str
config = {"timeout": 30} # int → WARNINGSuggestion: Document the rationale for warnings vs violations for type mismatches, or add a NITPICK: Exception Tuple OrderingLocation: Exception tuples follow alphabetical ordering per CLAUDE.md, but the ordering could be more consistent: # Line 294 - Good
except (AttributeError, IndexError, KeyError, TypeError) as e:
# Line 395 - Good
except (AttributeError, TypeError, ValueError) as e:All exception tuples appear correctly ordered. ✅ NITPICK: Environment Variable Naming DocumentationLocation: The environment variable naming derivation is excellently documented with examples and rationale. This is a model for good documentation. Minor Enhancement: Consider adding a note about potential collisions: # Potential collision example:
# "config.db.timeout" → TIMEOUT
# "api.timeout" → TIMEOUT (COLLISION!)This is already implicitly handled (last write wins), but documenting it would prevent confusion. Best Practices Adherence ✅CLAUDE.md Compliance
Structural Improvements (OMN-1116) ✅Good RefactoringThe relocation of
This demonstrates good architectural discipline. Performance ConsiderationsStrengths
Potential Concerns
Verdict: Performance is appropriate for the replay/testing use case. Documentation Quality ✅Outstanding Documentation
Example of Excellent Documentation (
RecommendationsFor Immediate ActionNone - The PR is ready to merge as-is. For Future Consideration
Final NotesThis PR demonstrates exemplary engineering practices:
The latest commit (3ca288d) addressed all prior review feedback, including:
Great work! This is production-ready code. 🚀 Approval✅ APPROVED - Ready to merge. Reviewed by: Claude Code (Sonnet 4.5) |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/omnibase_core/services/replay/service_config_override_injector.py (1)
867-884: Consider warning users about potential environment variable name collisions.The current logic uses
parts[-1].upper()for environment variable naming, which means:
"config.timeout"→"TIMEOUT""other.timeout"→"TIMEOUT"(collision!)While this is documented in the method docstring, silent overwrites could cause subtle bugs. Consider adding a warning log when collisions occur:
🔎 Proposed collision detection
for override in overrides.overrides: if override.injection_point != EnumOverrideInjectionPoint.ENVIRONMENT: continue # Environment variable names are the path (or last segment) # Path can be the full env var name or nested for grouping parts = override.path.split(".") env_name = parts[-1].upper() # Use last segment, uppercase by convention + if env_name in env_overlay: + logger.warning( + "Environment variable collision: '%s' from path '%s' " + "overwrites previous value from another path", + env_name, + override.path, + ) + # Convert value to string if override.value is None: value_str = ""
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
scripts/validation/validate_naming.pysrc/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_result.pysrc/omnibase_core/models/replay/model_config_override_set.pysrc/omnibase_core/services/replay/service_config_override_injector.pytests/unit/services/replay/test_service_config_override_injector.py
🚧 Files skipped from review as they are similar to previous changes (2)
- scripts/validation/validate_naming.py
- src/omnibase_core/models/replay/model_config_override_result.py
🧰 Additional context used
📓 Path-based instructions (4)
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: Always use Poetry for Python package management and task execution (poetry run pytest, poetry run mypy, poetry add, poetry install). Never use pip directly or python -m for execution.
Implement ONEX Four-Node Architecture following the unidirectional workflow: EFFECT → COMPUTE → REDUCER → ORCHESTRATOR. Each node must inherit from appropriate base class (NodeEffect, NodeCompute, NodeReducer, NodeOrchestrator).
Always call super().init(container) in node constructors, where container is of type ModelONEXContainer. This eliminates boilerplate initialization code.
Use ModelONEXContainer for dependency injection in node constructors, never use ModelContainer[T]. Resolve services using protocol names only (e.g., container.get_service('ProtocolEventBus')), never by concrete class names.
Use ModelOnexError with EnumCoreErrorCode for all exception handling. Never raise generic Exception. Use the @standard_error_handling decorator to eliminate try/catch boilerplate.
Use PEP 604 union syntax (str | None) instead of Optional[str] and Union types. This is enforced by ruff UP007.
Prefix exception tuples with standardized markers (# fallback-ok:, # catch-all-ok:, # cleanup-resilience-ok:, # boundary-ok:, # init-errors-ok:, # tool-resilience-ok:) to document intent of broad exception handlers.
Order exception tuples alphabetically by exception class name (AttributeError, TypeError, ValidationError, ValueError).
Use centralized exception groups from omnibase_core.errors.exception_groups (PYDANTIC_MODEL_ERRORS, VALIDATION_ERRORS, FILE_IO_ERRORS, etc.) instead of manual exception tuple construction.
Never share node instances across threads. Use thread-local instances or separate instances per thread. Node instances (NodeCompute, NodeEffect) are not thread-safe by default.
Use duck typing with protocols - never use isinstance() checks. Rely on protocol interfaces for type checking and service resolution.
Load runtime contracts using FileRegistry from omnibase_core.runti...
Files:
src/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/services/replay/service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pytests/unit/services/replay/test_service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_set.py
src/omnibase_core/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/omnibase_core/**/*.py: All Python files in src/omnibase_core/ must follow directory-specific naming prefixes enforced by checker_naming_convention.py. Examples: cli_, container_, decorator_, enum_, error_, exception_, factory_, model_, node_, mixin_, protocol_, service_, util_, validator_, checker_, resolver_, etc.
Maintain 100% mypy strict mode compliance and ensure pyright passes in CI. Type annotations must be complete and correct.
Files:
src/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/services/replay/service_config_override_injector.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_set.py
src/omnibase_core/models/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/omnibase_core/models/**/*.py: Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.
Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.
Files:
src/omnibase_core/models/replay/__init__.pysrc/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_set.py
tests/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
Test with: poetry run pytest tests/ for all tests, poetry run pytest tests/unit/ for unit tests only, poetry run pytest tests/ -n 0 -xvs for debug mode. Use markers: @pytest.mark.unit, @pytest.mark.integration, @pytest.mark.slow, @pytest.mark.smoke, @pytest.mark.performance.
Files:
tests/unit/services/replay/test_service_config_override_injector.py
🧠 Learnings (2)
📚 Learning: 2026-01-04T19:09:33.280Z
Learnt from: CR
Repo: OmniNode-ai/omnibase_core PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-04T19:09:33.280Z
Learning: Applies to src/omnibase_core/models/**/*.py : Canonical Pydantic base class for omnibase_core is BaseModel from pydantic. Use ConfigDict for configuration. Ensure model_config is properly set.
Applied to files:
src/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_set.py
📚 Learning: 2026-01-04T19:09:33.280Z
Learnt from: CR
Repo: OmniNode-ai/omnibase_core PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-04T19:09:33.280Z
Learning: Applies to src/omnibase_core/models/**/*.py : Add ConfigDict(from_attributes=True) to Pydantic models used as value objects for pytest-xdist parallel execution to prevent class identity issues across worker processes.
Applied to files:
src/omnibase_core/models/replay/model_config_override.pysrc/omnibase_core/models/replay/model_config_override_field_preview.pysrc/omnibase_core/models/replay/model_config_override_preview.pysrc/omnibase_core/models/replay/model_config_override_set.py
🧬 Code graph analysis (7)
src/omnibase_core/models/replay/__init__.py (7)
src/omnibase_core/models/manifest/model_execution_manifest.py (1)
ModelExecutionManifest(37-327)src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(22-60)src/omnibase_core/models/replay/model_config_override_field_preview.py (1)
ModelConfigOverrideFieldPreview(22-104)src/omnibase_core/models/replay/model_config_override_preview.py (1)
ModelConfigOverridePreview(20-104)src/omnibase_core/models/replay/model_config_override_result.py (1)
ModelConfigOverrideResult(18-49)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(23-98)src/omnibase_core/models/replay/model_replay_input.py (1)
ModelReplayInput(24-80)
src/omnibase_core/models/replay/model_config_override.py (1)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)
src/omnibase_core/services/replay/service_config_override_injector.py (6)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override_field_preview.py (1)
ModelConfigOverrideFieldPreview(22-104)src/omnibase_core/models/replay/model_config_override_preview.py (1)
ModelConfigOverridePreview(20-104)src/omnibase_core/models/replay/model_config_override_result.py (1)
ModelConfigOverrideResult(18-49)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(23-98)src/omnibase_core/models/replay/model_config_override_validation.py (1)
ModelConfigOverrideValidation(75-134)
src/omnibase_core/models/replay/model_config_override_field_preview.py (2)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/core/model_extension_data.py (1)
value_type(89-105)
tests/unit/services/replay/test_service_config_override_injector.py (4)
src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(22-60)src/omnibase_core/models/replay/model_config_override_set.py (1)
ModelConfigOverrideSet(23-98)src/omnibase_core/services/replay/service_config_override_injector.py (4)
preview(588-686)apply(688-783)apply_environment_overlay(785-886)_is_frozen_pydantic_model(140-182)src/omnibase_core/models/replay/model_config_override_preview.py (1)
to_markdown(74-104)
src/omnibase_core/models/replay/model_config_override_preview.py (2)
src/omnibase_core/models/replay/model_config_override_field_preview.py (2)
ModelConfigOverrideFieldPreview(22-104)to_markdown_row(74-86)src/omnibase_core/services/replay/service_config_override_injector.py (1)
preview(588-686)
src/omnibase_core/models/replay/model_config_override_set.py (2)
src/omnibase_core/enums/replay/enum_override_injection_point.py (1)
EnumOverrideInjectionPoint(16-52)src/omnibase_core/models/replay/model_config_override.py (1)
ModelConfigOverride(22-60)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (21)
- GitHub Check: Tests (Split 6/20)
- GitHub Check: Tests (Split 2/20)
- GitHub Check: Tests (Split 4/20)
- GitHub Check: Tests (Split 8/20)
- GitHub Check: Tests (Split 20/20)
- GitHub Check: Tests (Split 16/20)
- GitHub Check: Tests (Split 15/20)
- GitHub Check: Tests (Split 12/20)
- GitHub Check: Tests (Split 11/20)
- GitHub Check: Tests (Split 14/20)
- GitHub Check: Tests (Split 19/20)
- GitHub Check: Tests (Split 18/20)
- GitHub Check: Tests (Split 5/20)
- GitHub Check: Tests (Split 13/20)
- GitHub Check: Tests (Split 3/20)
- GitHub Check: Tests (Split 10/20)
- GitHub Check: Tests (Split 9/20)
- GitHub Check: Tests (Split 17/20)
- GitHub Check: Tests (Split 7/20)
- GitHub Check: Tests (Split 1/20)
- GitHub Check: claude-review
🔇 Additional comments (20)
src/omnibase_core/models/replay/model_config_override.py (1)
22-60: Excellent immutable model design with clear documentation.The
ModelConfigOverridemodel follows all best practices:
- Frozen Pydantic model ensures thread-safety
from_attributes=Truedocumented for pytest-xdist compatibility- Clear field descriptions and validation (min_length=1 on path)
- Comprehensive docstring with usage example
The design correctly enforces immutability and provides a solid foundation for the override injection system.
Based on learnings and coding guidelines.
src/omnibase_core/models/replay/model_config_override_set.py (2)
44-74: Well-designed grouping property for efficient batch processing.The
by_injection_pointproperty efficiently groups overrides by injection point, enabling batch application. The implementation correctly usessetdefault()for clean aggregation.Note: The property returns a mutable
dict, which is acceptable here since it's a fresh computation each time and not exposing internal state.
76-98: Perfect immutable update pattern implementation.The
with_overridemethod correctly implements the immutable update pattern:
- Creates a new instance rather than modifying the existing one
- Uses tuple unpacking to append the new override
- Consistent with
frozen=TrueconfigurationThis ensures thread-safety and prevents accidental mutations.
tests/unit/services/replay/test_service_config_override_injector.py (4)
98-139: Excellent fixture design with documented pytest-xdist considerations.The Pydantic model fixtures (
SamplePydanticConfig,FrozenPydanticConfig,NestedPydanticConfig) correctly document the rationale forfrom_attributes=Truein the context of parallel test execution. This demonstrates awareness of pytest-xdist's worker isolation challenges.The fixture organization is clean and provides good test data coverage.
Based on coding guidelines for pytest-xdist compatibility.
602-644: Robust immutability verification for environment overlays.The test correctly verifies that
apply_environment_overlayreturns an immutableMappingProxyType:
- Confirms the return type
- Tests that mutation raises
TypeError- Verifies that
dict()can create a mutable copy for subprocess useThis is excellent defensive testing that ensures callers cannot accidentally mutate the overlay.
860-908: Comprehensive thread-safety verification with realistic concurrency.The concurrent apply test uses 20 parallel operations with a
ThreadPoolExecutorto verify:
- No race conditions occur
- Original config remains unchanged
- Each operation produces independent results
This provides strong evidence of thread-safety. The test design is realistic and thorough.
830-849: Testing implementation detail is justified here.The test directly accesses
_is_frozen_pydantic_model, a private method. While normally testing implementation details should be avoided, this behavior is critical for correctness (frozen models must be detected and converted to dicts).This is an acceptable exception given the importance of the frozen model detection logic.
src/omnibase_core/models/replay/model_config_override_preview.py (1)
74-104: Clean markdown generation with smart conditional rendering.The
to_markdown()method produces well-structured output:
- Clear table format with headers
- Delegates row formatting to
ModelConfigOverrideFieldPreview.to_markdown_row()- Conditionally renders "New Paths" and "Type Mismatches" sections only when needed
This provides users with a clear, actionable preview before applying overrides.
src/omnibase_core/models/replay/__init__.py (1)
68-111: Well-organized package exports with clear categorization.The module exports are properly structured:
- New configuration override models grouped together (lines 93-99)
- Clear separation between model categories (configuration, corpus, replay)
- All imports match their corresponding modules
__all__list is complete and organizedThe documentation additions (lines 9-14, 58-60) clearly indicate the new functionality added in OMN-1205.
src/omnibase_core/models/replay/model_config_override_field_preview.py (1)
1-104: Well-structured Pydantic model following project conventions.The model correctly implements:
ConfigDict(frozen=True, extra="forbid", from_attributes=True)per coding guidelines for value objects- Clear docstrings with usage examples
__all__export (addressing previous feedback)- Thread-safety through immutability
The
_format_valuehelper appropriately handlesNone, strings (with quotes), and other types for markdown display. Based on learnings,from_attributes=Trueis correctly applied for pytest-xdist compatibility.src/omnibase_core/services/replay/service_config_override_injector.py (10)
1-138: Well-documented module with clear architectural design.The module docstring provides comprehensive documentation including:
- Thread-safety guarantees
- Three injection points architecture
- Complete usage examples
- Related ticket references
The class setup with
MAX_PATH_DEPTH = 20appropriately guards against DoS via deeply nested paths.
140-182: Correct frozen model detection logic.The method properly handles both dict-style and
ConfigDict(Mapping) configurations. The type checks are ordered correctly withdictbeforeMapping.
184-237: Path validation is thorough and secure.The method correctly:
- Guards against empty/whitespace paths
- Enforces depth limit (DoS protection)
- Validates identifier format with proper regex
- Allows numeric segments for array indexing
The ASCII-only identifier restriction (
^[a-zA-Z_][a-zA-Z0-9_]*$) is appropriate for configuration paths.
239-303: Robust path traversal with proper sentinel usage.The method correctly:
- Returns
MISSINGsentinel to distinguish "path not found" from "value is None"- Handles dict, Pydantic models, and list/tuple with index access
- Uses alphabetically ordered exception tuple with
# boundary-ok:marker
305-421: Path setting logic handles edge cases well.The method appropriately:
- Documents the in-place modification warning
- Creates intermediate dictionaries for nested paths
- Provides clear error messages for frozen Pydantic models
- Handles list index bounds checking
- Uses alphabetically ordered exception tuples with proper markers
423-460: Deep copy strategy correctly handles frozen models.The method properly:
- Detects frozen Pydantic models and converts to dict (required for mutability)
- Uses
model_copy(deep=True)for mutable Pydantic models (more efficient)- Falls back to
copy.deepcopyfor other typesThe
copy.deepcopy(config.model_dump())is correct sincemodel_dump()produces a shallow dict.
462-586: Comprehensive validation with clear severity levels.The method correctly implements the three-tier validation pattern (violations/warnings/suggestions):
- Violations: Invalid path syntax
- Warnings: Non-existent paths, type mismatches
- Suggestions: New environment variables, setting to None
The
strict_modeintegration and numeric type conversion allowance are well-designed.
588-686: Preview generation is pure and comprehensive.The method correctly:
- Handles
Noneinputs gracefully- Filters by injection point when specified
- Tracks both
paths_not_foundandtype_mismatchesfor user review- Uses
Noneforold_valuedisplay when path doesn't exist (appropriate for UI)
688-783: Apply method maintains immutability invariant.The method correctly:
- Deep copies config before any modification (thread-safety guarantee)
- Returns structured result with success/failure, applied count, created paths, and errors
- Handles
Noneinputs with proper error responses- Never mutates the original config
888-889: Appropriate public API exports.The
__all__correctly exports the service class andMISSINGsentinel for external use.
Summary
Add
ServiceConfigOverrideInjectorto inject configuration overrides during replay, enabling A/B testing of different models, parameters, or endpoints while keeping inputs identical.Key Features:
ModelConfigOverrideandModelConfigOverrideSetfor override specificationsModelReplayInput[T]to bundle inputs with optional overridesvalidate()→preview()→apply()os.environ)llm.openai.temperature)Non-negotiable invariants enforced:
Structural Fixes (from OMN-1116):
RecorderEffect,InjectorTime,InjectorRNGfrompipeline/replay/toservices/replay/Test Plan
Files Changed
New Files (OMN-1205):
src/omnibase_core/enums/replay/enum_override_injection_point.pysrc/omnibase_core/models/replay/model_config_override*.py(5 files)src/omnibase_core/models/replay/model_replay_input.pysrc/omnibase_core/services/replay/service_config_override_injector.pysrc/omnibase_core/services/replay/sentinel_missing.pytests/unit/services/replay/test_service_config_override_injector.pytests/unit/models/replay/test_model_config_overrides.pytests/unit/models/replay/test_model_replay_input.pyRelocated Files (OMN-1116 fix):
pipeline/replay/recorder_effect.py→services/replay/recorder_effect.pypipeline/replay/injector_time.py→services/replay/injector_time.pypipeline/replay/injector_rng.py→services/replay/injector_rng.pyModified Files:
scripts/validation/validate_naming.py(architectural exemption)*/__init__.pyfiles for exportsSummary by CodeRabbit
New Features
Refactor
Tests
✏️ Tip: You can customize this high-level summary in your review settings.