diff --git a/docs/INDEX.md b/docs/INDEX.md index 3b04eb04b3..c56d66e668 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -18,7 +18,7 @@ ## 📚 Documentation Structure -``` +```text omnibase_core/docs/ │ ├── Getting Started → New developer onboarding diff --git a/docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md b/docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md new file mode 100644 index 0000000000..9992d1505b --- /dev/null +++ b/docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md @@ -0,0 +1,1020 @@ +# Contract-Driven NodeCompute v1.0 Specification + +> **Version**: 1.0.0 +> **Date**: 2025-12-07 +> **Status**: DRAFT - Ready for Implementation +> **Ticket**: [OMN-465](https://linear.app/omninode/issue/OMN-465) +> **Full Roadmap**: [NODECOMPUTE_VERSIONING_ROADMAP.md](./NODECOMPUTE_VERSIONING_ROADMAP.md) + +--- + +## Executive Summary + +This document defines the **minimal v1.0 implementation** of contract-driven `NodeCompute`. The goal is a boring, stable foundation that can be shipped safely and extended incrementally. + +**v1.0 Scope**: Sequential-only pipelines with 6 built-in transformation types, simple mapping, and abort-on-first-failure semantics. No caching, no parallelism, no conditionals. + +--- + +## Table of Contents + +1. [Design Principles](#design-principles) +2. [v1.0 Scope](#v10-scope) +3. [Core Models](#core-models) +4. [Enums](#enums) +5. [Transformation Config Models](#transformation-config-models) +6. [Execution Model](#execution-model) +7. [Immutability Guarantees](#immutability-guarantees-v10) +8. [Error Model](#error-model-v10) +9. [NodeCompute Behavior](#v10-nodecompute-behavior) +10. [Schema Resolution](#schema-resolution) +11. [Example Contract](#example-contract) +12. [Implementation Plan](#implementation-plan) +13. [Acceptance Criteria](#acceptance-criteria) + +--- + +## Design Principles + +These principles apply to v1.0 and all future versions: + +1. **Zero Custom Code**: Developers inherit from `NodeCompute` without writing transformation logic +2. **Pipeline-Driven**: Computations are sequences of named steps +3. **Transformation Registry**: Built-in transformations only (no custom registration) +4. **Pure Functions**: All transformation and execution logic in utility module for testability +5. **Typed Boundaries**: All public surfaces use Pydantic models +6. **Deterministic**: Same input always produces same output + +### Thread Safety and State Management + +The core pipeline executor and transformation functions are designed to be **pure and stateless**: + +- All transformation functions (`transform_*`) operate only on their input parameters +- Each `execute_compute_pipeline()` invocation operates on its own data without shared state +- Safe for concurrent use from multiple threads + +**Known Limitation (v1.0)**: The validation step warning deduplication uses module-level state (`_validation_warning_emitted`) to prevent log spam. This is a minor exception to the pure function principle, affecting only warning output (not computation results). The pipeline execution itself remains deterministic and thread-safe. + +**TODO(v1.1)**: Refactor warning deduplication to use a context-based approach (e.g., per-execution context) rather than global state. See `compute_executor.py` for details. + +--- + +## v1.0 Scope + +### What's IN v1.0 + +| Feature | Description | +|---------|-------------| +| **Sequential Pipelines** | Steps execute in declaration order | +| **3 Step Types** | `VALIDATION`, `TRANSFORMATION`, `MAPPING` | +| **6 Transformation Types** | `IDENTITY`, `REGEX`, `CASE_CONVERSION`, `TRIM`, `NORMALIZE_UNICODE`, `JSON_PATH` | +| **Simple Mapping** | JSONPath-like selectors: `$.input`, `$.steps..output` | +| **Abort on First Failure** | Pipeline stops at first error | +| **Schema References** | `input_schema_ref` / `output_schema_ref` resolved at load time | +| **Pipeline Timeout** | Single `pipeline_timeout_ms` setting | + +### What's NOT in v1.0 + +| Feature | Deferred To | Rationale | +|---------|-------------|-----------| +| **Conditional Steps** | v1.1 | Expression language needs full spec | +| **Parallel Steps** | v1.1 | Merge semantics need careful design | +| **SPLIT/JOIN/TEMPLATE/TYPE_CONVERSION/SCHEMA_VALIDATE** | v1.1 | Start with minimal set; `SCHEMA_VALIDATE` transform duplicates `VALIDATION` step type | +| **Caching** | v1.2 | Adds cache invalidation complexity | +| **Detailed Error Enums** | v1.2 | Simple string `error_type` sufficient for v1.0 | +| **Full Expression Language** | v1.1/v1.3 | Basic expressions (equality, logical) in v1.1; arithmetic and functions in v1.3 | +| **FILTER/MAP/REDUCE/SORT** | v1.1 (optional) | Collection operations added only if production feedback demonstrates need | +| **Cancellation** | v1.3 | Requires cooperative cancellation infrastructure | +| **Transformation Versioning** | v1.3 | Not needed until transformations evolve | + +See [NODECOMPUTE_VERSIONING_ROADMAP.md](./NODECOMPUTE_VERSIONING_ROADMAP.md) for full roadmap. + +--- + +## Core Models + +### ModelComputeSubcontract + +```python +from pydantic import BaseModel, ConfigDict, Field +from typing import Literal + +class ModelComputeSubcontract(BaseModel): + """ + v1.0 Compute subcontract for sequential pipeline transformations. + + Defines transformation pipelines with abort-on-first-failure semantics. + """ + + # Identity + version: str = "1.0.0" + operation_name: str + operation_version: str + description: str = "" + + # Schema references (resolved at load time) + input_schema_ref: str | None = None + output_schema_ref: str | None = None + + # Pipeline definition + pipeline: list["ModelComputePipelineStep"] + + # v1.0 Performance (minimal) + pipeline_timeout_ms: int | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelComputePipelineStep + +```python +from pydantic import BaseModel, ConfigDict, Field, model_validator +from typing import Literal, Annotated + +class ModelComputePipelineStep(BaseModel): + """ + A single step in the compute pipeline. + + v1.0 supports: VALIDATION, TRANSFORMATION, MAPPING + + NOTE: No per-step timeout in v1.0. Use pipeline_timeout_ms on contract. + """ + + step_name: str + step_type: "EnumComputeStepType" + + # For transformation steps + transformation_type: "EnumTransformationType | None" = None + transformation_config: "ModelTransformationConfig | None" = None + + # For mapping steps + mapping_config: "ModelMappingConfig | None" = None + + # For validation steps + validation_config: "ModelValidationStepConfig | None" = None + + # Common options + enabled: bool = True + # v1.0: No per-step timeout - only pipeline-level timeout_ms on contract + + model_config = ConfigDict(extra="forbid", frozen=True) + + @model_validator(mode="after") + def validate_step_config(self) -> "ModelComputePipelineStep": + """Ensure correct config is provided for step type.""" + if self.step_type == EnumComputeStepType.TRANSFORMATION: + if self.transformation_type is None: + raise ValueError("transformation_type required for transformation steps") + if self.transformation_config is None and self.transformation_type != EnumTransformationType.IDENTITY: + raise ValueError("transformation_config required for non-identity transformations") + if self.step_type == EnumComputeStepType.MAPPING: + if self.mapping_config is None: + raise ValueError("mapping_config required for mapping steps") + if self.step_type == EnumComputeStepType.VALIDATION: + if self.validation_config is None: + raise ValueError("validation_config required for validation steps") + return self +``` + +### ModelComputeExecutionContext + +```python +from pydantic import BaseModel, ConfigDict, Field +from uuid import UUID + +class ModelComputeExecutionContext(BaseModel): + """ + Typed execution context for compute pipelines. + + v1.0: Minimal context for deterministic execution. + """ + + operation_id: UUID + correlation_id: UUID | None = None + node_id: str | None = None + + model_config = ConfigDict(frozen=True) +``` + +### ModelComputeStepMetadata + +```python +from pydantic import BaseModel, ConfigDict, Field + +class ModelComputeStepMetadata(BaseModel): + """Metadata for a single pipeline step execution.""" + + duration_ms: float + transformation_type: str | None = None + + model_config = ConfigDict(frozen=True) +``` + +### ModelComputeStepResult + +```python +from pydantic import BaseModel, ConfigDict, Field +from typing import Any + +class ModelComputeStepResult(BaseModel): + """Result of a single pipeline step.""" + + step_name: str + output: Any + success: bool = True + metadata: "ModelComputeStepMetadata" + error_type: str | None = None # v1.0: Simple string, not enum + error_message: str | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelComputePipelineResult + +```python +from pydantic import BaseModel, ConfigDict, Field +from typing import Any + +class ModelComputePipelineResult(BaseModel): + """Result of compute pipeline execution.""" + + success: bool + output: Any + processing_time_ms: float + steps_executed: list[str] + step_results: dict[str, "ModelComputeStepResult"] + error_type: str | None = None # v1.0: Simple string + error_message: str | None = None + error_step: str | None = None + + model_config = ConfigDict(frozen=True) +``` + +### ModelMappingConfig + +```python +from pydantic import BaseModel, ConfigDict, Field + +class ModelMappingConfig(BaseModel): + """ + v1.0 Mapping configuration. + + field_mappings uses simple path expressions: + - $.input - Access input data + - $.steps..output - Access step output + """ + + config_type: Literal["mapping"] = "mapping" + field_mappings: dict[str, str] + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +--- + +## Enums + +### EnumComputeStepType + +```python +from enum import Enum + +class EnumComputeStepType(str, Enum): + """ + v1.0 Pipeline step types. + + Only 3 types for v1.0 - CONDITIONAL and PARALLEL deferred. + """ + + VALIDATION = "validation" + TRANSFORMATION = "transformation" + MAPPING = "mapping" + # v1.1+: CONDITIONAL = "conditional" + # v1.1+: PARALLEL = "parallel" +``` + +### EnumTransformationType + +```python +from enum import Enum + +class EnumTransformationType(str, Enum): + """ + v1.0 Transformation types. + + Only 6 types for v1.0 - collection operations and SCHEMA_VALIDATE deferred. + + NOTE: Schema validation is handled by VALIDATION step type, not as a transformation. + This keeps step types semantically clean: + - VALIDATION → schema checks + - TRANSFORMATION → pure data transforms + - MAPPING → shape results + """ + + # v1.0 Types (6 transformations) + IDENTITY = "identity" + REGEX = "regex" + CASE_CONVERSION = "case_conversion" + TRIM = "trim" + NORMALIZE_UNICODE = "normalize_unicode" + JSON_PATH = "json_path" + + # v1.1+: SPLIT, JOIN, TEMPLATE, TYPE_CONVERSION, SCHEMA_VALIDATE + # v1.1+ (optional): FILTER, MAP, REDUCE, SORT +``` + +### Supporting Enums + +```python +from enum import Enum + +class EnumCaseMode(str, Enum): + """Case transformation modes.""" + UPPER = "uppercase" + LOWER = "lowercase" + TITLE = "titlecase" + + +class EnumRegexFlag(str, Enum): + """Supported regex flags.""" + IGNORECASE = "IGNORECASE" + MULTILINE = "MULTILINE" + DOTALL = "DOTALL" + + +class EnumUnicodeForm(str, Enum): + """Unicode normalization forms.""" + NFC = "NFC" + NFD = "NFD" + NFKC = "NFKC" + NFKD = "NFKD" + + +class EnumTrimMode(str, Enum): + """Whitespace trim modes.""" + BOTH = "both" + LEFT = "left" + RIGHT = "right" +``` + +--- + +## Transformation Config Models + +All config models have: + +- `config_type` literal field for discriminated union +- `model_config = ConfigDict(extra="forbid", frozen=True)` + +### ModelTransformRegexConfig + +```python +from pydantic import BaseModel, ConfigDict, Field +from typing import Literal + +class ModelTransformRegexConfig(BaseModel): + """Configuration for regex transformation.""" + + config_type: Literal["regex"] = "regex" + pattern: str + replacement: str = "" + flags: list[EnumRegexFlag] = Field(default_factory=list) + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelTransformCaseConfig + +```python +from pydantic import BaseModel, ConfigDict +from typing import Literal + +class ModelTransformCaseConfig(BaseModel): + """Configuration for case transformation.""" + + config_type: Literal["case_conversion"] = "case_conversion" + mode: EnumCaseMode + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelTransformTrimConfig + +```python +from pydantic import BaseModel, ConfigDict +from typing import Literal + +class ModelTransformTrimConfig(BaseModel): + """Configuration for whitespace trimming.""" + + config_type: Literal["trim"] = "trim" + mode: EnumTrimMode = EnumTrimMode.BOTH + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelTransformUnicodeConfig + +```python +from pydantic import BaseModel, ConfigDict +from typing import Literal + +class ModelTransformUnicodeConfig(BaseModel): + """Configuration for unicode normalization.""" + + config_type: Literal["normalize_unicode"] = "normalize_unicode" + form: EnumUnicodeForm = EnumUnicodeForm.NFC + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelTransformJsonPathConfig + +```python +from pydantic import BaseModel, ConfigDict +from typing import Literal + +class ModelTransformJsonPathConfig(BaseModel): + """Configuration for JSONPath extraction.""" + + config_type: Literal["json_path"] = "json_path" + path: str + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelValidationStepConfig + +```python +from pydantic import BaseModel, ConfigDict +from typing import Literal + +class ModelValidationStepConfig(BaseModel): + """ + Configuration for VALIDATION step type. + + NOTE: This is NOT a transformation config. It configures the VALIDATION step type. + Schema validation is semantically distinct from data transformation. + """ + + config_type: Literal["validation"] = "validation" + schema_ref: str + fail_on_error: bool = True + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +### ModelTransformationConfig (Union) + +```python +from typing import Annotated, Union +from pydantic import Field + +# v1.0 Discriminated union - only 5 types (IDENTITY has no config) +ModelTransformationConfig = Annotated[ + Union[ + ModelTransformRegexConfig, + ModelTransformCaseConfig, + ModelTransformTrimConfig, + ModelTransformUnicodeConfig, + ModelTransformJsonPathConfig, + # IDENTITY has no config - handled separately + # VALIDATION uses ModelValidationStepConfig on step, not here + ], + Field(discriminator="config_type"), +] +``` + +--- + +## Execution Model + +### Sequential Execution + +v1.0 uses **strictly sequential execution**: + +```text +Step 1 → Step 2 → Step 3 → ... → Step N → Output + ↓ ↓ ↓ + Result Result Result +``` + +**Rules**: + +1. Steps execute in declaration order +2. Each step receives output from previous step (or input for first step) +3. Step results accumulate in `step_results` dict +4. Mapping steps can reference any prior step result + +### Abort on First Failure + +When any step fails: + +1. Current step is marked as failed +2. **No subsequent steps execute** +3. Pipeline result is marked as failed +4. All executed step results are preserved +5. Error details recorded in `error_type`, `error_message`, `error_step` + +```text +Step 1: OK → Result preserved +Step 2: FAIL → Error recorded, pipeline aborts +Step 3: SKIPPED → Not executed +Step 4: SKIPPED → Not executed + +Result: FAIL +``` + +### Transformation Execution + +```python +async def execute_transformation( + data: Any, + transformation_type: EnumTransformationType, + config: ModelTransformationConfig | None, +) -> Any: + """ + Execute a single transformation. + + Pure function: (data, type, config) → result + + Raises: + ModelOnexError on transformation failure + """ + if transformation_type == EnumTransformationType.IDENTITY: + return data + + handler = TRANSFORMATION_REGISTRY.get(transformation_type) + if handler is None: + raise ModelOnexError(f"Unknown transformation type: {transformation_type}") + + return handler(data, config) +``` + +### Mapping Execution + +v1.0 mapping uses simple path expressions: + +```yaml +field_mappings: + result: "$.steps.transform_case.output" + original: "$.input.text" +``` + +### v1.0 Mapping Path Grammar (Intentionally Minimal) + +v1.0 does **NOT** implement full JSONPath. This is intentional. + +**Supported Forms**: + +| Pattern | Description | Example | +|---------|-------------|---------| +| `$.input` | Full input object (canonical) | `$.input` | +| `$input` | **Alias** for `$.input` (shorthand) | `$input` | +| `$.input.` | Direct child field | `$.input.text` | +| `$input.` | **Alias** for `$.input.` | `$input.text` | +| `$.input..` | Nested fields | `$.input.options.mode` | +| `$.steps.` | Step output (shorthand) | `$.steps.trim` | +| `$.steps..output` | Step output (explicit) | `$.steps.trim.output` | + +> **Note**: The `$input` alias (without the dot after `$`) is fully equivalent to `$.input`. Both forms work identically for accessing input data and its nested fields. Use whichever form your team prefers for consistency. + +#### Path Aliases and Equivalences + +**Input Access Alias**: +- `$input` is an alias for `$.input` +- Both forms are semantically equivalent and access the full input object +- The canonical form is `$.input`, but `$input` is accepted for brevity + +**Step Output Access**: +- `$.steps.` and `$.steps..output` are **equivalent** +- Both return the step's output value +- The shorthand form `$.steps.` is the more common usage pattern +- The explicit `.output` suffix is supported for clarity and forward compatibility + +**Forward Compatibility Note**: In v1.1+, step results may expose additional fields such as `.metadata` or `.duration_ms`. Using the explicit `.output` suffix future-proofs contracts against these additions. + +#### Path Resolution Examples + +```yaml +# Example: Multi-step pipeline with path references +pipeline: + - step_name: validate_input + step_type: validation + validation_config: + schema_ref: input_schema + + - step_name: trim_text + step_type: transformation + transformation_type: trim + transformation_config: + mode: both + + - step_name: normalize_case + step_type: transformation + transformation_type: case_conversion + transformation_config: + mode: lowercase + + - step_name: build_result + step_type: mapping + mapping_config: + field_mappings: + # These are equivalent - both access the step's output: + normalized: "$.steps.normalize_case" # Shorthand form + normalized_explicit: "$.steps.normalize_case.output" # Explicit form + + # These are equivalent - both access the input: + original: "$.input.text" # Canonical form + original_alt: "$input.text" # Shorthand alias (equally valid) + + # Nested input access: + options_mode: "$.input.options.mode" +``` + +**Best Practice**: Use `$.input` (canonical) and `$.steps.` (shorthand) for consistency and readability. + +**NOT Supported in v1.0**: + +| Pattern | Why Not | +|---------|---------| +| `$.input.items[0]` | Array indexing requires expression language | +| `$.input.items[*].value` | Wildcards require expression language | +| `$.input[?(@.active)]` | Filters require expression language | +| `$.input.a + $.input.b` | Arithmetic requires expression language | +| `len($.input.items)` | Functions require expression language | + +**Validation**: Any unsupported path pattern is a **contract validation error** at load time. + +**Resolution**: + +The actual implementation uses `resolve_pipeline_path()` from `compute_path_resolver.py`: + +```python +def resolve_pipeline_path( + path: str, + input_data: Any, + step_results: dict[str, Any], +) -> Any: + """ + Main entry point for resolving paths in pipeline mapping steps. + Dispatches to resolve_input_path() or resolve_step_path() based on prefix. + """ + _validate_path_start(path) # Ensures path starts with "$" + + # Handle input paths (including $input alias) + if path in ("$.input", "$input") or path.startswith(("$.input.", "$input.")): + return resolve_input_path(path, input_data) + + # Handle step paths + if path.startswith("$.steps."): + return resolve_step_path(path, step_results) + + # Invalid prefix + raise PathResolutionError( + message=f"Invalid path prefix: '{path}'. Must be '$.input', '$input', or '$.steps.'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + ) +``` + +See `src/omnibase_core/utils/compute_path_resolver.py` for the complete implementation including `resolve_input_path()` and `resolve_step_path()` helper functions. + +### IDENTITY Transformation Rules + +The `IDENTITY` transformation is special - it has **no config**: + +- `transformation_config` **must** be omitted or `null` +- The transformation simply returns the input data unchanged +- Useful as a no-op placeholder or for explicit passthrough steps + +**Example**: + +```yaml +- step_name: passthrough + step_type: transformation + transformation_type: identity + # NO transformation_config - must be omitted +``` + +**Validation**: If `transformation_type` is `IDENTITY` and `transformation_config` is provided, contract validation fails. + +--- + +## Immutability Guarantees (v1.0) + +All pipeline artifacts are immutable after creation: + +| Model | Immutable? | Enforcement | +|-------|------------|-------------| +| `ModelComputeExecutionContext` | ✅ Yes | `frozen=True` | +| `ModelComputeStepMetadata` | ✅ Yes | `frozen=True` | +| `ModelComputeStepResult` | ✅ Yes | `frozen=True` | +| `ModelComputePipelineResult` | ✅ Yes | `frozen=True` | + +The executor may maintain internal mutable structures while running, but anything exposed outside the executor is frozen. This prevents: + +- Accidental mutation between steps +- Mutation after pipeline completion +- Race conditions in future parallel execution (v1.2+) + +This sets up v1.2's replay/observability guarantees cleanly. + +--- + +## Error Model (v1.0) + +v1.0 uses **string-only error types**: + +```python +error_type: str | None = None # e.g., "validation_failed", "compute_error", "unexpected_error" +error_message: str | None = None +error_step: str | None = None +``` + +**Why not enums in v1.0?** + +- Enums require defining a complete taxonomy upfront +- v1.0 focuses on getting the pipeline mechanics right +- String errors are sufficient for debugging + +**v1.1+ Upgrade Path**: + +- `EnumComputeErrorType` will be introduced in v1.1 +- `ModelComputeError` structured error model in v1.2 +- `ModelParallelStepError` for parallel step aggregation in v1.2 + +Existing v1.0 contracts will continue to work - error type becomes an enum value instead of free-form string. + +--- + +## v1.0 NodeCompute Behavior + +### Startup Behavior + +When a `NodeCompute` instance starts: + +1. **Load Contract**: Read `compute_operations` from node contract +2. **Validate Structure**: Validate `ModelComputeSubcontract` with `extra="forbid"` +3. **Resolve Schemas**: Resolve `input_schema_ref` / `output_schema_ref` from registry +4. **Fail Fast**: If contract or schemas are invalid, node **must not start** + +### Runtime Behavior + +When `process(ModelComputeInput)` is called: + +1. **Build Context**: Create `ModelComputeExecutionContext` with operation_id, correlation_id +2. **Execute Pipeline**: Call `MixinComputeExecution.execute_compute_pipeline()` +3. **Check Timeout**: Abort if `pipeline_timeout_ms` exceeded +4. **Build Output**: Convert `ModelComputePipelineResult` to `ModelComputeOutput` +5. **Return**: Return typed output to caller + +### Lifecycle + +```text +┌─────────────────┐ +│ Contract Load │──▶ Validates structure, resolves schemas +└────────┬────────┘ + │ + ▼ +┌─────────────────┐ +│ Node Ready │──▶ Contract frozen, schemas cached +└────────┬────────┘ + │ + ▼ +┌─────────────────┐ +│ process() │──▶ Builds context, executes pipeline, returns result +└─────────────────┘ +``` + +--- + +## Schema Resolution + +### Resolution Rules + +1. `input_schema_ref` and `output_schema_ref` are resolved at **contract load time** +2. If schema reference cannot be resolved → **contract validation failure** +3. NodeCompute instance **must not start** if schema resolution fails +4. No version ranges - exact match required + +### Schema Reference Format + +```yaml +input_schema_ref: text_processor_input_v1 +output_schema_ref: text_processor_output_v1 +``` + +Format: `{name}_v{major}` (e.g., `text_processor_input_v1`) + +### Validation Step Schema Resolution + +Within validation steps, `schema_ref` can use aliases: + +- `input_schema` → Resolves to `input_schema_ref` +- `output_schema` → Resolves to `output_schema_ref` +- Otherwise → Resolve from global schema registry + +--- + +## Example Contract + +### Simple Text Processor + +```yaml +# examples/contracts/compute/text_processor_v1.yaml +node_type: COMPUTE +node_name: text_processor +node_version: "1.0.0" + +compute_operations: + version: "1.0.0" + operation_name: text_normalization + operation_version: "1.0.0" + description: "Normalize and clean text data" + + input_schema_ref: text_input_v1 + output_schema_ref: text_output_v1 + + pipeline_timeout_ms: 5000 + + pipeline: + # Step 1: Validate input (uses ModelValidationStepConfig, NOT a transformation) + - step_name: validate_input + step_type: validation + validation_config: + config_type: validation + schema_ref: input_schema + fail_on_error: true + + # Step 2: Normalize unicode + - step_name: normalize_unicode + step_type: transformation + transformation_type: normalize_unicode + transformation_config: + config_type: normalize_unicode + form: NFC + + # Step 3: Trim whitespace + - step_name: trim_whitespace + step_type: transformation + transformation_type: trim + transformation_config: + config_type: trim + mode: both + + # Step 4: Collapse multiple spaces + - step_name: collapse_spaces + step_type: transformation + transformation_type: regex + transformation_config: + config_type: regex + pattern: "\\s+" + replacement: " " + + # Step 5: Convert to lowercase + - step_name: to_lowercase + step_type: transformation + transformation_type: case_conversion + transformation_config: + config_type: case_conversion + mode: lowercase + + # Step 6: Identity passthrough (demonstrates no-config transformation) + - step_name: passthrough + step_type: transformation + transformation_type: identity + # NO transformation_config for identity + + # Step 7: Map output + - step_name: build_output + step_type: mapping + mapping_config: + config_type: mapping + field_mappings: + normalized_text: "$.steps.passthrough.output" + original_length: "$.input.length" +``` + +### Usage + +```python +from omnibase_core.nodes import NodeCompute + +class NodeTextProcessor(NodeCompute): + """Text processor - all logic from contract.""" + pass +``` + +--- + +## Implementation Plan + +### Phase 1: Core Models & Enums (~3 days) + +| Task | File | Priority | +|------|------|----------| +| EnumComputeStepType | `enums/enum_compute_step_type.py` | P0 | +| EnumTransformationType | `enums/enum_transformation_type.py` | P0 | +| EnumCaseMode | `enums/enum_case_mode.py` | P0 | +| EnumRegexFlag | `enums/enum_regex_flag.py` | P0 | +| EnumUnicodeForm | `enums/enum_unicode_form.py` | P0 | +| EnumTrimMode | `enums/enum_trim_mode.py` | P0 | +| ModelComputeExecutionContext | `models/compute/model_compute_execution_context.py` | P0 | +| ModelComputeStepMetadata | `models/compute/model_compute_step_metadata.py` | P0 | +| ModelComputeStepResult | `models/compute/model_compute_step_result.py` | P0 | +| ModelComputePipelineResult | `models/compute/model_compute_pipeline_result.py` | P0 | + +### Phase 2: Config Models (~2 days) + +| Task | File | Priority | +|------|------|----------| +| ModelTransformRegexConfig | `models/transformations/model_transform_regex_config.py` | P0 | +| ModelTransformCaseConfig | `models/transformations/model_transform_case_config.py` | P0 | +| ModelTransformTrimConfig | `models/transformations/model_transform_trim_config.py` | P0 | +| ModelTransformUnicodeConfig | `models/transformations/model_transform_unicode_config.py` | P0 | +| ModelTransformJsonPathConfig | `models/transformations/model_transform_json_path_config.py` | P0 | +| ModelValidationStepConfig | `models/transformations/model_validation_step_config.py` | P0 | +| ModelMappingConfig | `models/transformations/model_mapping_config.py` | P0 | +| ModelTransformationConfig | `models/transformations/types.py` | P0 | + +### Phase 3: Contract & Execution (~3 days) + +| Task | File | Priority | +|------|------|----------| +| ModelComputePipelineStep | `models/contracts/subcontracts/model_compute_pipeline_step.py` | P0 | +| ModelComputeSubcontract | `models/contracts/subcontracts/model_compute_subcontract.py` | P0 | +| compute_executor.py | `utils/compute_executor.py` | P0 | +| compute_transformations.py | `utils/compute_transformations.py` | P0 | +| MixinComputeExecution | `mixins/mixin_compute_execution.py` | P0 | +| Refactor NodeCompute | `nodes/node_compute.py` | P0 | + +### Phase 4: Testing (~2 days) + +| Task | File | Priority | +|------|------|----------| +| Unit tests for enums | `tests/unit/enums/test_compute_*.py` | P0 | +| Unit tests for models | `tests/unit/models/compute/test_*.py` | P0 | +| Unit tests for executor | `tests/unit/utils/test_compute_executor.py` | P0 | +| Unit tests for transformations | `tests/unit/utils/test_compute_transformations.py` | P0 | +| Integration tests | `tests/integration/test_compute_pipeline.py` | P0 | + +### Total Estimate + +- **Files**: ~25 files +- **Code**: ~800 lines +- **Tests**: ~600 lines +- **Timeline**: 10 working days + +--- + +## Acceptance Criteria + +### Functional Requirements + +- [ ] `ModelComputeSubcontract` validates contracts with `extra="forbid"` +- [ ] All 6 transformation types implemented and tested (IDENTITY, REGEX, CASE_CONVERSION, TRIM, NORMALIZE_UNICODE, JSON_PATH) +- [ ] VALIDATION step type uses separate `ModelValidationStepConfig` (not a transformation) +- [ ] Pipeline executes steps sequentially +- [ ] Pipeline aborts on first failure +- [ ] Mapping resolves `$.input` and `$.steps..output` paths +- [ ] Schema references validated at contract load time +- [ ] Pipeline timeout enforced + +### Type Safety Requirements + +- [ ] All config models have `config_type` discriminator field +- [ ] All models use `ConfigDict(extra="forbid", frozen=True)` where appropriate +- [ ] `ModelTransformationConfig` is proper discriminated union +- [ ] mypy --strict passes with zero errors + +### Testing Requirements + +- [ ] Unit tests for each enum +- [ ] Unit tests for each model +- [ ] Unit tests for each transformation type +- [ ] Integration test with example contract +- [ ] 90%+ code coverage + +### Documentation Requirements + +- [ ] Example contract in `examples/contracts/compute/` +- [ ] API documentation for public functions +- [ ] Migration guide from code-driven NodeCompute + +--- + +## References + + +- **Versioning Roadmap**: [NODECOMPUTE_VERSIONING_ROADMAP.md](./NODECOMPUTE_VERSIONING_ROADMAP.md) +- **Example Contract**: [user_profile_normalizer.yaml](../../examples/contracts/compute/user_profile_normalizer.yaml) +- **Linear Issue**: [OMN-465](https://linear.app/omninode/issue/OMN-465) +- **NodeReducer Pattern**: [node_reducer.py](../../src/omnibase_core/nodes/node_reducer.py) +- **NodeOrchestrator Pattern**: [node_orchestrator.py](../../src/omnibase_core/nodes/node_orchestrator.py) + +--- + +**Last Updated**: 2025-12-07 +**Version**: 1.0.0 +**Status**: DRAFT - Ready for Implementation diff --git a/docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md b/docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md new file mode 100644 index 0000000000..478ec78b7e --- /dev/null +++ b/docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md @@ -0,0 +1,997 @@ +# NodeCompute Versioning Roadmap + +> **Version**: 1.0.0 +> **Date**: 2025-12-07 +> **Status**: Active +> **Ticket**: [OMN-465](https://linear.app/omninode/issue/OMN-465) +> **Author**: ONEX Framework Team + +--- + +## Executive Summary + +This document defines the versioning strategy for NodeCompute's contract-driven architecture. NodeCompute is a **platform primitive**, not a utility library. Its versioning philosophy prioritizes stability, predictability, and incremental complexity. + +**Core Principle**: Ship boring first, earn complexity later. + +--- + +## Table of Contents + +1. [Versioning Philosophy](#versioning-philosophy) +2. [Version Summary](#version-summary) +3. [v1.0 - Core Sequential Pipelines](#v10---core-sequential-pipelines) +4. [v1.1 Roadmap - Control Flow](#v11-roadmap---control-flow) +5. [v1.2 Roadmap - Observability and Caching](#v12-roadmap---observability-and-caching) +6. [v1.3 Roadmap - Evolution and Governance](#v13-roadmap---evolution-and-governance) +7. [NOT in 1.x](#not-in-1x) +8. [Migration Path](#migration-path) +9. [References](#references) + +--- + +## Versioning Philosophy + +### NodeCompute is a Platform Primitive + +NodeCompute is not a utility library that can be versioned casually. It is a **platform primitive** - a foundational building block upon which other ONEX components depend. This distinction demands a fundamentally different approach to versioning. + +**What makes NodeCompute a platform primitive:** + +1. **Ledger Integration**: NodeCompute transformations are recorded in the ONEX ledger for replay and audit. Breaking changes to transformation semantics break historical replay. + +2. **Contract-Driven Execution**: Agents and orchestrators generate NodeCompute contracts programmatically. Schema instability causes agent failures at scale. + +3. **Cross-Service Dependencies**: Multiple services depend on NodeCompute's transformation guarantees. Breaking changes cascade across the entire platform. + +4. **Long-Lived State**: Cached results and ledger entries may reference transformation versions from months or years ago. Version stability is not optional. + +### The "Boring First" Principle + +v1.0 must be **small, boring, and stable**. This means: + +**Small**: Include only what is necessary to prove the architecture works in production. Every additional feature is a liability until proven valuable. + +**Boring**: No clever optimizations, no exotic features, no speculative additions. Boring code is debuggable code. Boring APIs are learnable APIs. + +**Stable**: Once v1.0 ships, its behavior is frozen. No "minor breaking changes." No "technically backward compatible but different." Frozen. + +### Earned Complexity + +Each version after v1.0 must **earn its complexity** through demonstrated need: + +1. **v1.0 proves**: "Can we wire production transforms through contracts?" +2. **v1.1 proves**: "Do we actually need control flow in NodeCompute?" +3. **v1.2 proves**: "Is observability worth the model complexity?" +4. **v1.3 proves**: "Can we evolve contracts without breaking replay?" + +A feature graduates from "roadmap" to "implementation" only when: + +- Real production workloads require it +- Simpler alternatives have been tried and rejected +- The implementation maintains all prior guarantees + +### Version Guarantees + +**Within 1.x (v1.0 through v1.3):** + +| Guarantee | Description | +|-----------|-------------| +| **Schema Stability** | All v1.0 contracts remain valid in v1.3 | +| **Transformation Semantics** | Same input + same contract = same output | +| **Ledger Compatibility** | v1.0 ledger entries replayable by v1.3 executor | +| **Error Codes** | Error types are additive only, never removed | +| **Config Models** | Fields are additive only, never removed or retyped | + +**What can change within 1.x:** + +- New transformation types (additive) +- New step types (additive) +- New optional config fields with sensible defaults +- Performance improvements that preserve semantics +- New error types for new failure modes + +**What cannot change within 1.x:** + +- Existing transformation behavior +- Existing config field types or semantics +- Existing error type meanings +- Required field additions to existing models +- Removal of any public API + +### Version Numbering + +NodeCompute follows semantic versioning with platform-primitive semantics: + +```text +1.x.y +| | | +| | +-- Patch: Bug fixes, performance improvements (no API changes) +| +---- Minor: New features, new transformations (backward compatible) ++------ Major: Breaking changes (new replay epoch) +``` + +**Major version increments (2.0, 3.0) signal:** +- Ledger replay compatibility boundary +- Potential schema migration required +- Explicit opt-in from consumers + +--- + +## Version Summary + +| Version | Focus | Status | Documentation | +|---------|-------|--------|---------------| +| **v1.0** | Core sequential pipelines | Implementation Ready | [CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md](CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md) | +| **v1.1** | Control flow (conditional, parallel) | Planned | [Section below](#v11-roadmap---control-flow) | +| **v1.2** | Caching, observability, replay | Planned | [Section below](#v12-roadmap---observability-and-caching) | +| **v1.3** | Cancellation, versioning, schema policy | Planned | [Section below](#v13-roadmap---evolution-and-governance) | + +### Version Timeline (Tentative) + +```text +Milestone 1: v1.0 Implementation and Production Validation +Milestone 2: v1.1 Design and Implementation (if warranted, after v1.0 production validation) +Milestone 3: v1.2 Design and Implementation (if warranted, after v1.1 stabilization) +Milestone 4: v1.3 Design and Implementation (if warranted, after v1.2 stabilization) +``` + +**Note**: Each version's implementation depends on production feedback validating the need for additional complexity. Milestones are sequenced based on dependency completion rather than calendar dates. + +--- + +## v1.0 - Core Sequential Pipelines + +### Goal + +> "Can I wire real production transforms through contracts without embarrassment?" + +v1.0 answers this question and nothing more. It proves that: + +1. Contracts can define transformation pipelines +2. Pipelines execute deterministically +3. Results are suitable for ledger recording +4. The architecture scales to real workloads + +### What v1.0 Includes + +**Step Types (3):** +- `VALIDATION` - Schema validation against input/output refs (uses `ModelValidationStepConfig`) +- `TRANSFORMATION` - Apply a built-in transformation +- `MAPPING` - Shape results using simple path expressions (JSONPath-like) + +**Note**: Validation is a **step type**, not a transformation. This keeps semantic boundaries clean: +- VALIDATION → schema checks +- TRANSFORMATION → pure data transforms +- MAPPING → shape results + +**Transformation Types (6):** + +| Type | Purpose | Config Model | +|------|---------|--------------| +| `regex` | Pattern matching and replacement | `ModelTransformRegexConfig` | +| `case_conversion` | Uppercase/lowercase/titlecase | `ModelTransformCaseConfig` | +| `normalize_unicode` | Unicode normalization (NFC, NFD, etc.) | `ModelTransformUnicodeConfig` | +| `trim` | Whitespace trimming | `ModelTransformTrimConfig` | +| `json_path` | JSONPath extraction | `ModelTransformJsonPathConfig` | +| `identity` | Pass-through (no-op) | (no config) | + +**Core Models:** +- `ModelComputeSubcontract` - Contract definition +- `ModelComputePipelineStep` - Step definition +- `ModelComputeStepResult` - Step execution result +- Per-transformation config models (all with `extra="forbid"`, `frozen=True`) + +**Execution Guarantees:** +- Sequential step execution in declaration order +- Deterministic output for same input +- Immediate abort on any step failure +- Schema references resolved at load time + +### What v1.0 Excludes + +**Explicitly deferred to v1.1:** +- `CONDITIONAL` step type +- `PARALLEL` step type +- `SPLIT`, `JOIN`, `TEMPLATE`, `TYPE_CONVERSION`, `SCHEMA_VALIDATE` transformation types +- Boolean expression language +- Arithmetic expressions + +**Explicitly deferred to v1.2:** +- Caching configuration +- Debug metadata +- Performance metrics and timing +- Parallel execution statistics + +**Explicitly deferred to v1.3:** +- Cooperative cancellation +- Transformation versioning +- Ledger replay version matching +- Schema version strictness + +### v1.0 Contract Example + +```yaml +node_type: COMPUTE +node_name: text_normalizer +node_version: "1.0.0" + +compute_operations: + version: "1.0.0" + operation_name: normalize_text + operation_version: "1.0.0" + description: "Normalize text for consistent processing" + + input_schema_ref: text_input_v1 + output_schema_ref: text_output_v1 + pipeline_timeout_ms: 5000 + + pipeline: + - step_name: validate_input + step_type: validation + validation_config: + config_type: validation + schema_ref: input_schema + fail_on_error: true + + - step_name: normalize_unicode + step_type: transformation + transformation_type: normalize_unicode + transformation_config: + config_type: normalize_unicode + form: NFC + + - step_name: trim_whitespace + step_type: transformation + transformation_type: trim + transformation_config: + config_type: trim + mode: both + + - step_name: collapse_spaces + step_type: transformation + transformation_type: regex + transformation_config: + config_type: regex + pattern: "\\s+" + replacement: " " + + - step_name: build_output + step_type: mapping + mapping_config: + config_type: mapping + field_mappings: + normalized_text: "$.steps.collapse_spaces.output" +``` + +### v1.0 Success Criteria + +- [ ] All 6 transformation types implemented with typed configs +- [ ] VALIDATION step type uses `ModelValidationStepConfig` (not a transformation) +- [ ] Sequential pipeline execution works end-to-end +- [ ] Schema validation at load time catches malformed contracts +- [ ] Deterministic execution verified via property-based tests +- [ ] At least 3 production workloads running on v1.0 +- [ ] No `dict[str, Any]` at semantic boundaries +- [ ] mypy strict mode passes with zero errors + +### Reference + +See [CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md](CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md) for the complete v1.0 specification. + +--- + +## v1.1 Roadmap - Control Flow + +### Goal + +> "Can I express non-trivial control flow without writing Python?" + +v1.1 introduces conditional branching and parallel execution. These features are **only implemented if v1.0 production usage demonstrates clear need**. + +### Prerequisites + +Before implementing v1.1: + +1. v1.0 has been in production for at least 4 weeks +2. At least 2 production use cases explicitly require control flow +3. Workarounds using multiple nodes have proven unacceptable +4. Design review completed by ONEX architecture team + +### What v1.1 Adds + +**New Step Types:** + +| Step Type | Purpose | +|-----------|---------| +| `CONDITIONAL` | Branch execution based on expression | +| `PARALLEL` | Execute substeps concurrently, merge results | + +**New Transformation Types:** + +| Type | Purpose | Config Model | +|------|---------|--------------| +| `split` | String to array splitting | `ModelTransformSplitConfig` | +| `join` | Array to string joining | `ModelTransformJoinConfig` | +| `template` | String templating | `ModelTransformTemplateConfig` | +| `type_conversion` | Type coercion | `ModelTransformTypeConversionConfig` | +| `schema_validate` | Schema validation as transform | `ModelTransformValidationConfig` | + +**Note**: `MAPPING` step type with simple path expressions is already in v1.0. + +**New Configuration Models:** + +```python +class ModelConditionalConfig(BaseModel): + """Conditional execution configuration.""" + config_type: Literal["conditional"] = "conditional" + condition: str # ONEX Expression Language + then_steps: list[ModelComputePipelineStep] + else_steps: list[ModelComputePipelineStep] = Field(default_factory=list) + + model_config = ConfigDict(extra="forbid", frozen=True) + + +class EnumParallelMergeStrategy(str, Enum): + """Deterministic merge strategies for parallel steps.""" + OBJECT = "object" # Merge dicts + ARRAY = "array" # Concatenate arrays + FIRST = "first" # Take first result + LAST = "last" # Take last result + + +class ModelParallelConfig(BaseModel): + """Parallel execution configuration.""" + config_type: Literal["parallel"] = "parallel" + steps: list[ModelComputePipelineStep] + merge_strategy: EnumParallelMergeStrategy = EnumParallelMergeStrategy.OBJECT + max_parallelism: int | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) + + +class ModelMappingConfig(BaseModel): + """Field mapping configuration.""" + config_type: Literal["mapping"] = "mapping" + field_mappings: dict[str, str] # output_field -> path expression + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +**Basic Expression Language:** + +v1.1 introduces a minimal expression language for conditions and mappings: + +```ebnf +path = "$" segment* ; +segment = "." identifier | "[" index "]" ; +identifier = letter (letter | digit | "_")* ; +index = digit+ ; + +expr = path | path "==" literal | path "!=" literal | "!" expr ; +literal = string | number | "true" | "false" | "null" ; +``` + +**Supported operators (v1.1 only):** +- `==`, `!=` - Equality comparison +- `!` - Logical NOT +- `&&`, `||` - Logical AND/OR + +**Deferred operators (v1.3):** +- `<`, `<=`, `>`, `>=` - Numeric comparison +- `+`, `-`, `*`, `/`, `%` - Arithmetic +- `in` - Membership testing +- Function calls (`len()`, `exists()`, etc.) + +**Optional New Transformations:** + +If production use cases demand, v1.1 may include: + +| Type | Purpose | Config Model | +|------|---------|--------------| +| `filter` | Filter collection elements | `ModelTransformFilterConfig` | +| `map` | Transform collection elements | `ModelTransformMapConfig` | +| `reduce` | Aggregate collection to scalar | `ModelTransformReduceConfig` | + +These are **optional** - they are only added if v1.0 production feedback demonstrates need. + +### What v1.1 Still Defers + +**Deferred to v1.2:** +- Caching configuration +- Debug metadata and observability +- Performance timing and metrics + +**Deferred to v1.3:** +- Cancellation +- Transformation versioning +- Schema evolution policies + +### v1.1 Contract Example + +```yaml +compute_operations: + version: "1.1.0" + operation_name: format_aware_processor + operation_version: "1.0.0" + description: "Process data based on detected format" + + pipeline: + - step_name: detect_format + step_type: transformation + transformation_type: json_path + config: + path: "$.format" + + - step_name: process_by_format + step_type: conditional + conditional_config: + condition: "$.steps.detect_format.output == 'json'" + then_steps: + - step_name: normalize_json + step_type: transformation + transformation_type: identity # Placeholder for future json_normalize + else_steps: + - step_name: normalize_text + step_type: transformation + transformation_type: trim + config: + mode: both + + - step_name: aggregate_results + step_type: parallel + parallel_config: + steps: + - step_name: compute_length + step_type: transformation + transformation_type: json_path + config: + path: "$.length" + - step_name: compute_hash + step_type: transformation + transformation_type: identity # Placeholder + merge_strategy: object + + - step_name: output_mapping + step_type: mapping + mapping_config: + field_mappings: + result: "$.steps.process_by_format.output" + metadata.length: "$.steps.compute_length.output" + metadata.hash: "$.steps.compute_hash.output" +``` + +### v1.1 Success Criteria + +- [ ] Conditional branching executes correctly +- [ ] Parallel steps execute concurrently with deterministic merge +- [ ] Mapping expressions resolve paths correctly +- [ ] Expression evaluation is type-safe (no implicit coercion) +- [ ] All v1.0 contracts remain valid and produce identical output +- [ ] Forward references in expressions fail at contract load time + +--- + +## v1.2 Roadmap - Observability and Caching + +### Goal + +> "Can I observe, cache, and reliably replay what happened?" + +v1.2 adds production-grade observability and caching. These features require v1.0/v1.1 to be stable and well-understood. + +### Prerequisites + +Before implementing v1.2: + +1. v1.1 has been in production for at least 4 weeks (or v1.0 if v1.1 was skipped) +2. Production monitoring reveals specific observability gaps +3. Cache hit rates justify caching complexity +4. Replay requirements are documented and tested + +### What v1.2 Adds + +**Caching Configuration:** + +```python +class EnumEvictionPolicy(str, Enum): + """Cache eviction policies.""" + LRU = "lru" + LFU = "lfu" + FIFO = "fifo" + TTL_ONLY = "ttl_only" + + +class ModelComputeCaching(BaseModel): + """Caching configuration for compute operations.""" + enabled: bool = False + key_fields: list[str] = Field(default_factory=list) + ttl_seconds: int = 3600 + eviction_policy: EnumEvictionPolicy = EnumEvictionPolicy.LRU + max_size: int | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +**Cache Key Semantics (Pseudocode)**: + +```python +# Cache key derivation (v1.2) +cache_key = hash( + operation_name, + operation_version, + transformation_version, # Bound to omnibase_core version + tuple(resolve_key_field(data, field) for field in key_fields), +) +``` + +**Key fields must:** +- Be dot-notation paths into input data +- Be validated against `input_schema_ref` at load time +- Fail gracefully at runtime if unresolvable (cache miss, warning in debug metadata) + +**Debug Metadata:** + +v1.2 introduces the **only** approved use of `dict[str, Any]` - opaque debug metadata: + +```python +class ModelComputeDebugMetadata(BaseModel): + """ + Opaque debug metadata for pipeline execution. + + CRITICAL: This is the ONLY place where dict[str, Any] is allowed. + This data MUST NOT affect deterministic behavior. + This data MUST NOT be used for cache key derivation. + This data MAY be omitted entirely in production mode. + """ + data: dict[str, Any] = Field(default_factory=dict) + + model_config = ConfigDict(extra="allow") # Intentionally permissive +``` + +**Performance Metrics:** + +```python +class ModelComputePerformance(BaseModel): + """Performance configuration and metrics.""" + timeout_ms: int | None = None + pipeline_timeout_ms: int | None = None + batch_size: int | None = None + max_retries: int = 0 + retry_delay_ms: int = 100 + + model_config = ConfigDict(extra="forbid", frozen=True) + + +class ModelComputeStepMetadata(BaseModel): + """Execution metadata for a single step.""" + duration_ms: float + cache_hit: bool = False + warnings: list[str] = Field(default_factory=list) + transformation_type: str | None = None + input_type: str | None = None + output_type: str | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +**Error Type Taxonomy:** + +```python +class EnumComputeErrorType(str, Enum): + """Structured error types for compute failures.""" + CONTRACT_VALIDATION = "contract_validation" + SCHEMA_VALIDATION = "schema_validation" + TRANSFORMATION = "transformation" + EXPRESSION = "expression" + TIMEOUT = "timeout" + PARALLEL_FAILURE = "parallel_failure" + FORWARD_REFERENCE = "forward_reference" + CACHE_KEY_RESOLUTION = "cache_key_resolution" + + +class ModelComputeError(BaseModel): + """Structured error for pipeline failures.""" + error_type: EnumComputeErrorType + message: str + step_name: str | None = None + expression: str | None = None + path: str | None = None + stack_trace: str | None = None # Debug mode only + + model_config = ConfigDict(extra="forbid", frozen=True) + + +class ModelParallelStepError(BaseModel): + """Aggregated error for parallel step failures.""" + substep_errors: list[ModelComputeError] + total_substeps: int + failed_substeps: int + completed_substeps: int + + model_config = ConfigDict(extra="forbid", frozen=True) +``` + +**Immutability Enforcement:** + +v1.2 enforces `frozen=True` on all context and result models: + +```python +class ModelComputeExecutionContext(BaseModel): + """Typed execution context for compute pipelines.""" + operation_id: UUID + computation_type: EnumComputationType + correlation_id: UUID | None = None + tenant_id: str | None = None + node_id: str | None = None + extra: dict[str, Any] = Field(default_factory=dict) # Opaque only + + model_config = ConfigDict(frozen=True) # FROZEN + + +class ModelComputeStepResult(BaseModel): + """Result of a single pipeline step.""" + step_name: str + output: Any + success: bool = True + metadata: ModelComputeStepMetadata + error: ModelComputeError | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) # FROZEN +``` + +### What v1.2 Still Defers + +**Deferred to v1.3:** +- Cooperative cancellation +- Transformation version binding +- Ledger replay version matching +- Schema version strictness policies + +### v1.2 Success Criteria + +- [ ] Caching reduces redundant computation in production +- [ ] Cache keys are derived deterministically +- [ ] Debug metadata is non-semantic (removing it doesn't change output) +- [ ] Performance metrics accurately reflect execution time +- [ ] Error types cover all observed failure modes +- [ ] All models are immutable (frozen=True verified) +- [ ] All v1.0 and v1.1 contracts remain valid + +--- + +## v1.3 Roadmap - Evolution and Governance + +### Goal + +> "Can I evolve this thing without hating myself?" + +v1.3 adds the governance and evolution mechanisms needed for long-term maintenance. These features are complex and should only be implemented after v1.0-v1.2 are stable. + +### Prerequisites + +Before implementing v1.3: + +1. v1.2 has been in production for at least 8 weeks +2. At least one schema evolution has been manually managed +3. Cancellation requirements are documented from orchestrator usage +4. Ledger replay has been validated against production entries + +### What v1.3 Adds + +**Cooperative Cancellation:** + +```python +class ModelComputeCancellation(BaseModel): + """Cancellation state for pipeline execution.""" + cancelled: bool = False + reason: str | None = None + requested_at: datetime | None = None + + model_config = ConfigDict(frozen=True) +``` + +**Cancellation semantics:** + +| Phase | Behavior | +|-------|----------| +| Between steps | Check cancellation, abort if requested | +| During step | Step completes, then abort | +| During parallel | All substeps complete, then abort | + +Cancellation is **cooperative**, not preemptive. Steps are never interrupted mid-execution. + +**Transformation Versioning:** + +```python +# Bound to omnibase_core version +TRANSFORMATION_VERSION = "1.3.0" + +class ModelComputeSubcontract(BaseModel): + # ... existing fields ... + + # Recorded at contract load time + transformation_version: str = Field(default=TRANSFORMATION_VERSION) +``` + +**Ledger replay semantics:** + +1. Ledger entries record `transformation_version` +2. Replay executor validates version compatibility +3. Mismatched versions fail with clear error +4. No automatic version migration (explicit operator action required) + +**Schema Version Strictness:** + +Schemas are frozen at contract load time: + +```text +At load time: +1. Resolve input_schema_ref from registry +2. Freeze resolved schema into contract +3. Schema updates in registry have no effect on loaded contract +4. New contracts get new schema version +``` + +**Expanded Expression Language:** + +v1.3 completes the expression language: + +```ebnf +# Arithmetic (v1.3) +add_expr = mul_expr ( ("+" | "-") mul_expr )* ; +mul_expr = unary_expr ( ("*" | "/" | "%") unary_expr )* ; + +# Comparison (v1.3) +comp_op = "==" | "!=" | "<" | "<=" | ">" | ">=" | "in" ; + +# Functions (v1.3) +func_call = identifier "(" (expr ("," expr)*)? ")" ; +``` + +**Supported functions (v1.3):** + +| Function | Description | Example | +|----------|-------------|---------| +| `len(x)` | Length of string/array | `len($.items) > 0` | +| `exists(path)` | Path exists and not null | `exists($.optional)` | +| `type(x)` | Type of value | `type($.value) == "string"` | +| `lower(s)` | Lowercase string | `lower($.text)` | +| `upper(s)` | Uppercase string | `upper($.text)` | + +**Registry Lockdown:** + +The transformation registry is **internal only**: + +```python +def register_transformation( + name: EnumTransformationType, + handler: Callable[[Any, BaseModel], Any], +) -> None: + """ + Register a transformation handler. + + NOTE: This is an INTERNAL runtime extension hook. + - Application code CANNOT introduce new transformation types + - All allowed transformations MUST be in EnumTransformationType + - This function is ONLY called during ONEX Core initialization + """ +``` + +User-defined transformations are **not supported** in NodeCompute. Custom logic requires a different node family (NodeScript, NodeLambda, or custom nodes). + +### v1.3 Success Criteria + +- [ ] Cancellation works correctly from orchestrator +- [ ] Ledger replay validates transformation version +- [ ] Schema freezing prevents accidental version drift +- [ ] Expression language passes comprehensive fuzzing +- [ ] Registry lockdown prevents user-defined transformations +- [ ] All v1.0, v1.1, and v1.2 contracts remain valid + +--- + +## NOT in 1.x + +The following features are **explicitly out of scope** for the 1.x version family. They require major version increments (2.0+) or belong in separate products. + +### Hot-Swappable Contracts (2.x) + +Changing contracts on running NodeCompute instances is not supported in 1.x: + +- Once attached, contracts cannot be changed +- New contracts require new NodeCompute instances +- In-flight executions use the original contract +- Hot-swapping introduces state management complexity that 1.x avoids + +**Rationale**: Contract hot-swapping requires sophisticated state transfer, in-flight execution tracking, and rollback mechanisms. This complexity is not justified until v1.x has proven stable in production. + +### Schema Migration Engine (Separate Product) + +Automatic schema migration and backward compatibility checking is **not** part of NodeCompute: + +- Schemas are frozen at load time +- Version mismatches fail explicitly +- No automatic type coercion or field mapping +- Migration is an operator responsibility + +**Rationale**: Schema migration is a complex problem that deserves its own product. Embedding migration logic in NodeCompute would violate the "boring first" principle and introduce failure modes that are difficult to debug. + +### User-Defined Transformations + +NodeCompute does not support user-defined transformations: + +- All transformation types are in `EnumTransformationType` +- No CUSTOM transformation type +- No runtime transformation registration +- Registry is internal to ONEX Core + +**Rationale**: User-defined transformations break: + +1. **Determinism**: Custom code can have side effects +2. **Replay**: Custom code may not be available at replay time +3. **Security**: Custom code can access arbitrary resources +4. **Versioning**: Custom code has independent versioning + +**Alternative**: Users requiring custom transformations should use: + +- **NodeScript**: For lightweight scripting (future) +- **NodeLambda**: For serverless functions (future) +- **Custom nodes**: For complex business logic (existing) + +### Non-Deterministic Merge Strategies + +Parallel step merge strategies in NodeCompute are strictly deterministic: + +- No random selection +- No timing-based ordering +- No "fastest wins" semantics +- Declaration order is canonical + +**Rationale**: Non-deterministic merge breaks ledger replay and makes debugging nearly impossible. + +### Dynamic Step Generation + +Pipeline steps are static and defined at contract load time: + +- No runtime step injection +- No dynamic pipeline modification +- No conditional step generation based on data +- Pipeline structure is frozen + +**Rationale**: Dynamic step generation would make contracts unpredictable and break static analysis tools. + +--- + +## Migration Path + +### v1.0 to v1.1 + +**What changes:** +- New step types: `CONDITIONAL`, `PARALLEL`, `MAPPING` +- New config models for new step types +- Basic expression language + +**Migration effort:** None required + +All v1.0 contracts work unchanged in v1.1. New features are additive only. + +**Validation**: + +```bash +# Verify v1.0 contract still works +poetry run pytest tests/contract_compatibility/test_v1_0_on_v1_1.py +``` + +### v1.1 to v1.2 + +**What changes:** +- Caching configuration added to subcontract +- Debug metadata added to results +- Performance metrics added to step results +- Error type taxonomy expanded +- Immutability enforcement + +**Migration effort:** None required + +All v1.0 and v1.1 contracts work unchanged in v1.2. New fields have sensible defaults. + +**Validation**: + +```bash +# Verify v1.0 and v1.1 contracts still work +poetry run pytest tests/contract_compatibility/test_v1_x_on_v1_2.py +``` + +### v1.2 to v1.3 + +**What changes:** +- Cancellation support added +- Transformation version recorded in subcontract +- Schema version strictness enforced +- Expression language expanded +- Registry lockdown enforced + +**Migration effort:** None required + +All v1.0, v1.1, and v1.2 contracts work unchanged in v1.3. Transformation version defaults to current version for existing contracts. + +**Validation**: + +```bash +# Verify all prior contracts still work +poetry run pytest tests/contract_compatibility/test_v1_x_on_v1_3.py +``` + +### Backward Compatibility Promise + +**Within 1.x:** + +```text +Any contract valid in v1.0 MUST be valid in v1.3 +Any contract valid in v1.0 MUST produce identical output in v1.3 +Any ledger entry from v1.0 MUST be replayable in v1.3 +``` + +This promise is **enforced by tests**, not just documented. The test suite includes: + +1. **Contract Compatibility Tests**: Run all historic contracts on current version +2. **Output Determinism Tests**: Verify identical output across versions +3. **Ledger Replay Tests**: Replay production ledger entries + +### When to Expect Breaking Changes (2.0) + +A major version (2.0) will only be released when: + +1. Accumulated technical debt makes 1.x maintenance unsustainable +2. Fundamental architecture changes are required for scaling +3. The ONEX ledger format requires incompatible changes +4. Security vulnerabilities require breaking changes + +Major version releases will include: + +- 6-month deprecation notice +- Automated migration tooling where possible +- Parallel support for 1.x and 2.x during transition +- Clear documentation of all breaking changes + +--- + +## References + +### Related Documents + +| Document | Description | +|----------|-------------| +| [CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md](CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md) | v1.0 implementation specification | +| [SUBCONTRACT_ARCHITECTURE.md](SUBCONTRACT_ARCHITECTURE.md) | Subcontract pattern documentation | +| [DECLARATIVE_IMPLEMENTATION_PLAN.md](DECLARATIVE_IMPLEMENTATION_PLAN.md) | Overall declarative node implementation | + + + + +### Linear Tickets + +| Ticket | Description | +|--------|-------------| +| [OMN-465](https://linear.app/omninode/issue/OMN-465) | Contract-Driven NodeCompute | + +### External Resources + +- [Semantic Versioning 2.0.0](https://semver.org/) +- [Pydantic v2 Documentation](https://docs.pydantic.dev/latest/) + +--- + +## Changelog + +| Version | Date | Changes | +|---------|------|---------| +| 1.0.0 | 2025-12-07 | Initial roadmap document | + +--- + +**Last Updated**: 2025-12-07 +**Document Version**: 1.0.0 +**Status**: Active +**Maintainer**: ONEX Framework Team diff --git a/docs/patterns/EVENT_DRIVEN_ARCHITECTURE.md b/docs/patterns/EVENT_DRIVEN_ARCHITECTURE.md index f461bc5163..0003dee7a6 100644 --- a/docs/patterns/EVENT_DRIVEN_ARCHITECTURE.md +++ b/docs/patterns/EVENT_DRIVEN_ARCHITECTURE.md @@ -12,7 +12,7 @@ Event-driven architecture patterns for ONEX services using ModelEventEnvelope an All events use `ModelEventEnvelope` for consistent communication: -``` +```python from omnibase_core.models.events.model_event_envelope import ModelEventEnvelope from uuid import UUID from datetime import datetime @@ -29,7 +29,7 @@ envelope = ModelEventEnvelope( REDUCER nodes emit Intents instead of performing side effects: -``` +```python def reduce_state( state: ModelState, action: ModelAction @@ -54,7 +54,7 @@ def reduce_state( ### Pattern 1: Request-Response -``` +```text ┌─────────┐ event ┌─────────┐ event ┌─────────┐ │ Client │ ──────────────> │ Node │ ──────────────> │ Client │ └─────────┘ └─────────┘ └─────────┘ @@ -62,7 +62,7 @@ def reduce_state( ### Pattern 2: Publish-Subscribe -``` +```text ┌─────────┐ ┌─────────────┐ │Publisher│ ───────────────>│ Event Bus │ └─────────┘ └──────┬──────┘ @@ -77,7 +77,7 @@ def reduce_state( ### Pattern 3: Intent → Action Flow (Pure FSM) -``` +```text ┌─────────┐ Action ┌─────────┐ Intents ┌────────┐ │Orchestr │ ─────────> │ Reducer │ ────────> │ Effect │ └─────────┘ └─────────┘ └────────┘ @@ -90,7 +90,7 @@ def reduce_state( ### Event Publishing -``` +```python class MyNode(ModelServiceCompute): async def process(self, data: dict) -> dict: # Process data @@ -111,7 +111,7 @@ class MyNode(ModelServiceCompute): ### Event Subscription -``` +```python class MyNode(ModelServiceEffect): async def initialize(self): event_bus = self.container.get_service("ProtocolEventBus") @@ -133,7 +133,7 @@ class MyNode(ModelServiceEffect): ### Intent Execution -``` +```python class MyEffectNode(ModelServiceEffect): async def execute_intent(self, intent: ModelIntent): """Execute Intent from REDUCER.""" @@ -159,7 +159,7 @@ class MyEffectNode(ModelServiceEffect): ### 1. Use Correlation IDs -``` +```python envelope = ModelEventEnvelope( event_type="DATA_PROCESSED", payload=data, @@ -169,7 +169,7 @@ envelope = ModelEventEnvelope( ### 2. Type-Safe Event Types -``` +```python class EnumEventType(str, Enum): DATA_PROCESSED = "data_processed" ERROR_OCCURRED = "error_occurred" @@ -184,8 +184,8 @@ envelope = ModelEventEnvelope( ### 3. Structured Event Payloads -``` -# ❌ Bad: Unstructured payload +```python +# Bad: Unstructured payload payload = {"data": "something", "other": 123} # ✅ Good: Pydantic model @@ -198,7 +198,7 @@ payload = DataProcessedPayload(result="success", metrics={...}) ## Testing Event-Driven Code -``` +```python @pytest.mark.asyncio async def test_event_publishing(): # Arrange diff --git a/docs/quality/DOCUMENTATION_VALIDATION_REPORT.md b/docs/quality/DOCUMENTATION_VALIDATION_REPORT.md index 4af301c64c..48ee40eba6 100644 --- a/docs/quality/DOCUMENTATION_VALIDATION_REPORT.md +++ b/docs/quality/DOCUMENTATION_VALIDATION_REPORT.md @@ -73,13 +73,13 @@ Comprehensive validation report for documentation quality, accuracy, and complet ### Link Validation -``` +```bash poetry run python scripts/validation/validate_markdown_links.py ``` ### Code Example Testing -``` +```bash # Extract code examples poetry run python scripts/validation/extract_code_examples.py @@ -89,7 +89,7 @@ poetry run pytest extracted_examples/ ### Type Validation -``` +```bash poetry run mypy src/ ``` diff --git a/docs/quality/TERMINOLOGY_AUDIT_REPORT.md b/docs/quality/TERMINOLOGY_AUDIT_REPORT.md index 7f85e9d0fd..2c162a83c5 100644 --- a/docs/quality/TERMINOLOGY_AUDIT_REPORT.md +++ b/docs/quality/TERMINOLOGY_AUDIT_REPORT.md @@ -168,9 +168,9 @@ class NodeFileBackupEffect ← Code - `CLAUDE.md:180-215` - Critical distinction highlighted - Multiple examples showing correct usage patterns -✅ **Consistent Code Examples**: -``` -# ✅ Correct - DI container in node constructor +**Consistent Code Examples**: +```python +# Correct - DI container in node constructor def __init__(self, container: ModelONEXContainer): super().__init__(container) @@ -202,8 +202,8 @@ config = ModelContainer.create(value="production") **Evidence**: -✅ **Consistent Patterns**: -``` +**Consistent Patterns**: +```python # Get services by protocol interface (never by concrete class) event_bus = container.get_service("ProtocolEventBus") ``` diff --git a/docs/testing/TESTMON_USAGE.md b/docs/testing/TESTMON_USAGE.md index 36be48c3a3..8aa263272a 100644 --- a/docs/testing/TESTMON_USAGE.md +++ b/docs/testing/TESTMON_USAGE.md @@ -6,7 +6,7 @@ pytest-testmon is an intelligent test selection tool that tracks code dependenci ## Quick Start -``` +```bash # Run only tests affected by recent changes poetry run pytest --testmon @@ -34,7 +34,7 @@ pytest-testmon: ### Local Development (Recommended) -``` +```bash # Fast iteration: Run only affected tests poetry run pytest --testmon @@ -46,14 +46,14 @@ poetry run pytest --testmon-noselect ### Pre-Commit Validation -``` +```bash # Validate your changes before commit poetry run pytest --testmon --cov --cov-fail-under=60 ``` ### Full Suite (Periodic) -``` +```bash # Run full suite periodically to catch integration issues poetry run pytest tests/ ``` @@ -83,7 +83,7 @@ See [CI_TEST_STRATEGY.md](./CI_TEST_STRATEGY.md) for recommended CI test approac ### Testmon selecting too many/few tests -``` +```bash # Clear database and rebuild rm .testmondata poetry run pytest --testmon-noselect @@ -91,7 +91,7 @@ poetry run pytest --testmon-noselect ### Testmon not detecting changes -``` +```bash # Ensure you're using poetry run poetry run pytest --testmon # Correct pytest --testmon # Wrong (uses system pytest) @@ -99,7 +99,7 @@ pytest --testmon # Wrong (uses system pytest) ### Performance issues -``` +```bash # Testmon database may be corrupt rm .testmondata poetry run pytest --testmon-noselect diff --git a/docs/testing/TEST_COUNT_CLARIFICATION_SUMMARY.md b/docs/testing/TEST_COUNT_CLARIFICATION_SUMMARY.md index 771b737325..dfa17c4898 100644 --- a/docs/testing/TEST_COUNT_CLARIFICATION_SUMMARY.md +++ b/docs/testing/TEST_COUNT_CLARIFICATION_SUMMARY.md @@ -20,7 +20,7 @@ The omnibase_core documentation contained multiple inconsistent and outdated tes ## Verified Actual Count -``` +```bash $ poetry run pytest --collect-only -q 2>/dev/null | tail -n 1 12198 tests collected in 20.10s ``` @@ -115,7 +115,7 @@ The 12,198 tests are distributed across: To verify the test count at any time: -``` +```bash # Quick count (accurate) poetry run pytest --collect-only -q 2>/dev/null | tail -n 1 diff --git a/examples/contracts/compute/user_profile_normalizer.yaml b/examples/contracts/compute/user_profile_normalizer.yaml new file mode 100644 index 0000000000..e54ef0e5a9 --- /dev/null +++ b/examples/contracts/compute/user_profile_normalizer.yaml @@ -0,0 +1,358 @@ +# Example: Contract-Driven NodeCompute Pipeline - User Profile Normalizer +# +# This example demonstrates the v1.0 NodeCompute contract-driven pipeline system +# for normalizing user profile data before storage or processing. +# +# USE CASE: Normalize user-submitted profile data to ensure consistency: +# - Validate incoming data against schema +# - Normalize unicode characters (handle accented names, etc.) +# - Trim whitespace from text fields +# - Standardize email format (lowercase) +# - Clean phone numbers (remove non-digits) +# - Map fields to final output structure +# +# Node Implementation (minimal code needed): +# class NodeUserProfileNormalizerCompute(NodeCompute): +# pass # All pipeline logic driven by this YAML contract +# +# Features Demonstrated: +# - All 3 step types: VALIDATION, TRANSFORMATION, MAPPING +# - 5 of 6 transformation types: TRIM, NORMALIZE_UNICODE, CASE_CONVERSION, REGEX, JSON_PATH +# - Path resolution: $.input, $.input., $.steps..output +# - Sequential execution with abort-on-first-failure semantics +# - Pipeline-level timeout configuration +# +# v1.0 Contract Schema Version +# ============================================================================ + +# --------------------------------------------------------------------------- +# COMPUTE SUBCONTRACT: Defines the transformation pipeline +# --------------------------------------------------------------------------- +# The compute subcontract specifies the sequential transformation pipeline +# that the NodeCompute will execute. Steps are processed in order, and +# any failure causes immediate abort (abort-on-first-failure semantics). + +compute: + # Schema version for forward compatibility + version: "1.0.0" + + # Operation identification + operation_name: user_profile_normalizer + operation_version: "1.0.0" + description: | + Normalizes user profile data for consistent storage and processing. + Handles unicode normalization, whitespace trimming, email standardization, + and phone number cleaning. Validates input/output against defined schemas. + + # Schema references (resolved at load time by schema registry) + # These reference JSON schemas in your project's schema directory + input_schema_ref: "schemas/user_profile_input.json" + output_schema_ref: "schemas/user_profile_normalized.json" + + # Pipeline-level timeout (no per-step timeouts in v1.0) + pipeline_timeout_ms: 5000 + + # --------------------------------------------------------------------------- + # PIPELINE: Sequential list of transformation steps + # --------------------------------------------------------------------------- + # Each step is executed in order. If any step fails, the pipeline aborts. + # Steps can reference: + # - $.input - The original pipeline input + # - $.input. - A specific field from the input + # - $.steps..output - Output from a named previous step + + pipeline: + # ========================================================================= + # STEP 1: INPUT VALIDATION + # ========================================================================= + # Validates the incoming data against the input schema before processing. + # This is a VALIDATION step type - it requires validation_config. + # + # Step Type: VALIDATION + # Purpose: Ensure input data meets expected structure and constraints + # On Failure: Pipeline aborts immediately (fail_on_error: true) + + - step_name: validate_input + step_type: validation + enabled: true + validation_config: + config_type: validation + # Reference to JSON schema for input validation + schema_ref: "schemas/user_profile_input.json" + # If validation fails, abort the pipeline + fail_on_error: true + + # ========================================================================= + # STEP 2: UNICODE NORMALIZATION + # ========================================================================= + # Normalizes unicode characters to NFC form for consistent storage. + # Handles accented characters, combining characters, etc. + # + # Step Type: TRANSFORMATION + # Transformation Type: NORMALIZE_UNICODE + # Input: $.input.display_name (from original input) + # Output: Normalized unicode string + # + # Example: + # Input: "Jose\u0301" (Jose with combining acute accent) + # Output: "Jos\u00e9" (Jose with precomposed e-acute) + + - step_name: normalize_display_name + step_type: transformation + enabled: true + transformation_type: normalize_unicode + transformation_config: + config_type: normalize_unicode + # NFC = Canonical Decomposition, followed by Canonical Composition + # This is the recommended form for most use cases + form: NFC + + # ========================================================================= + # STEP 3: WHITESPACE TRIMMING + # ========================================================================= + # Removes leading and trailing whitespace from the display name. + # + # Step Type: TRANSFORMATION + # Transformation Type: TRIM + # Input: $.steps.normalize_display_name.output (from previous step) + # Output: Trimmed string + # + # Example: + # Input: " John Smith " + # Output: "John Smith" + + - step_name: trim_display_name + step_type: transformation + enabled: true + transformation_type: trim + transformation_config: + config_type: trim + # BOTH = trim from both left and right ends + mode: both + + # ========================================================================= + # STEP 4: EMAIL NORMALIZATION (CASE CONVERSION) + # ========================================================================= + # Converts email address to lowercase for consistent matching and storage. + # + # Step Type: TRANSFORMATION + # Transformation Type: CASE_CONVERSION + # Input: $.input.email (from original input) + # Output: Lowercase email string + # + # Example: + # Input: "John.Smith@Example.COM" + # Output: "john.smith@example.com" + + - step_name: normalize_email + step_type: transformation + enabled: true + transformation_type: case_conversion + transformation_config: + config_type: case_conversion + mode: lowercase + + # ========================================================================= + # STEP 5: PHONE NUMBER CLEANING (REGEX) + # ========================================================================= + # Removes all non-digit characters from phone number for consistent format. + # + # Step Type: TRANSFORMATION + # Transformation Type: REGEX + # Input: $.input.phone (from original input) + # Output: Digits-only phone string + # + # Example: + # Input: "+1 (555) 123-4567" + # Output: "15551234567" + + - step_name: clean_phone_number + step_type: transformation + enabled: true + transformation_type: regex + transformation_config: + config_type: regex + # Pattern matches any non-digit character + pattern: "[^0-9]" + # Replace with empty string (deletion) + replacement: "" + # No special flags needed for this pattern + flags: [] + + # ========================================================================= + # STEP 6: EXTRACT USER ID FROM METADATA (JSON PATH) + # ========================================================================= + # Extracts nested user ID from metadata using JSONPath expression. + # + # Step Type: TRANSFORMATION + # Transformation Type: JSON_PATH + # Input: $.input (entire input object) + # Output: Extracted user_id value + # + # Example: + # Input: {"metadata": {"ids": {"user_id": "usr_12345"}}} + # Output: "usr_12345" + + - step_name: extract_user_id + step_type: transformation + enabled: true + transformation_type: json_path + transformation_config: + config_type: json_path + # JSONPath expression to extract nested user_id + path: "$.metadata.ids.user_id" + + # ========================================================================= + # STEP 7: IDENTITY TRANSFORMATION (PASSTHROUGH) + # ========================================================================= + # Demonstrates the IDENTITY transformation - passes data through unchanged. + # Useful for conditional pipelines or placeholder steps. + # + # Step Type: TRANSFORMATION + # Transformation Type: IDENTITY + # Note: IDENTITY transformations must NOT have transformation_config + # + # Example: + # Input: "any value" + # Output: "any value" (unchanged) + + - step_name: passthrough_bio + step_type: transformation + enabled: true + transformation_type: identity + # Note: No transformation_config for IDENTITY - it would cause validation error! + + # ========================================================================= + # STEP 8: OUTPUT MAPPING + # ========================================================================= + # Maps all processed fields to the final output structure. + # This is a MAPPING step type - it requires mapping_config. + # + # Step Type: MAPPING + # Purpose: Combine results from multiple steps into final output + # + # Path Resolution: + # - $.input - Original pipeline input + # - $.input. - Specific field from original input + # - $.steps..output - Output from a named step + + - step_name: build_output + step_type: mapping + enabled: true + mapping_config: + config_type: mapping + field_mappings: + # From previous transformation steps + display_name: "$.steps.trim_display_name.output" + email: "$.steps.normalize_email.output" + phone: "$.steps.clean_phone_number.output" + user_id: "$.steps.extract_user_id.output" + bio: "$.steps.passthrough_bio.output" + + # Direct from input (unchanged fields) + created_at: "$.input.created_at" + preferences: "$.input.preferences" + + # Nested field access from input + country_code: "$.input.address.country_code" + + # ========================================================================= + # STEP 9: OUTPUT VALIDATION + # ========================================================================= + # Validates the final mapped output against the output schema. + # This ensures the pipeline produces valid, well-formed output. + # + # Step Type: VALIDATION + # Purpose: Verify output meets downstream system requirements + + - step_name: validate_output + step_type: validation + enabled: true + validation_config: + config_type: validation + schema_ref: "schemas/user_profile_normalized.json" + fail_on_error: true + +# --------------------------------------------------------------------------- +# EXAMPLE INPUT/OUTPUT +# --------------------------------------------------------------------------- +# This section shows what input the pipeline expects and what output it produces. +# Not part of the contract schema - just documentation. + +# Example Input: +# { +# "display_name": " Jose\u0301 Smith ", +# "email": "Jose.Smith@EXAMPLE.COM", +# "phone": "+1 (555) 123-4567", +# "bio": "Software developer and coffee enthusiast.", +# "created_at": "2024-01-15T10:30:00Z", +# "preferences": {"theme": "dark", "notifications": true}, +# "address": {"country_code": "US", "postal_code": "12345"}, +# "metadata": {"ids": {"user_id": "usr_12345", "org_id": "org_789"}} +# } + +# Example Output: +# { +# "display_name": "Jos\u00e9 Smith", +# "email": "jose.smith@example.com", +# "phone": "15551234567", +# "user_id": "usr_12345", +# "bio": "Software developer and coffee enthusiast.", +# "created_at": "2024-01-15T10:30:00Z", +# "preferences": {"theme": "dark", "notifications": true}, +# "country_code": "US" +# } + +# --------------------------------------------------------------------------- +# USAGE EXAMPLE +# --------------------------------------------------------------------------- +# How to use this contract with NodeCompute: +# +# from omnibase_core.nodes import NodeCompute +# from omnibase_core.models import ModelComputeInput +# import yaml +# +# # Load contract +# with open("user_profile_normalizer.yaml") as f: +# contract_data = yaml.safe_load(f) +# +# # Create node (contract drives all logic) +# class NodeUserProfileNormalizerCompute(NodeCompute): +# pass +# +# node = NodeUserProfileNormalizerCompute(container, contract=contract_data) +# +# # Process data +# input_data = ModelComputeInput( +# data={ +# "display_name": " Jose\u0301 Smith ", +# "email": "Jose.Smith@EXAMPLE.COM", +# "phone": "+1 (555) 123-4567", +# "bio": "Software developer.", +# "created_at": "2024-01-15T10:30:00Z", +# "preferences": {"theme": "dark"}, +# "address": {"country_code": "US"}, +# "metadata": {"ids": {"user_id": "usr_12345"}} +# } +# ) +# +# result = await node.process(input_data) +# print(result.data) # Normalized profile data + +# --------------------------------------------------------------------------- +# CONTRACT METADATA +# --------------------------------------------------------------------------- + +metadata: + version: {major: 1, minor: 0, patch: 0} + author: "ONEX Framework Team" + description: | + Contract-driven compute pipeline for user profile normalization. + Demonstrates all v1.0 step types and transformation types. + tags: + - compute + - pipeline + - normalization + - user-profile + - example + documentation_url: "https://docs.onex.ai/compute/user-profile-normalizer" diff --git a/src/omnibase_core/enums/__init__.py b/src/omnibase_core/enums/__init__.py index 2b84713836..a423b7394d 100644 --- a/src/omnibase_core/enums/__init__.py +++ b/src/omnibase_core/enums/__init__.py @@ -20,6 +20,9 @@ from .enum_backoff_strategy import EnumBackoffStrategy from .enum_business_logic_pattern import EnumBusinessLogicPattern +# Case mode enums (contract-driven NodeCompute v1.0) +from .enum_case_mode import EnumCaseMode + # Category filter enums from .enum_category_filter import EnumCategoryFilter @@ -28,6 +31,9 @@ # Computation and processing enums from .enum_computation_type import EnumComputationType + +# Compute step type enums (contract-driven NodeCompute v1.0) +from .enum_compute_step_type import EnumComputeStepType from .enum_contract_compliance import EnumContractCompliance from .enum_coordination_mode import EnumCoordinationMode from .enum_core_error_code import ( @@ -148,6 +154,9 @@ EnumReductionType, EnumStreamingMode, ) + +# Regex flag enums (contract-driven NodeCompute v1.0) +from .enum_regex_flag import EnumRegexFlag from .enum_registry_error_code import EnumRegistryErrorCode # Registry-related enums @@ -182,12 +191,21 @@ from .enum_tool_status import EnumToolStatus from .enum_tool_type import EnumToolType +# Transformation types (contract-driven NodeCompute v1.0) +from .enum_transformation_type import EnumTransformationType + # State management enums from .enum_transition_type import EnumTransitionType # Tree sync enums from .enum_tree_sync_status import EnumTreeSyncStatus +# Trim mode enums (contract-driven NodeCompute v1.0) +from .enum_trim_mode import EnumTrimMode + +# Unicode form enums (contract-driven NodeCompute v1.0) +from .enum_unicode_form import EnumUnicodeForm + # URI-related enums from .enum_uri_type import EnumUriType from .enum_validation import EnumErrorSeverity, EnumValidationLevel, EnumValidationMode @@ -318,6 +336,13 @@ "EnumOnexReplyStatus", # Computation and processing domain "EnumComputationType", + # Contract-driven NodeCompute v1.0 domain + "EnumCaseMode", + "EnumComputeStepType", + "EnumRegexFlag", + "EnumTransformationType", + "EnumTrimMode", + "EnumUnicodeForm", # Tool lifecycle domain "EnumToolStatus", "EnumBusinessLogicPattern", diff --git a/src/omnibase_core/enums/enum_case_mode.py b/src/omnibase_core/enums/enum_case_mode.py new file mode 100644 index 0000000000..cdd642e333 --- /dev/null +++ b/src/omnibase_core/enums/enum_case_mode.py @@ -0,0 +1,22 @@ +""" +Case transformation modes for contract-driven NodeCompute. + +This module defines the case modes available for CASE_CONVERSION transformations. +""" + +from enum import Enum + + +class EnumCaseMode(str, Enum): + """ + Case transformation modes. + + Attributes: + UPPER: Convert text to uppercase. + LOWER: Convert text to lowercase. + TITLE: Convert text to titlecase. + """ + + UPPER = "uppercase" + LOWER = "lowercase" + TITLE = "titlecase" diff --git a/src/omnibase_core/enums/enum_compute_step_type.py b/src/omnibase_core/enums/enum_compute_step_type.py new file mode 100644 index 0000000000..9f38ba68b4 --- /dev/null +++ b/src/omnibase_core/enums/enum_compute_step_type.py @@ -0,0 +1,91 @@ +""" +v1.0 Pipeline step types for contract-driven NodeCompute. + +This module defines the step types available in v1.0 compute pipelines. +Each step type represents a different category of operation that can be +performed on data flowing through the pipeline. + +v1.0 Supported Types: + VALIDATION: + Data validation against a schema. v1.0 implements pass-through + behavior while logging a warning. Full schema validation with + JSON Schema support is planned for v1.1. + + TRANSFORMATION: + Applies a pure transformation function to the data. The specific + transformation is determined by the transformation_type field in + the step configuration. See EnumTransformationType for available + transformations. + + MAPPING: + Constructs a new data structure by combining values from the + pipeline input and previous step outputs using path expressions. + Useful for reshaping data or extracting specific fields. + +v1.2+ Planned Types: + CONDITIONAL (deferred): Branch execution based on conditions + PARALLEL (deferred): Execute multiple transformations concurrently + +Thread Safety: + Enum values are immutable and thread-safe. + +Example: + >>> from omnibase_core.enums import EnumComputeStepType + >>> + >>> # Check step type + >>> if step.step_type == EnumComputeStepType.TRANSFORMATION: + ... print(f"Applying {step.transformation_type} transformation") + >>> elif step.step_type == EnumComputeStepType.MAPPING: + ... print(f"Building output with {len(step.mapping_config.field_mappings)} fields") + +See Also: + - omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step: Step model + - omnibase_core.enums.enum_transformation_type: Transformation types + - omnibase_core.utils.compute_executor: Uses this enum for step dispatch +""" + +from enum import Enum + + +class EnumComputeStepType(str, Enum): + """ + v1.0 Pipeline step types for compute pipeline operations. + + Defines the categories of operations available in contract-driven compute + pipelines. Each step type has specific configuration requirements and + execution behavior. + + v1.0 includes only the three essential step types. CONDITIONAL and PARALLEL + step types are planned for v1.2+ to enable branching logic and concurrent + execution patterns. + + Attributes: + VALIDATION: Validates input data against a schema reference. In v1.0, + this is a pass-through operation that logs a warning - full schema + validation is planned for v1.1. Requires validation_config in the + step definition. + TRANSFORMATION: Applies a pure transformation function to the data. + The specific transformation is specified by the transformation_type + field (e.g., CASE_CONVERSION, REGEX). Requires transformation_type + and transformation_config (except for IDENTITY which must have no + config) in the step definition. + MAPPING: Constructs a new data structure by resolving path expressions + to values from the pipeline input or previous step outputs. Useful + for data reshaping, field extraction, and combining multiple sources. + Requires mapping_config with field_mappings in the step definition. + + Example: + >>> # Dispatch logic based on step type + >>> if step.step_type == EnumComputeStepType.TRANSFORMATION: + ... result = execute_transformation(data, step.transformation_type, step.transformation_config) + >>> elif step.step_type == EnumComputeStepType.MAPPING: + ... result = execute_mapping_step(step, input_data, step_results) + >>> elif step.step_type == EnumComputeStepType.VALIDATION: + ... result = execute_validation_step(step, data) + """ + + VALIDATION = "validation" + TRANSFORMATION = "transformation" + MAPPING = "mapping" + # v1.2+: CONDITIONAL = "conditional" + # v1.2+: PARALLEL = "parallel" diff --git a/src/omnibase_core/enums/enum_node_type.py b/src/omnibase_core/enums/enum_node_type.py index 9c968ec91c..015ef193e1 100644 --- a/src/omnibase_core/enums/enum_node_type.py +++ b/src/omnibase_core/enums/enum_node_type.py @@ -217,16 +217,11 @@ def get_node_kind(cls, node_type: EnumNodeType) -> EnumNodeKind: try: result = _KIND_MAP[node_type] except KeyError as e: - from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode - from omnibase_core.models.errors.model_onex_error import ModelOnexError - - raise ModelOnexError( - message=f"No kind mapping for node type '{node_type}'", - error_code=EnumCoreErrorCode.VALIDATION_ERROR, - context={ - "node_type": str(node_type), - "available_types": [str(k) for k in _KIND_MAP], - }, + # Use ValueError instead of ModelOnexError to avoid enum->model import + # (architectural violation: enums must not import models) + raise ValueError( + f"No kind mapping for node type '{node_type}'. " + f"Available types: {[str(k) for k in _KIND_MAP]}" ) from e else: return result diff --git a/src/omnibase_core/enums/enum_regex_flag.py b/src/omnibase_core/enums/enum_regex_flag.py new file mode 100644 index 0000000000..72a8b71ee0 --- /dev/null +++ b/src/omnibase_core/enums/enum_regex_flag.py @@ -0,0 +1,22 @@ +""" +Supported regex flags for contract-driven NodeCompute. + +This module defines the regex flags available for REGEX transformations. +""" + +from enum import Enum + + +class EnumRegexFlag(str, Enum): + """ + Supported regex flags. + + Attributes: + IGNORECASE: Case-insensitive matching. + MULTILINE: Multi-line mode (^ and $ match line boundaries). + DOTALL: Dot matches all characters including newlines. + """ + + IGNORECASE = "IGNORECASE" + MULTILINE = "MULTILINE" + DOTALL = "DOTALL" diff --git a/src/omnibase_core/enums/enum_transformation_type.py b/src/omnibase_core/enums/enum_transformation_type.py new file mode 100644 index 0000000000..1d6e7543d6 --- /dev/null +++ b/src/omnibase_core/enums/enum_transformation_type.py @@ -0,0 +1,34 @@ +""" +v1.0 Transformation types for contract-driven NodeCompute. + +This module defines the transformation types available in v1.0 compute pipelines. +Only 6 types are supported in v1.0 - collection operations are deferred to v1.1+. +""" + +from enum import Enum + + +class EnumTransformationType(str, Enum): + """ + v1.0 Transformation types. + + Only 6 types for v1.0 - collection operations deferred to v1.1+. + + Attributes: + IDENTITY: Returns input unchanged (no-op passthrough). + REGEX: Applies regex pattern replacement. + CASE_CONVERSION: Converts text case (upper/lower/title). + TRIM: Trims whitespace from text. + NORMALIZE_UNICODE: Normalizes unicode to a standard form. + JSON_PATH: Extracts data using JSONPath expression. + """ + + # v1.0 Types (6 transformations) + IDENTITY = "identity" + REGEX = "regex" + CASE_CONVERSION = "case_conversion" + TRIM = "trim" + NORMALIZE_UNICODE = "normalize_unicode" + JSON_PATH = "json_path" + # v1.1+: SPLIT, JOIN, TEMPLATE, TYPE_CONVERSION + # v1.2+: FILTER, MAP, REDUCE, SORT diff --git a/src/omnibase_core/enums/enum_trim_mode.py b/src/omnibase_core/enums/enum_trim_mode.py new file mode 100644 index 0000000000..ae1880fe5f --- /dev/null +++ b/src/omnibase_core/enums/enum_trim_mode.py @@ -0,0 +1,22 @@ +""" +Whitespace trim modes for contract-driven NodeCompute. + +This module defines the trim modes available for TRIM transformations. +""" + +from enum import Enum + + +class EnumTrimMode(str, Enum): + """ + Whitespace trim modes. + + Attributes: + BOTH: Trim whitespace from both ends. + LEFT: Trim whitespace from the left (start) only. + RIGHT: Trim whitespace from the right (end) only. + """ + + BOTH = "both" + LEFT = "left" + RIGHT = "right" diff --git a/src/omnibase_core/enums/enum_unicode_form.py b/src/omnibase_core/enums/enum_unicode_form.py new file mode 100644 index 0000000000..19685bcbb9 --- /dev/null +++ b/src/omnibase_core/enums/enum_unicode_form.py @@ -0,0 +1,24 @@ +""" +Unicode normalization forms for contract-driven NodeCompute. + +This module defines the unicode normalization forms available for NORMALIZE_UNICODE transformations. +""" + +from enum import Enum + + +class EnumUnicodeForm(str, Enum): + """ + Unicode normalization forms. + + Attributes: + NFC: Canonical Decomposition, followed by Canonical Composition. + NFD: Canonical Decomposition. + NFKC: Compatibility Decomposition, followed by Canonical Composition. + NFKD: Compatibility Decomposition. + """ + + NFC = "NFC" + NFD = "NFD" + NFKC = "NFKC" + NFKD = "NFKD" diff --git a/src/omnibase_core/errors/__init__.py b/src/omnibase_core/errors/__init__.py index 3074b5d01f..cd99e74870 100644 --- a/src/omnibase_core/errors/__init__.py +++ b/src/omnibase_core/errors/__init__.py @@ -4,6 +4,9 @@ # These symbols are re-exported via __all__ and resolved at runtime # through __getattr__ to avoid circular import dependencies. if TYPE_CHECKING: + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) from omnibase_core.errors.runtime_errors import ( ContractValidationError, EventBusError, @@ -41,6 +44,7 @@ # to avoid circular dependencies __all__ = [ + "ComputePipelineError", "ContractValidationError", "EnumCLIExitCode", "EnumCoreErrorCode", @@ -103,6 +107,13 @@ def __getattr__(name: str) -> Any: from omnibase_core.errors.runtime_errors import ContractValidationError return ContractValidationError + # Compute pipeline errors (OMN-465) + if name == "ComputePipelineError": + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + return ComputePipelineError # Raise standard AttributeError for unknown attributes # Cannot use ModelOnexError here as it would cause circular import raise AttributeError( # error-ok: avoid circular import in lazy loader diff --git a/src/omnibase_core/errors/exception_compute_pipeline_error.py b/src/omnibase_core/errors/exception_compute_pipeline_error.py new file mode 100644 index 0000000000..efefa2c115 --- /dev/null +++ b/src/omnibase_core/errors/exception_compute_pipeline_error.py @@ -0,0 +1,134 @@ +""" +Compute Pipeline Error (OMN-465). + +Custom exception for compute pipeline execution errors with structured context. + +This exception is raised when a compute pipeline step fails during execution, +providing detailed context about the failure including step name, error type, +and optional correlation tracking. + +MVP Classes: +- ComputePipelineError: Base error for all compute pipeline operations + +Error Invariants (MVP Requirements): +- All errors MUST include step_name when applicable +- All errors SHOULD include correlation_id for tracking +- Structured fields for logging and observability +- Raw stack traces MUST NOT appear in error envelopes + +Design Principles: +- Inherit from ModelOnexError for consistency +- Minimal boilerplate (leverage base class features) +- Type-safe with mypy strict mode compliance +- Serializable for event bus and logging +- No circular dependencies + +Usage: + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + # Pipeline step failure + raise ComputePipelineError( + "Transformation failed: unsupported input type", + step_name="normalize_case", + operation="transformation", + ) + + # With correlation tracking + raise ComputePipelineError( + "Validation step failed", + step_name="validate_input", + operation="validation", + correlation_id=context.correlation_id, + ) +""" + +from datetime import datetime +from typing import Any +from uuid import UUID + +from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode +from omnibase_core.enums.enum_onex_status import EnumOnexStatus +from omnibase_core.models.errors.model_onex_error import ModelOnexError + + +class ComputePipelineError(ModelOnexError): + """ + Error for compute pipeline execution failures. + + Raised when a compute pipeline step fails during execution. Includes + step_name for debugging and observability, plus optional operation type. + + Attributes: + message: Human-readable error message + error_code: Optional error code (defaults to PROCESSING_ERROR) + step_name: Name of the pipeline step that failed + operation: Optional operation type (e.g., "transformation", "mapping", "validation") + correlation_id: UUID for tracking across system (auto-generated if not provided) + **context: Additional structured context + + Example: + raise ComputePipelineError( + "Case transformation failed: input is not a string", + step_name="normalize_case", + operation="transformation", + input_type=type(input_data).__name__, + ) + """ + + # NOTE: **context: Any is intentional - accepts arbitrary structured context + # fields for logging, debugging, and observability (e.g., input_type, expected_type, etc.) + def __init__( + self, + message: str, + step_name: str | None = None, + error_code: EnumCoreErrorCode | str | None = None, + status: EnumOnexStatus = EnumOnexStatus.ERROR, + correlation_id: UUID | None = None, + timestamp: datetime | None = None, + operation: str | None = None, + **context: Any, + ) -> None: + """ + Initialize ComputePipelineError with pipeline context. + + Args: + message: Human-readable error message + step_name: Name of the pipeline step that failed (optional) + error_code: Optional error code (defaults to PROCESSING_ERROR if None) + status: Error status (default: ERROR) + correlation_id: Optional correlation ID for tracking + timestamp: Optional timestamp (auto-generated if None) + operation: Optional operation type (e.g., "transformation", "mapping") + **context: Additional structured context + """ + # Use PROCESSING_ERROR as default code if not specified + final_error_code = error_code or EnumCoreErrorCode.PROCESSING_ERROR + + # Add step_name to context if provided + if step_name is not None: + context["step_name"] = step_name + + # Add operation to context if provided + if operation is not None: + context["operation"] = operation + + # Call parent with all structured fields + super().__init__( + message=message, + error_code=final_error_code, + status=status, + correlation_id=correlation_id, + timestamp=timestamp, + **context, + ) + + # Store step_name and operation as attributes for direct access + self.step_name = step_name + self.operation = operation + + +__all__ = [ + "ComputePipelineError", +] diff --git a/src/omnibase_core/infrastructure/node_base.py b/src/omnibase_core/infrastructure/node_base.py index acbc562200..b440c601fb 100644 --- a/src/omnibase_core/infrastructure/node_base.py +++ b/src/omnibase_core/infrastructure/node_base.py @@ -81,6 +81,14 @@ class NodeBase[T_INPUT_STATE, T_OUTPUT_STATE]( - Correlation tracking for observability - Structured logging with provenance information - Signal orchestration for hub communication + + **THREAD SAFETY AND STATE**: + - All mutable state is instance-level (no global mutable state) + - Contract loading uses instance-level caching via ProtocolContractLoader + - Each NodeBase instance maintains independent state (_container, _main_tool, etc.) + - Node instances should NOT be shared across threads without synchronization + - For concurrent execution, create separate NodeBase instances per thread + - See docs/guides/THREADING.md for complete thread safety guidelines """ def __init__( @@ -722,7 +730,9 @@ def _emit_initialization_failure(self, error: Exception) -> None: LogLevel.ERROR, f"NodeBase initialization failed: {error!s}", { - "node_id": str(self.node_id) if hasattr(self, "node_id") else "unknown", + "node_id": str(self.node_id) + if hasattr(self, "node_id") and self.node_id is not None + else "unknown", "contract_path": str(self._contract_path), "error": str(error), "error_type": type(error).__name__, diff --git a/src/omnibase_core/mixins/__init__.py b/src/omnibase_core/mixins/__init__.py index 3f3145d96e..c846616d3e 100644 --- a/src/omnibase_core/mixins/__init__.py +++ b/src/omnibase_core/mixins/__init__.py @@ -24,6 +24,7 @@ MixinCanonicalYAMLSerializer, ) from omnibase_core.mixins.mixin_cli_handler import MixinCLIHandler +from omnibase_core.mixins.mixin_compute_execution import MixinComputeExecution # Models and protocols extracted from mixin_event_bus from omnibase_core.mixins.mixin_completion_data import MixinCompletionData @@ -74,6 +75,7 @@ __all__ = [ "MixinCanonicalYAMLSerializer", + "MixinComputeExecution", "MixinDiscoveryResponder", "MixinHashComputation", "MixinCLIHandler", diff --git a/src/omnibase_core/mixins/mixin_compute_execution.py b/src/omnibase_core/mixins/mixin_compute_execution.py new file mode 100644 index 0000000000..137b40aff3 --- /dev/null +++ b/src/omnibase_core/mixins/mixin_compute_execution.py @@ -0,0 +1,219 @@ +""" +Mixin for contract-driven compute pipeline execution. + +This module provides a mixin class that adds contract-driven pipeline execution +capabilities to NodeCompute instances. It wraps the synchronous pipeline executor +in an async-compatible interface and provides contract validation utilities. + +Thread Safety: + The mixin methods are stateless and operate on passed arguments only. + The underlying execute_compute_pipeline function is pure and thread-safe. + However, the mixing class (e.g., NodeCompute) may have its own thread-safety + constraints - consult the mixing class documentation. + +Usage: + This mixin is designed to be used with NodeCompute or similar node classes + that need to execute contract-driven transformation pipelines. + +Example: + >>> from omnibase_core.mixins import MixinComputeExecution + >>> from omnibase_core.nodes import NodeCompute + >>> + >>> class MyTransformNode(NodeCompute, MixinComputeExecution): + ... async def process(self, input_data): + ... result = await self.execute_contract_pipeline( + ... self.contract.compute_operations, + ... input_data.data, + ... correlation_id=input_data.correlation_id, + ... ) + ... return result + +See Also: + - omnibase_core.utils.compute_executor: Core pipeline execution logic + - omnibase_core.models.contracts.subcontracts: Contract models + - omnibase_core.nodes.node_compute: Base NodeCompute class + - docs/guides/node-building/03_COMPUTE_NODE_TUTORIAL.md: Compute node tutorial +""" + +from collections import Counter +from typing import Any +from uuid import UUID, uuid4 + +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.compute.model_compute_pipeline_result import ( + ModelComputePipelineResult, +) +from omnibase_core.models.contracts.subcontracts.model_compute_subcontract import ( + ModelComputeSubcontract, +) +from omnibase_core.utils.compute_executor import execute_compute_pipeline + + +class MixinComputeExecution: + """ + Mixin for contract-driven compute pipeline execution. + + Provides async pipeline execution and contract validation methods for + NodeCompute instances that use contract-driven transformation pipelines. + This mixin wraps the synchronous pipeline executor in an async-compatible + interface for use in async node processing methods. + + Thread Safety: + The mixin methods themselves are stateless and thread-safe. However, + the mixing class may have thread-safety constraints. When using with + NodeCompute, each instance should be used from a single thread or + with appropriate synchronization. + + Attributes: + node_id (UUID): Expected to be provided by the mixing class. Used to + populate the execution context for tracing purposes. + + Example: + >>> class MyComputeNode(NodeCompute, MixinComputeExecution): + ... async def process(self, input_data): + ... # Validate contract at startup (optional but recommended) + ... errors = self.validate_compute_contract(self.contract.compute_operations) + ... if errors: + ... raise ValueError(f"Invalid contract: {errors}") + ... + ... # Execute the pipeline + ... result = await self.execute_contract_pipeline( + ... self.contract.compute_operations, + ... input_data.data, + ... ) + ... return result + """ + + # Type hints for attributes that should exist on the mixing class + node_id: UUID + + async def execute_contract_pipeline( + self, + contract: ModelComputeSubcontract, + input_data: Any, # Any: pipeline accepts dict, Pydantic model, or JSON-compatible data + correlation_id: UUID | None = None, + ) -> ModelComputePipelineResult: + """ + Execute a contract-driven compute pipeline asynchronously. + + Wraps the synchronous execute_compute_pipeline function in an async + interface for use in async node processing methods. Creates an execution + context with operation tracking information. + + Thread Safety: + This method is async but the underlying pipeline execution is synchronous. + The function is pure and does not modify shared state, making it safe + for concurrent use from multiple async tasks. + + Args: + contract: The compute subcontract defining the pipeline steps, + transformations, and mappings to execute. + input_data: Input data to process through the pipeline. Can be a + dictionary, Pydantic model, or any JSON-compatible structure. + correlation_id: Optional UUID for distributed tracing. If provided, + will be included in the execution context for correlation with + upstream/downstream operations. + + Returns: + ModelComputePipelineResult containing: + - success: Whether all steps completed successfully + - output: Final pipeline output, or None on failure + - processing_time_ms: Total execution time + - steps_executed: List of executed step names + - step_results: Individual results for each step + - Error details if the pipeline failed + + Example: + >>> result = await self.execute_contract_pipeline( + ... self.contract.compute_operations, + ... {"text": "hello world"}, + ... correlation_id=request.correlation_id, + ... ) + >>> if result.success: + ... return result.output + ... else: + ... raise PipelineError(result.error_message) + """ + # Build execution context + context = ModelComputeExecutionContext( + operation_id=uuid4(), + correlation_id=correlation_id, + node_id=( + str(self.node_id) + if hasattr(self, "node_id") and self.node_id is not None + else None + ), + ) + + # Execute pipeline (sync function, but wrapped for async compatibility) + result = execute_compute_pipeline(contract, input_data, context) + + return result + + def validate_compute_contract(self, contract: ModelComputeSubcontract) -> list[str]: + """ + Validate a compute contract at load time. + + Performs static validation of the contract structure to catch configuration + errors before runtime execution. This should be called during node + initialization to fail fast on invalid contracts. + + Thread Safety: + This method is pure and stateless - safe for concurrent use. + + Validation Checks: + 1. Duplicate step names: Each step must have a unique name + 2. Forward references: Mapping paths can only reference steps that + execute before the current step + + Args: + contract: The compute subcontract to validate. + + Returns: + A list of validation error messages. Empty list indicates the + contract is valid. Each error message describes a specific + validation failure. + + Example: + >>> errors = self.validate_compute_contract(contract) + >>> if errors: + ... for error in errors: + ... logger.error(f"Contract validation error: {error}") + ... raise ValueError(f"Invalid contract: {len(errors)} errors found") + + Note: + This performs static analysis only. Runtime errors (e.g., type + mismatches, missing input fields) are caught during pipeline execution. + """ + errors: list[str] = [] + + # Check for duplicate step names using Counter for O(n) performance + step_names = [step.step_name for step in contract.pipeline] + name_counts = Counter(step_names) + duplicates = {name for name, count in name_counts.items() if count > 1} + if duplicates: + errors.append(f"Duplicate step names: {duplicates}") + + # Validate mapping paths reference existing steps + executed_steps: set[str] = set() + for step in contract.pipeline: + if step.mapping_config is not None: + for field, path in step.mapping_config.field_mappings.items(): + if path.startswith("$.steps."): + # Extract step name from path + remaining = path[8:] # Remove "$.steps." + ref_step = remaining.split(".")[0] + if ref_step not in executed_steps: + errors.append( + f"Step '{step.step_name}' references unknown step '{ref_step}' in mapping" + ) + executed_steps.add(step.step_name) + + return errors + + +__all__ = [ + "MixinComputeExecution", +] diff --git a/src/omnibase_core/mixins/mixin_event_handler.py b/src/omnibase_core/mixins/mixin_event_handler.py index b118101e61..f98b7d5dcc 100644 --- a/src/omnibase_core/mixins/mixin_event_handler.py +++ b/src/omnibase_core/mixins/mixin_event_handler.py @@ -109,13 +109,17 @@ def _setup_event_handlers(self) -> None: event_bus.subscribe(self._handle_introspection_request) event_bus.subscribe(self._handle_node_discovery_request) - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="_setup_event_handlers", calling_line=67, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.DEBUG, @@ -203,13 +207,17 @@ def _handle_introspection_request( ) # Emit response event (simplified - would need full implementation) - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="_handle_introspection_request", calling_line=124, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.DEBUG, @@ -218,13 +226,17 @@ def _handle_introspection_request( ) except Exception as e: - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="_handle_introspection_request", calling_line=137, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.ERROR, @@ -273,13 +285,17 @@ def _handle_node_discovery_request( try: # Respond with basic node information - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="_handle_node_discovery_request", calling_line=170, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.DEBUG, @@ -288,13 +304,17 @@ def _handle_node_discovery_request( ) except Exception as e: - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="_handle_node_discovery_request", calling_line=183, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.ERROR, @@ -320,8 +340,8 @@ def _should_respond_to_request(self, event: OnexEvent) -> bool: # Check node ID filter target_node_id = getattr(event.metadata, "target_node_id", None) if target_node_id: - node_id = getattr(self, "_node_id", "unknown") - return fnmatch.fnmatch(node_id, target_node_id) + node_id = getattr(self, "_node_id", None) or "" + return fnmatch.fnmatch(str(node_id), target_node_id) # Check node name filter target_node_name = getattr(event.metadata, "target_node_name", None) @@ -367,13 +387,17 @@ def cleanup_event_handlers(self) -> None: event_bus.unsubscribe(self._handle_introspection_request) event_bus.unsubscribe(self._handle_node_discovery_request) - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="cleanup_event_handlers", calling_line=248, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.DEBUG, @@ -381,13 +405,17 @@ def cleanup_event_handlers(self) -> None: context=context, ) except Exception as e: - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="cleanup_event_handlers", calling_line=261, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else None + ), ) emit_log_event_sync( LogLevel.WARNING, diff --git a/src/omnibase_core/mixins/mixin_introspection_publisher.py b/src/omnibase_core/mixins/mixin_introspection_publisher.py index c1c216d1fa..a207a7b6d8 100644 --- a/src/omnibase_core/mixins/mixin_introspection_publisher.py +++ b/src/omnibase_core/mixins/mixin_introspection_publisher.py @@ -47,10 +47,10 @@ def _publish_introspection_event(self) -> None: event_bus = getattr(self, "event_bus", None) if not event_bus: return - node_id = getattr(self, "_node_id", "unknown") + node_id_raw = getattr(self, "_node_id", None) + node_id = node_id_raw if node_id_raw is not None else "" try: introspection_data = self._gather_introspection_data() - from uuid import UUID, uuid4 correlation_id = uuid4() # Require explicit get_node_type() implementation - no fallback @@ -61,8 +61,14 @@ def _publish_introspection_event(self) -> None: error_code=EnumCoreErrorCode.VALIDATION_ERROR, ) node_type = self.get_node_type() + # For creating the event, we need a valid UUID - generate one if unset + event_node_id = ( + UUID(node_id) + if isinstance(node_id, str) and node_id != "" + else (node_id_raw if isinstance(node_id_raw, UUID) else uuid4()) + ) introspection_event = ModelNodeIntrospectionEvent.create_from_node_info( - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=event_node_id, node_name=introspection_data.node_name, version=introspection_data.version, node_type=node_type, @@ -77,7 +83,7 @@ def _publish_introspection_event(self) -> None: calling_function="_publish_introspection_event", calling_line=71, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=node_id_raw if isinstance(node_id_raw, UUID) else None, ) emit_log_event_sync( LogLevel.INFO, @@ -90,11 +96,11 @@ def _publish_introspection_event(self) -> None: calling_function="_publish_introspection_event", calling_line=95, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=node_id_raw if isinstance(node_id_raw, UUID) else None, ) emit_log_event_sync( LogLevel.ERROR, - f"💥 FAIL-FAST: Introspection validation failed: {e}", + f"FAIL-FAST: Introspection validation failed: {e}", context=context, ) raise @@ -104,11 +110,11 @@ def _publish_introspection_event(self) -> None: calling_function="_publish_introspection_event", calling_line=95, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=node_id_raw if isinstance(node_id_raw, UUID) else None, ) emit_log_event_sync( LogLevel.ERROR, - f"💥 FAIL-FAST: Failed to publish introspection event: {e}", + f"FAIL-FAST: Failed to publish introspection event: {e}", context=context, ) raise @@ -135,17 +141,18 @@ def _gather_introspection_data(self) -> MixinNodeIntrospectionData: ) except Exception as e: # fallback-ok: Introspection failures use fallback data with logging - node_id = getattr(self, "_node_id", "unknown") + node_id_raw = getattr(self, "_node_id", None) + node_id = node_id_raw if node_id_raw is not None else "" context = ModelLogContext( calling_module=_COMPONENT_NAME, calling_function="_gather_introspection_data", calling_line=127, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=node_id_raw if isinstance(node_id_raw, UUID) else None, ) emit_log_event_sync( LogLevel.WARNING, - f"Failed to gather full introspection data, using fallback: {e}", + f"Failed to gather full introspection data for node {node_id}, using fallback: {e}", context=context, ) return MixinNodeIntrospectionData( @@ -332,7 +339,7 @@ def _detect_health_endpoint(self) -> str | None: """Detect if this node has a health endpoint.""" try: if hasattr(self, "health_check"): - node_id = getattr(self, "_node_id", "unknown") + node_id = getattr(self, "_node_id", None) or "" return f"/health/{node_id}" except Exception: pass @@ -343,19 +350,20 @@ def _publish_with_retry(self, event: Any, max_retries: int = 3) -> None: event_bus = getattr(self, "event_bus", None) if not event_bus: return - node_id = getattr(self, "_node_id", "unknown") + node_id_raw = getattr(self, "_node_id", None) + node_id = node_id_raw if node_id_raw is not None else "" from omnibase_core.models.events.model_event_envelope import ModelEventEnvelope source_node_id_str: str - if isinstance(node_id, str): - source_node_id_str = node_id - elif isinstance(node_id, UUID): - source_node_id_str = str(node_id) + if isinstance(node_id_raw, UUID): + source_node_id_str = str(node_id_raw) + elif isinstance(node_id_raw, str): + source_node_id_str = node_id_raw else: - source_node_id_str = "unknown" + source_node_id_str = "" # Ensure source_node_id is UUID - if isinstance(node_id, UUID): - source_uuid = node_id + if isinstance(node_id_raw, UUID): + source_uuid = node_id_raw else: source_uuid = uuid4() @@ -375,7 +383,7 @@ def _publish_with_retry(self, event: Any, max_retries: int = 3) -> None: calling_function="_publish_with_retry", calling_line=350, timestamp=datetime.now().isoformat(), - node_id=UUID(node_id) if isinstance(node_id, str) else node_id, + node_id=node_id_raw if isinstance(node_id_raw, UUID) else None, ) emit_log_event_sync( LogLevel.ERROR, diff --git a/src/omnibase_core/models/cli/model_result_summary.py b/src/omnibase_core/models/cli/model_result_summary.py index c0251f3b21..84f1b5125c 100644 --- a/src/omnibase_core/models/cli/model_result_summary.py +++ b/src/omnibase_core/models/cli/model_result_summary.py @@ -33,13 +33,13 @@ class ModelResultSummary(BaseModel): target_node: str | None = Field(description="Target node name") success: bool = Field(description="Whether execution succeeded") exit_code: int = Field(description="Exit code") - duration_ms: float = Field(description="Duration in milliseconds") - retry_count: int = Field(description="Number of retries") + duration_ms: float = Field(ge=0, description="Duration in milliseconds") + retry_count: int = Field(ge=0, description="Number of retries") has_errors: bool = Field(description="Whether there are errors") has_warnings: bool = Field(description="Whether there are warnings") - error_count: int = Field(description="Number of errors") - warning_count: int = Field(description="Number of warnings") - critical_error_count: int = Field(description="Number of critical errors") + error_count: int = Field(ge=0, description="Number of errors") + warning_count: int = Field(ge=0, description="Number of warnings") + critical_error_count: int = Field(ge=0, description="Number of critical errors") model_config = { "extra": "ignore", diff --git a/src/omnibase_core/models/cli/model_trace_data.py b/src/omnibase_core/models/cli/model_trace_data.py index 3b423f2b97..e2c714d278 100644 --- a/src/omnibase_core/models/cli/model_trace_data.py +++ b/src/omnibase_core/models/cli/model_trace_data.py @@ -31,7 +31,7 @@ class ModelTraceData(BaseModel): ) start_time: datetime = Field(description="Start timestamp") end_time: datetime = Field(description="End timestamp") - duration_ms: float = Field(description="Duration in milliseconds") + duration_ms: float = Field(ge=0, description="Duration in milliseconds") tags: dict[str, str] = Field(default_factory=dict, description="Trace tags") logs: list[str] = Field(default_factory=list, description="Trace log entries") diff --git a/src/omnibase_core/models/common/model_validation_metadata.py b/src/omnibase_core/models/common/model_validation_metadata.py index 4ef2325687..a3c9aad50b 100644 --- a/src/omnibase_core/models/common/model_validation_metadata.py +++ b/src/omnibase_core/models/common/model_validation_metadata.py @@ -16,6 +16,7 @@ class ModelValidationMetadata(BaseModel): ) duration_ms: int | None = Field( default=None, + ge=0, description="Validation duration in milliseconds", ) files_processed: int | None = Field( diff --git a/src/omnibase_core/models/compute/__init__.py b/src/omnibase_core/models/compute/__init__.py new file mode 100644 index 0000000000..7973a8964c --- /dev/null +++ b/src/omnibase_core/models/compute/__init__.py @@ -0,0 +1,78 @@ +""" +Compute pipeline models for contract-driven NodeCompute v1.0. + +This module provides the core models for compute pipeline execution as defined +in the CONTRACT_DRIVEN_NODECOMPUTE_V1_0 specification. These models support +deterministic, traceable data transformation pipelines with abort-on-first-failure +semantics. + +Key Components: + ModelComputeExecutionContext: + Typed execution context carrying operation IDs, correlation IDs, and + node identification for distributed tracing and observability. + + ModelComputeStepMetadata: + Execution metadata for individual steps including timing information + and transformation type for performance analysis. + + ModelComputeStepResult: + Complete result of a single pipeline step including output data, + success status, metadata, and error details. + + ModelComputePipelineResult: + Aggregated result of entire pipeline execution with overall status, + final output, timing, and per-step results. + +Thread Safety: + All models in this module are immutable (frozen=True) after creation, + making them thread-safe for concurrent read access. + +Example: + >>> from uuid import uuid4 + >>> from omnibase_core.models.compute import ( + ... ModelComputeExecutionContext, + ... ModelComputePipelineResult, + ... ) + >>> from omnibase_core.utils.compute_executor import execute_compute_pipeline + >>> + >>> # Create execution context + >>> context = ModelComputeExecutionContext( + ... operation_id=uuid4(), + ... correlation_id=request_correlation_id, + ... ) + >>> + >>> # Execute pipeline and get result + >>> result: ModelComputePipelineResult = execute_compute_pipeline( + ... contract, input_data, context + ... ) + >>> if result.success: + ... print(f"Output: {result.output}") + ... else: + ... print(f"Error at '{result.error_step}': {result.error_message}") + +See Also: + - omnibase_core.utils.compute_executor: Pipeline execution logic + - omnibase_core.utils.compute_transformations: Transformation functions + - omnibase_core.models.contracts.subcontracts: Contract definitions + - docs/guides/node-building/03_COMPUTE_NODE_TUTORIAL.md: Compute node tutorial +""" + +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.compute.model_compute_pipeline_result import ( + ModelComputePipelineResult, +) +from omnibase_core.models.compute.model_compute_step_metadata import ( + ModelComputeStepMetadata, +) +from omnibase_core.models.compute.model_compute_step_result import ( + ModelComputeStepResult, +) + +__all__ = [ + "ModelComputeExecutionContext", + "ModelComputeStepMetadata", + "ModelComputeStepResult", + "ModelComputePipelineResult", +] diff --git a/src/omnibase_core/models/compute/model_compute_execution_context.py b/src/omnibase_core/models/compute/model_compute_execution_context.py new file mode 100644 index 0000000000..321f09c43f --- /dev/null +++ b/src/omnibase_core/models/compute/model_compute_execution_context.py @@ -0,0 +1,95 @@ +""" +Typed execution context for compute pipelines. + +This module provides the execution context model used to track and trace +compute pipeline executions. The context carries identifiers for operation +tracking, distributed tracing, and node identification. + +Thread Safety: + ModelComputeExecutionContext is immutable (frozen=True) and therefore + thread-safe for read access after creation. + +v1.0 Scope: + This is a minimal context for deterministic execution. Future versions + may add additional context fields for: + - Execution configuration overrides + - Resource limits and timeouts + - User/tenant identification + - Feature flags + +Example: + >>> from uuid import uuid4 + >>> from omnibase_core.models.compute import ModelComputeExecutionContext + >>> + >>> context = ModelComputeExecutionContext( + ... operation_id=uuid4(), + ... correlation_id=uuid4(), # From upstream request + ... node_id="node-transform-001", + ... ) + +See Also: + - omnibase_core.utils.compute_executor: Uses this context during execution + - omnibase_core.mixins.mixin_compute_execution: Creates context in async wrapper +""" + +from uuid import UUID + +from pydantic import BaseModel, ConfigDict + +__all__ = [ + "ModelComputeExecutionContext", +] + + +class ModelComputeExecutionContext(BaseModel): + """ + Typed execution context for compute pipelines. + + Provides tracing and identification information for a single pipeline + execution. This context is passed through the execution pipeline and + can be used for logging, metrics, and distributed tracing. + + Thread Safety: + This model is immutable (frozen=True) after creation, making it + safe for concurrent read access from multiple threads or async tasks. + + v1.0 Scope: + Minimal context for deterministic execution. Contains only the + essential identifiers needed for basic operation tracking. + + Attributes: + operation_id: Unique identifier for this specific operation execution. + Generated fresh for each pipeline invocation. Used for: + - Correlating log entries within a single execution + - Identifying specific executions in metrics + - Debugging failed operations + correlation_id: Optional correlation ID for distributed tracing. + Typically propagated from upstream services to enable end-to-end + request tracing across service boundaries. If not provided, + only the operation_id is available for tracking. + node_id: Optional identifier for the node executing this operation. + Useful for multi-node deployments to identify which instance + processed the request. String format for flexibility (can be + UUID, hostname, or custom identifier). + + Example: + >>> from uuid import uuid4 + >>> context = ModelComputeExecutionContext( + ... operation_id=uuid4(), + ... correlation_id=uuid4(), + ... node_id="compute-node-east-1", + ... ) + >>> # Use in pipeline execution + >>> result = execute_compute_pipeline(contract, data, context) + >>> # Access for logging + >>> logger.info( + ... "Pipeline completed", + ... extra={"operation_id": str(context.operation_id)} + ... ) + """ + + operation_id: UUID + correlation_id: UUID | None = None + node_id: str | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/compute/model_compute_pipeline_result.py b/src/omnibase_core/models/compute/model_compute_pipeline_result.py new file mode 100644 index 0000000000..78e1eb6f07 --- /dev/null +++ b/src/omnibase_core/models/compute/model_compute_pipeline_result.py @@ -0,0 +1,130 @@ +""" +Result of compute pipeline execution. + +This module provides the result model for compute pipeline executions, +aggregating all step results and providing overall success/failure status +with detailed error information when failures occur. + +Thread Safety: + ModelComputePipelineResult is immutable (frozen=True) and therefore + thread-safe for read access after creation. + +The result provides: + - Overall success/failure status + - Final output from the last successful step + - Timing metrics for performance analysis + - Individual step results for debugging + - Detailed error context on failure + +Example: + >>> from omnibase_core.utils.compute_executor import execute_compute_pipeline + >>> + >>> result = execute_compute_pipeline(contract, data, context) + >>> if result.success: + ... print(f"Output: {result.output}") + ... print(f"Time: {result.processing_time_ms:.2f}ms") + ... for step_name in result.steps_executed: + ... step = result.step_results[step_name] + ... print(f" {step_name}: {step.metadata.duration_ms:.2f}ms") + ... else: + ... print(f"Failed at '{result.error_step}': {result.error_message}") + +See Also: + - omnibase_core.utils.compute_executor: Creates this result + - omnibase_core.models.compute.model_compute_step_result: Individual step results +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any + +from pydantic import BaseModel, ConfigDict, Field + +if TYPE_CHECKING: + from omnibase_core.models.compute.model_compute_step_result import ( + ModelComputeStepResult, + ) + +__all__ = [ + "ModelComputePipelineResult", +] + + +class ModelComputePipelineResult(BaseModel): + """ + Result of compute pipeline execution. + + Aggregates the results of all executed steps and provides overall + success/failure status with detailed error information. This model + is returned by execute_compute_pipeline and contains everything + needed to understand what happened during execution. + + Thread Safety: + This model is immutable (frozen=True) after creation, making it + safe for concurrent read access from multiple threads or async tasks. + + Success Case: + When success=True, the output field contains the final result from + the last step, and all step_results will have success=True. + + Failure Case: + When success=False, the error_* fields identify what went wrong: + - error_step: Name of the step that failed + - error_type: Error classification (e.g., "validation_error") + - error_message: Human-readable description + - output: Will be None + - step_results: Contains results up to and including the failed step + + Attributes: + success: Whether the entire pipeline completed successfully. True only + if all enabled steps executed without errors. + output: Final output from the pipeline. On success, this is the output + from the last executed step. On failure, this is None. + Type is Any because it depends on the pipeline's transformation + configuration. + processing_time_ms: Total pipeline execution time in milliseconds, + measured from start to finish (including failed step if applicable). + steps_executed: List of step names that were executed, in order. + Includes all steps up to and including any failed step. + step_results: Dictionary mapping step names to their individual results. + Contains detailed timing and error information for each step. + error_type: Error classification string if pipeline failed, None otherwise. + v1.0 uses simple strings (e.g., "validation_error", "operation_failed"). + Future versions may use an enum for more structured error handling. + error_message: Human-readable error description if pipeline failed, + None otherwise. Suitable for logging and error reporting. + error_step: Name of the step where the error occurred if pipeline failed, + None otherwise. Useful for debugging and error recovery. + + Example: + >>> # Successful execution + >>> result = execute_compute_pipeline(contract, {"text": "hello"}, context) + >>> assert result.success + >>> print(result.output) # "HELLO" (if uppercase transform) + >>> print(f"Took {result.processing_time_ms:.2f}ms") + >>> + >>> # Failed execution + >>> result = execute_compute_pipeline(bad_contract, data, context) + >>> assert not result.success + >>> print(f"Failed at step '{result.error_step}'") + >>> print(f"Error: {result.error_message}") + """ + + success: bool + output: Any # Any: output type depends on pipeline configuration + processing_time_ms: float = Field(ge=0) + steps_executed: list[str] + step_results: dict[str, ModelComputeStepResult] + error_type: str | None = None # v1.0: Simple string + error_message: str | None = None + error_step: str | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) + + +# Import at runtime for forward ref resolution +from omnibase_core.models.compute.model_compute_step_result import ( + ModelComputeStepResult, +) + +ModelComputePipelineResult.model_rebuild() diff --git a/src/omnibase_core/models/compute/model_compute_step_metadata.py b/src/omnibase_core/models/compute/model_compute_step_metadata.py new file mode 100644 index 0000000000..3c0abe57af --- /dev/null +++ b/src/omnibase_core/models/compute/model_compute_step_metadata.py @@ -0,0 +1,77 @@ +""" +Metadata for a single pipeline step execution. + +This module provides the ModelComputeStepMetadata model that captures execution +metrics and observability information for individual pipeline steps. This data +is essential for performance monitoring, debugging, and optimization. + +Thread Safety: + ModelComputeStepMetadata is immutable (frozen=True) and therefore thread-safe + for read access after creation. + +The metadata captures: + - Execution timing (duration_ms) for performance analysis + - Transformation type for debugging and logging + +Example: + >>> from omnibase_core.models.compute import ModelComputeStepMetadata + >>> + >>> metadata = ModelComputeStepMetadata( + ... duration_ms=2.45, + ... transformation_type="CASE_CONVERSION", + ... ) + >>> print(f"Step took {metadata.duration_ms:.2f}ms") + +See Also: + - omnibase_core.models.compute.model_compute_step_result: Uses metadata for step results + - omnibase_core.utils.compute_executor: Creates metadata during step execution +""" + +from pydantic import BaseModel, ConfigDict, Field + +__all__ = [ + "ModelComputeStepMetadata", +] + + +class ModelComputeStepMetadata(BaseModel): + """ + Metadata for a single pipeline step execution. + + Captures timing and observability information for an individual step in the + compute pipeline. This metadata is essential for performance monitoring, + debugging slow pipelines, and understanding transformation behavior. + + Thread Safety: + This model is immutable (frozen=True) after creation, making it safe + for concurrent read access from multiple threads or async tasks. + + Performance Analysis: + The duration_ms field enables identification of slow steps in pipelines. + Aggregate step durations may be less than total pipeline time due to + overhead in result collection and context management. + + Attributes: + duration_ms: Execution time for this step in milliseconds. Measured + from step start to completion (or failure). Must be >= 0. + Typical values range from <1ms for simple transformations to + >100ms for complex operations. + transformation_type: Type of transformation applied if this is a + TRANSFORMATION step. Contains the string value of the + EnumTransformationType (e.g., "CASE_CONVERSION", "REGEX"). + None for MAPPING and VALIDATION steps. + + Example: + >>> # Analyzing step performance + >>> for step_name, result in pipeline_result.step_results.items(): + ... meta = result.metadata + ... if meta.duration_ms > 10.0: + ... print(f"Slow step: {step_name} took {meta.duration_ms:.2f}ms") + ... if meta.transformation_type: + ... print(f" Transform type: {meta.transformation_type}") + """ + + duration_ms: float = Field(ge=0) + transformation_type: str | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/compute/model_compute_step_result.py b/src/omnibase_core/models/compute/model_compute_step_result.py new file mode 100644 index 0000000000..ea767d06d7 --- /dev/null +++ b/src/omnibase_core/models/compute/model_compute_step_result.py @@ -0,0 +1,130 @@ +""" +Result of a single pipeline step. + +This module provides the ModelComputeStepResult model that captures the complete +outcome of executing a single step in a compute pipeline, including the output +data, success status, timing information, and error details if applicable. + +Thread Safety: + ModelComputeStepResult is immutable (frozen=True) and therefore thread-safe + for read access after creation. + +The step result provides: + - Output data from the step transformation + - Success/failure status + - Execution metadata (timing, transformation type) + - Error details if the step failed + +Example: + >>> from omnibase_core.models.compute import ModelComputeStepResult, ModelComputeStepMetadata + >>> + >>> # Successful step result + >>> result = ModelComputeStepResult( + ... step_name="normalize", + ... output="HELLO WORLD", + ... success=True, + ... metadata=ModelComputeStepMetadata( + ... duration_ms=1.5, + ... transformation_type="CASE_CONVERSION", + ... ), + ... ) + >>> + >>> # Failed step result + >>> failed_result = ModelComputeStepResult( + ... step_name="validate", + ... output=None, + ... success=False, + ... metadata=ModelComputeStepMetadata(duration_ms=0.3), + ... error_type="validation_error", + ... error_message="Required field 'name' is missing", + ... ) + +See Also: + - omnibase_core.models.compute.model_compute_pipeline_result: Aggregated pipeline results + - omnibase_core.models.compute.model_compute_step_metadata: Step execution metadata + - omnibase_core.utils.compute_executor: Creates step results during execution +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any + +from pydantic import BaseModel, ConfigDict + +if TYPE_CHECKING: + from omnibase_core.models.compute.model_compute_step_metadata import ( + ModelComputeStepMetadata, + ) + +__all__ = [ + "ModelComputeStepResult", +] + + +class ModelComputeStepResult(BaseModel): + """ + Result of a single pipeline step. + + Captures the complete outcome of executing one step in a compute pipeline. + This includes the transformation output, execution status, timing metadata, + and detailed error information when a step fails. + + Thread Safety: + This model is immutable (frozen=True) after creation, making it safe + for concurrent read access from multiple threads or async tasks. + + Success Case: + When success=True, the output field contains the step's transformation + result and error_type/error_message are None. + + Failure Case: + When success=False: + - output is typically None (but may contain partial results) + - error_type identifies the error category + - error_message provides a human-readable description + + Attributes: + step_name: Unique name identifying this step within the pipeline. + Used for logging, debugging, and referencing in mapping paths + (e.g., "$.steps.step_name.output"). + output: The output data from this step's transformation. Type depends + on the transformation configuration. On success, contains the + transformed data. On failure, typically None but may contain + partial results depending on the error point. + success: Whether the step completed successfully. True if the + transformation executed without errors, False otherwise. + metadata: Execution metadata including timing (duration_ms) and + transformation type. Always populated regardless of success/failure. + error_type: Error classification string if step failed, None otherwise. + v1.0 uses simple strings (e.g., "validation_error", "operation_failed"). + Future versions may use a structured enum for better error handling. + error_message: Human-readable error description if step failed, + None otherwise. Suitable for logging and user-facing error messages. + + Example: + >>> # Accessing step result in pipeline result + >>> pipeline_result = execute_compute_pipeline(contract, data, context) + >>> for step_name in pipeline_result.steps_executed: + ... step = pipeline_result.step_results[step_name] + ... if step.success: + ... print(f"{step_name}: completed in {step.metadata.duration_ms:.2f}ms") + ... else: + ... print(f"{step_name}: FAILED - {step.error_message}") + """ + + step_name: str + output: Any + success: bool = True + metadata: ModelComputeStepMetadata + error_type: str | None = None # v1.0: Simple string, not enum + error_message: str | None = None + + model_config = ConfigDict(extra="forbid", frozen=True) + + +# Import at runtime for forward ref resolution +from omnibase_core.models.compute.model_compute_step_metadata import ( + ModelComputeStepMetadata, +) + +ModelComputeStepResult.model_rebuild() diff --git a/src/omnibase_core/models/contracts/subcontracts/__init__.py b/src/omnibase_core/models/contracts/subcontracts/__init__.py index d3c3653345..f81a7840a2 100644 --- a/src/omnibase_core/models/contracts/subcontracts/__init__.py +++ b/src/omnibase_core/models/contracts/subcontracts/__init__.py @@ -36,6 +36,8 @@ from .model_circuit_breaker_subcontract import ModelCircuitBreakerSubcontract from .model_component_health import ModelComponentHealth from .model_component_health_collection import ModelComponentHealthCollection +from .model_compute_pipeline_step import ModelComputePipelineStep +from .model_compute_subcontract import ModelComputeSubcontract from .model_configuration_source import ModelConfigurationSource from .model_configuration_subcontract import ModelConfigurationSubcontract from .model_configuration_validation import ModelConfigurationValidation @@ -110,6 +112,9 @@ "ModelCachePerformance", # Circuit breaker subcontracts "ModelCircuitBreakerSubcontract", + # Compute subcontracts and components + "ModelComputePipelineStep", + "ModelComputeSubcontract", # Configuration subcontracts and components "ModelConfigurationSubcontract", "ModelConfigurationSource", diff --git a/src/omnibase_core/models/contracts/subcontracts/model_compute_pipeline_step.py b/src/omnibase_core/models/contracts/subcontracts/model_compute_pipeline_step.py new file mode 100644 index 0000000000..acb1acaccb --- /dev/null +++ b/src/omnibase_core/models/contracts/subcontracts/model_compute_pipeline_step.py @@ -0,0 +1,202 @@ +""" +A single step in the compute pipeline. + +This module defines the ModelComputePipelineStep model for v1.0 contract-driven +NodeCompute pipelines. Each step represents a discrete operation in the +sequential transformation pipeline. + +Step Types (v1.0): + VALIDATION: + Validates data against a schema. v1.0 implements pass-through behavior + while logging a warning. Full schema validation planned for v1.1. + + TRANSFORMATION: + Applies a transformation function to the data. Supports multiple + transformation types: IDENTITY, REGEX, CASE_CONVERSION, TRIM, + NORMALIZE_UNICODE, and JSON_PATH. + + MAPPING: + Builds a new data structure by combining values from the pipeline + input and/or previous step outputs using path expressions. + +Thread Safety: + ModelComputePipelineStep is immutable (frozen=True) after creation, + making it thread-safe for concurrent read access. + +Configuration Requirements: + Each step type requires specific configuration fields: + - VALIDATION: validation_config (required) + - TRANSFORMATION: transformation_type (required), transformation_config + (required for all except IDENTITY, which must NOT have config) + - MAPPING: mapping_config (required) + +Example YAML: + .. code-block:: yaml + + # Transformation step + - step_name: "normalize_text" + step_type: "transformation" + transformation_type: "case_conversion" + transformation_config: + mode: "uppercase" + + # Mapping step + - step_name: "build_output" + step_type: "mapping" + mapping_config: + field_mappings: + original_input: "$.input.text" + normalized: "$.steps.normalize_text.output" + +Example Python: + >>> from omnibase_core.models.contracts.subcontracts import ModelComputePipelineStep + >>> from omnibase_core.enums import EnumComputeStepType, EnumTransformationType + >>> from omnibase_core.models.transformations import ModelTransformCaseConfig + >>> from omnibase_core.enums import EnumCaseMode + >>> + >>> step = ModelComputePipelineStep( + ... step_name="uppercase", + ... step_type=EnumComputeStepType.TRANSFORMATION, + ... transformation_type=EnumTransformationType.CASE_CONVERSION, + ... transformation_config=ModelTransformCaseConfig(mode=EnumCaseMode.UPPER), + ... ) + +See Also: + - omnibase_core.models.contracts.subcontracts.model_compute_subcontract: Parent contract + - omnibase_core.utils.compute_executor.execute_pipeline_step: Step execution logic + - omnibase_core.enums.enum_compute_step_type: Step type enumeration +""" + +from pydantic import BaseModel, ConfigDict, model_validator + +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.models.transformations.model_mapping_config import ModelMappingConfig +from omnibase_core.models.transformations.model_validation_step_config import ( + ModelValidationStepConfig, +) +from omnibase_core.models.transformations.types import ModelTransformationConfig + + +class ModelComputePipelineStep(BaseModel): + """ + A single step in the compute pipeline. + + Represents one discrete operation in a sequential transformation pipeline. + v1.0 supports three step types: VALIDATION, TRANSFORMATION, and MAPPING. + + Thread Safety: + This model is immutable (frozen=True) after creation, making it safe + for concurrent read access from multiple threads or async tasks. + + Step Type Configuration: + Each step type requires specific configuration fields. The model validator + enforces these requirements at construction time: + + VALIDATION: + - validation_config: Required. Specifies schema_ref for validation. + + TRANSFORMATION: + - transformation_type: Required. Specifies which transformation to apply. + - transformation_config: Required for all types EXCEPT IDENTITY. + IDENTITY transformation MUST NOT have a config (enforced). + + MAPPING: + - mapping_config: Required. Specifies field_mappings with path expressions. + + Path Expression Syntax (for mapping_config): + - $.input: Full pipeline input + - $.input.: Input field access + - $.steps..output: Previous step's output + + Attributes: + step_name: Unique identifier for this step within the pipeline. Used for + logging, error messages, and path references in subsequent steps + (e.g., "$.steps.step_name.output"). Must be unique across all steps. + step_type: The category of operation this step performs. One of: + VALIDATION, TRANSFORMATION, or MAPPING. + transformation_type: For TRANSFORMATION steps only. Specifies which + transformation function to apply. One of: IDENTITY, REGEX, + CASE_CONVERSION, TRIM, NORMALIZE_UNICODE, JSON_PATH. + transformation_config: Configuration for the transformation. Required + for all transformation types except IDENTITY (which must have None). + Type depends on transformation_type (e.g., ModelTransformCaseConfig + for CASE_CONVERSION). + mapping_config: For MAPPING steps only. Contains field_mappings dict + that specifies how to build the output structure from path expressions. + validation_config: For VALIDATION steps only. Contains schema_ref + for data validation. v1.0: schema not enforced, logs warning. + enabled: Whether this step should be executed. When False, the step + is skipped and the previous step's output (or input for first step) + passes through unchanged. Defaults to True. + + Example: + >>> # TRANSFORMATION step with configuration + >>> step = ModelComputePipelineStep( + ... step_name="normalize", + ... step_type=EnumComputeStepType.TRANSFORMATION, + ... transformation_type=EnumTransformationType.CASE_CONVERSION, + ... transformation_config=ModelTransformCaseConfig(mode=EnumCaseMode.UPPER), + ... ) + >>> + >>> # IDENTITY transformation (no config allowed) + >>> identity = ModelComputePipelineStep( + ... step_name="passthrough", + ... step_type=EnumComputeStepType.TRANSFORMATION, + ... transformation_type=EnumTransformationType.IDENTITY, + ... ) + + Note: + Per-step timeout is not supported in v1.0. Use pipeline_timeout_ms on + the parent ModelComputeSubcontract for overall timeout control. + """ + + step_name: str + step_type: EnumComputeStepType + + # For transformation steps + transformation_type: EnumTransformationType | None = None + transformation_config: ModelTransformationConfig | None = None + + # For mapping steps + mapping_config: ModelMappingConfig | None = None + + # For validation steps + validation_config: ModelValidationStepConfig | None = None + + # Common options + enabled: bool = True + # v1.0: No per-step timeout - only pipeline-level timeout_ms on contract + + model_config = ConfigDict(extra="forbid", frozen=True) + + @model_validator(mode="after") + def validate_step_config(self) -> "ModelComputePipelineStep": + """Ensure correct config is provided for step type.""" + if self.step_type == EnumComputeStepType.TRANSFORMATION: + if self.transformation_type is None: + raise ValueError( + "transformation_type required for transformation steps" + ) + if ( + self.transformation_config is None + and self.transformation_type != EnumTransformationType.IDENTITY + ): + raise ValueError( + "transformation_config required for non-identity transformations" + ) + # IDENTITY must NOT have config + if ( + self.transformation_type == EnumTransformationType.IDENTITY + and self.transformation_config is not None + ): + raise ValueError( + "transformation_config must be None for IDENTITY transformations" + ) + if self.step_type == EnumComputeStepType.MAPPING: + if self.mapping_config is None: + raise ValueError("mapping_config required for mapping steps") + if self.step_type == EnumComputeStepType.VALIDATION: + if self.validation_config is None: + raise ValueError("validation_config required for validation steps") + return self diff --git a/src/omnibase_core/models/contracts/subcontracts/model_compute_subcontract.py b/src/omnibase_core/models/contracts/subcontracts/model_compute_subcontract.py new file mode 100644 index 0000000000..f375d9b03b --- /dev/null +++ b/src/omnibase_core/models/contracts/subcontracts/model_compute_subcontract.py @@ -0,0 +1,181 @@ +""" +v1.0 Compute subcontract for sequential pipeline transformations. + +This module defines the ModelComputeSubcontract model which represents the +complete compute contract for a NodeCompute node. It specifies the transformation +pipeline with abort-on-first-failure semantics, supporting deterministic and +traceable data processing workflows. + +The compute subcontract is part of the ONEX contract system and defines: + - Pipeline identity (name, version, description) + - Schema references for input/output validation + - Sequential transformation steps + - Performance constraints (timeout) + +Thread Safety: + ModelComputeSubcontract is immutable (frozen=True) after creation, + making it thread-safe for concurrent read access. + +Pipeline Execution Model: + Steps are executed sequentially in definition order. If any step fails, + the pipeline aborts immediately and returns the error. This ensures + consistent behavior and simplifies error handling. + +Example YAML Contract: + .. code-block:: yaml + + compute_operations: + version: "1.0.0" + operation_name: "text_normalizer" + operation_version: "1.0.0" + description: "Normalize text input to uppercase trimmed format" + input_schema_ref: "schemas/text_input.json" + output_schema_ref: "schemas/text_output.json" + pipeline_timeout_ms: 5000 + pipeline: + - step_name: "trim" + step_type: "transformation" + transformation_type: "TRIM" + transformation_config: + mode: "BOTH" + - step_name: "uppercase" + step_type: "transformation" + transformation_type: "CASE_CONVERSION" + transformation_config: + mode: "UPPER" + +Example Python Usage: + >>> from omnibase_core.models.contracts.subcontracts import ModelComputeSubcontract + >>> from omnibase_core.models.contracts.subcontracts import ModelComputePipelineStep + >>> from omnibase_core.enums import EnumComputeStepType, EnumTransformationType + >>> + >>> contract = ModelComputeSubcontract( + ... operation_name="text_normalizer", + ... operation_version="1.0.0", + ... description="Normalize text to uppercase", + ... pipeline=[...], # List of ModelComputePipelineStep + ... ) + +See Also: + - omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step: Step definitions + - omnibase_core.utils.compute_executor: Executes these contracts + - omnibase_core.mixins.mixin_compute_execution: Contract validation utilities + - docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md: Full specification +""" + +from pydantic import BaseModel, ConfigDict, Field, field_validator + +from omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step import ( + ModelComputePipelineStep, +) + + +class ModelComputeSubcontract(BaseModel): + """ + v1.0 Compute subcontract for sequential pipeline transformations. + + Defines transformation pipelines with abort-on-first-failure semantics. + Steps are executed sequentially in the order defined in the pipeline list. + If any step fails, the pipeline aborts immediately and returns the error. + + Thread Safety: + This model is immutable (frozen=True) after creation, making it safe + for concurrent read access from multiple threads or async tasks. + + Contract Versioning: + The version field tracks the subcontract schema version, not the operation + version. This allows schema evolution while maintaining backward compatibility + for contract parsing. + + Schema References: + input_schema_ref and output_schema_ref are resolved at contract load time + (not at execution time). v1.0 logs a warning but does not enforce schema + validation - this is planned for v1.1. + + Attributes: + version: Semantic version of the subcontract schema format. Defaults to + "1.0.0". Used for contract parser compatibility checking. + operation_name: Unique name identifying this compute operation. Should be + descriptive and follow naming conventions (e.g., "text_normalizer", + "user_data_validator"). + operation_version: Semantic version of this operation implementation. + Allows multiple versions of the same operation to coexist. + description: Human-readable description of what this operation does. + Used for documentation and debugging. Defaults to empty string. + input_schema_ref: Optional reference to a JSON schema for input validation. + Path or URI to the schema definition. v1.0: resolved but not enforced. + output_schema_ref: Optional reference to a JSON schema for output validation. + Path or URI to the schema definition. v1.0: resolved but not enforced. + pipeline: Ordered list of pipeline steps to execute. Steps run sequentially + in list order. Each step must have a unique step_name within the pipeline. + pipeline_timeout_ms: Optional maximum execution time for the entire pipeline + in milliseconds. Must be > 0 and <= 3600000 (1 hour) if specified. + v1.0: declared but not enforced. + + Example: + >>> # Minimal contract with identity transform + >>> contract = ModelComputeSubcontract( + ... operation_name="echo", + ... operation_version="1.0.0", + ... pipeline=[ + ... ModelComputePipelineStep( + ... step_name="identity", + ... step_type=EnumComputeStepType.TRANSFORMATION, + ... transformation_type=EnumTransformationType.IDENTITY, + ... ), + ... ], + ... ) + """ + + # Identity + version: str = "1.0.0" + operation_name: str + operation_version: str + description: str = "" + + # Schema references (resolved at load time) + input_schema_ref: str | None = None + output_schema_ref: str | None = None + + # Pipeline definition + pipeline: list[ModelComputePipelineStep] + + # v1.0 Performance (minimal) + # Upper bound of 3600000ms (1 hour) prevents unreasonable timeout values + pipeline_timeout_ms: int | None = Field(default=None, gt=0, le=3600000) + + model_config = ConfigDict(extra="forbid", frozen=True) + + @field_validator("pipeline") + @classmethod + def validate_unique_step_names( + cls, v: list[ModelComputePipelineStep] + ) -> list[ModelComputePipelineStep]: + """Validate that all pipeline step names are unique. + + Each step in the pipeline must have a unique step_name to enable + unambiguous path references (e.g., $.steps..output). + + Args: + v: List of pipeline steps to validate. + + Returns: + The validated list of pipeline steps. + + Raises: + ValueError: If duplicate step names are found. + """ + if v: + step_names = [step.step_name for step in v] + if len(step_names) != len(set(step_names)): + # Find duplicates for better error message + seen: set[str] = set() + duplicates: list[str] = [] + for name in step_names: + if name in seen and name not in duplicates: + duplicates.append(name) + seen.add(name) + raise ValueError( + f"Pipeline step names must be unique. Duplicates found: {duplicates}" + ) + return v diff --git a/src/omnibase_core/models/discovery/model_node_health_event.py b/src/omnibase_core/models/discovery/model_node_health_event.py index 55d426e391..6ceeaab0e6 100644 --- a/src/omnibase_core/models/discovery/model_node_health_event.py +++ b/src/omnibase_core/models/discovery/model_node_health_event.py @@ -67,6 +67,7 @@ class ModelHealthMetrics(BaseModel): ) health_check_duration_ms: float | None = Field( default=None, + ge=0, description="Duration of last health check in milliseconds", ) diff --git a/src/omnibase_core/models/events/model_intent_execution_result.py b/src/omnibase_core/models/events/model_intent_execution_result.py index e302b32c3c..432d143ab3 100644 --- a/src/omnibase_core/models/events/model_intent_execution_result.py +++ b/src/omnibase_core/models/events/model_intent_execution_result.py @@ -50,5 +50,6 @@ class ModelIntentExecutionResult(BaseModel): ) execution_duration_ms: float | None = Field( default=None, + ge=0, description="How long execution took in milliseconds", ) diff --git a/src/omnibase_core/models/model_compute_input.py b/src/omnibase_core/models/model_compute_input.py index df78e724b6..6a0ab2f545 100644 --- a/src/omnibase_core/models/model_compute_input.py +++ b/src/omnibase_core/models/model_compute_input.py @@ -1,7 +1,46 @@ """ -ModelComputeInput - Strongly typed input model for NodeCompute operations. +Strongly typed input model for NodeCompute operations. -Provides type-safe input wrapper with metadata tracking for computation operations. +This module provides the ModelComputeInput generic model that wraps computation +input data with metadata for operation tracking, caching configuration, and +parallel execution control. + +Thread Safety: + ModelComputeInput is mutable by default. If thread-safety is needed, + create the instance with all required values and treat as read-only + after creation. + +Key Features: + - Generic type parameter T_Input for type-safe input data + - Automatic operation_id generation for tracking + - Configurable caching and parallel execution + - Timestamp tracking for audit and debugging + +Example: + >>> from omnibase_core.models import ModelComputeInput + >>> + >>> # Simple input with caching enabled + >>> input_data = ModelComputeInput( + ... data={"text": "hello world"}, + ... computation_type="text_transform", + ... cache_enabled=True, + ... ) + >>> + >>> # Typed input for specific data structure + >>> from pydantic import BaseModel + >>> class UserData(BaseModel): + ... name: str + ... age: int + >>> + >>> typed_input: ModelComputeInput[UserData] = ModelComputeInput( + ... data=UserData(name="Alice", age=30), + ... computation_type="user_validation", + ... parallel_enabled=True, + ... ) + +See Also: + - omnibase_core.models.model_compute_output: Corresponding output model + - omnibase_core.nodes.node_compute: NodeCompute.process() uses this model """ from datetime import datetime @@ -19,8 +58,47 @@ class ModelComputeInput[T_Input](BaseModel): """ Input model for NodeCompute operations. - Strongly typed input wrapper that ensures type safety - and provides metadata for computation tracking. + Strongly typed input wrapper that ensures type safety and provides metadata + for computation tracking, caching control, and parallel execution configuration. + + Type Parameters: + T_Input: The type of the data being processed. Can be any type including + primitives, dictionaries, lists, or Pydantic models. + + Attributes: + data: The actual input data to process. Type is determined by the + generic parameter T_Input. + operation_id: Unique identifier for this operation instance. Auto-generated + UUID by default. Used for logging, tracing, and correlation. + computation_type: String identifier for the type of computation to perform. + Used to select the appropriate computation function from the registry. + Defaults to "default". + cache_enabled: Whether to use cached results if available. When True, + the compute node will check its cache before executing and store + results for future use. Defaults to True. + parallel_enabled: Whether to allow parallel execution for batch operations. + When True and the data supports it (e.g., list input), the compute + node may process items in parallel. Defaults to False. + metadata: Additional context metadata as key-value pairs. Can be used + for custom tracking, feature flags, or computation parameters. + timestamp: When this input was created. Auto-generated to current time. + Useful for audit trails and timeout calculations. + + Example: + >>> # Basic usage with default settings + >>> input_data = ModelComputeInput( + ... data="hello world", + ... computation_type="string_uppercase", + ... ) + >>> + >>> # With all options configured + >>> input_data = ModelComputeInput( + ... data=[1, 2, 3, 4, 5], + ... computation_type="sum_numbers", + ... cache_enabled=False, # Always recompute + ... parallel_enabled=True, # Allow parallel processing + ... metadata={"source": "user_input", "priority": "high"}, + ... ) """ data: T_Input diff --git a/src/omnibase_core/models/model_compute_output.py b/src/omnibase_core/models/model_compute_output.py index afdc533c98..285dc3a82f 100644 --- a/src/omnibase_core/models/model_compute_output.py +++ b/src/omnibase_core/models/model_compute_output.py @@ -1,13 +1,49 @@ """ -Output model for NodeCompute operations. +Strongly typed output model for NodeCompute operations. -Provides strongly typed output wrapper with computation metadata and performance metrics. +This module provides the ModelComputeOutput generic model that wraps computation +results with metadata including operation tracking, performance metrics, and +cache/parallelism information. + +Thread Safety: + ModelComputeOutput is immutable (frozen=True) after creation, making it + thread-safe for concurrent read access from multiple threads or async tasks. + +Key Features: + - Generic type parameter T_Output for type-safe result data + - Processing time measurement for performance analysis + - Cache hit tracking for optimization insights + - Parallel execution status for debugging + +Example: + >>> from omnibase_core.models import ModelComputeOutput + >>> from uuid import uuid4 + >>> + >>> # Create output after computation + >>> output = ModelComputeOutput( + ... result={"transformed": "HELLO WORLD"}, + ... operation_id=uuid4(), + ... computation_type="text_transform", + ... processing_time_ms=1.5, + ... cache_hit=False, + ... parallel_execution_used=False, + ... ) + >>> + >>> # Check performance + >>> if output.processing_time_ms > 100: + ... print(f"Slow computation: {output.processing_time_ms}ms") + >>> if output.cache_hit: + ... print("Result served from cache") + +See Also: + - omnibase_core.models.model_compute_input: Corresponding input model + - omnibase_core.nodes.node_compute: NodeCompute.process() returns this model """ from typing import Any from uuid import UUID -from pydantic import BaseModel, Field +from pydantic import BaseModel, ConfigDict, Field __all__ = [ "ModelComputeOutput", @@ -18,14 +54,50 @@ class ModelComputeOutput[T_Output](BaseModel): """ Output model for NodeCompute operations. - Strongly typed output wrapper that includes computation - metadata and performance metrics. + Strongly typed output wrapper that includes computation metadata, performance + metrics, and execution details. Created by NodeCompute.process() to return + computation results with full observability information. + + Type Parameters: + T_Output: The type of the computation result. Can be any type including + primitives, dictionaries, lists, or Pydantic models. + + Attributes: + result: The computation output data. Type is determined by the generic + parameter T_Output and the computation function's return type. + operation_id: UUID from the corresponding ModelComputeInput. Enables + correlation between input and output for tracing and debugging. + computation_type: String identifier of the computation that was executed. + Matches the computation_type from the input. + processing_time_ms: Actual execution time in milliseconds. Measured from + computation start to completion, excluding cache lookup time. + Value is 0.0 for cache hits. + cache_hit: Whether this result was retrieved from cache rather than + computed. True if the result was cached from a previous identical + computation, False if freshly computed. + parallel_execution_used: Whether parallel execution was utilized for + this computation. True if the input data was processed using the + thread pool, False for sequential execution. + metadata: Additional context metadata from the computation. May include + input size, cache status details, or computation-specific information. + + Example: + >>> # Typical output inspection + >>> output = node.process(input_data) + >>> print(f"Result: {output.result}") + >>> print(f"Time: {output.processing_time_ms:.2f}ms") + >>> if output.cache_hit: + ... print("Served from cache") + >>> if output.parallel_execution_used: + ... print("Used parallel processing") """ result: T_Output operation_id: UUID computation_type: str - processing_time_ms: float + processing_time_ms: float = Field(ge=0) cache_hit: bool = False parallel_execution_used: bool = False metadata: dict[str, Any] = Field(default_factory=dict) + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/operations/model_event_metadata.py b/src/omnibase_core/models/operations/model_event_metadata.py index 3ac21e2534..9645836217 100644 --- a/src/omnibase_core/models/operations/model_event_metadata.py +++ b/src/omnibase_core/models/operations/model_event_metadata.py @@ -30,6 +30,15 @@ class ModelEventMetadata(BaseModel): - Identifiable: UUID-based identification - Serializable: Data serialization/deserialization - Validatable: Validation and verification + + Mutability Note: + This model is intentionally mutable (not frozen) to support the execute() + protocol method which updates event processing state (processed, retry_count, + processing_duration_ms) during event handling. The validate_assignment=True + setting ensures type safety is maintained when fields are modified. + + For thread-safe access in concurrent scenarios, callers should use appropriate + synchronization mechanisms or create immutable copies when sharing across threads. """ event_id: UUID = Field( @@ -45,8 +54,10 @@ class ModelEventMetadata(BaseModel): # Event processing processed: bool = Field(default=False, description="Whether event was processed") - processing_duration_ms: int = Field(default=0, description="Processing duration") - retry_count: int = Field(default=0, description="Number of retry attempts") + processing_duration_ms: int = Field( + default=0, ge=0, description="Processing duration" + ) + retry_count: int = Field(default=0, ge=0, description="Number of retry attempts") # Event routing target_handlers: dict[str, str] = Field( diff --git a/src/omnibase_core/models/operations/model_execution_metadata.py b/src/omnibase_core/models/operations/model_execution_metadata.py index 1e37b6e275..797261df36 100644 --- a/src/omnibase_core/models/operations/model_execution_metadata.py +++ b/src/omnibase_core/models/operations/model_execution_metadata.py @@ -47,6 +47,7 @@ class ModelExecutionMetadata(BaseModel): ) duration_ms: int = Field( default=0, + ge=0, description="Execution duration in milliseconds", ) status: EnumExecutionStatusV2 = Field( @@ -77,8 +78,12 @@ class ModelExecutionMetadata(BaseModel): cpu_usage_percent: float = Field(default=0.0, description="CPU usage percentage") # Error information - error_count: int = Field(default=0, description="Number of errors encountered") - warning_count: int = Field(default=0, description="Number of warnings encountered") + error_count: int = Field( + default=0, ge=0, description="Number of errors encountered" + ) + warning_count: int = Field( + default=0, ge=0, description="Number of warnings encountered" + ) model_config = { "extra": "ignore", diff --git a/src/omnibase_core/models/transformations/__init__.py b/src/omnibase_core/models/transformations/__init__.py new file mode 100644 index 0000000000..8a01a4d282 --- /dev/null +++ b/src/omnibase_core/models/transformations/__init__.py @@ -0,0 +1,40 @@ +""" +Transformation models for contract-driven NodeCompute v1.0. + +This package provides the configuration models for all transformation types +supported in the v1.0 Contract-Driven NodeCompute specification. + +Transformation Config Models: + - ModelTransformRegexConfig: Configuration for REGEX transformations + - ModelTransformCaseConfig: Configuration for CASE_CONVERSION transformations + - ModelTransformTrimConfig: Configuration for TRIM transformations + - ModelTransformUnicodeConfig: Configuration for NORMALIZE_UNICODE transformations + - ModelTransformJsonPathConfig: Configuration for JSON_PATH transformations + +Step Config Models: + - ModelMappingConfig: Configuration for MAPPING step type + - ModelValidationStepConfig: Configuration for VALIDATION step type + +Union Types: + - ModelTransformationConfig: Discriminated union of all transformation configs +""" + +from .model_mapping_config import ModelMappingConfig +from .model_transform_case_config import ModelTransformCaseConfig +from .model_transform_json_path_config import ModelTransformJsonPathConfig +from .model_transform_regex_config import ModelTransformRegexConfig +from .model_transform_trim_config import ModelTransformTrimConfig +from .model_transform_unicode_config import ModelTransformUnicodeConfig +from .model_validation_step_config import ModelValidationStepConfig +from .types import ModelTransformationConfig + +__all__ = [ + "ModelTransformRegexConfig", + "ModelTransformCaseConfig", + "ModelTransformTrimConfig", + "ModelTransformUnicodeConfig", + "ModelTransformJsonPathConfig", + "ModelMappingConfig", + "ModelValidationStepConfig", + "ModelTransformationConfig", +] diff --git a/src/omnibase_core/models/transformations/model_mapping_config.py b/src/omnibase_core/models/transformations/model_mapping_config.py new file mode 100644 index 0000000000..472830eb7c --- /dev/null +++ b/src/omnibase_core/models/transformations/model_mapping_config.py @@ -0,0 +1,37 @@ +""" +v1.0 Mapping configuration for field mapping steps. + +This module defines the configuration model for MAPPING step types +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict, field_validator + + +class ModelMappingConfig(BaseModel): + """ + v1.0 Mapping configuration. + + field_mappings uses simple path expressions: + - $.input - Access input data + - $.steps..output - Access step output + + Attributes: + config_type: Discriminator field for union type resolution. + field_mappings: Dictionary mapping output field names to path expressions (must be non-empty). + """ + + config_type: Literal["mapping"] = "mapping" + field_mappings: dict[str, str] + + model_config = ConfigDict(extra="forbid", frozen=True) + + @field_validator("field_mappings") + @classmethod + def validate_field_mappings(cls, v: dict[str, str]) -> dict[str, str]: + """Validate that field_mappings is not empty.""" + if not v: + raise ValueError("field_mappings cannot be empty") + return v diff --git a/src/omnibase_core/models/transformations/model_transform_case_config.py b/src/omnibase_core/models/transformations/model_transform_case_config.py new file mode 100644 index 0000000000..6e22c8214e --- /dev/null +++ b/src/omnibase_core/models/transformations/model_transform_case_config.py @@ -0,0 +1,27 @@ +""" +Configuration for case transformation. + +This module defines the configuration model for CASE_CONVERSION transformations +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict + +from omnibase_core.enums.enum_case_mode import EnumCaseMode + + +class ModelTransformCaseConfig(BaseModel): + """ + Configuration for case transformation. + + Attributes: + config_type: Discriminator field for union type resolution. + mode: The case transformation mode to apply. + """ + + config_type: Literal["case_conversion"] = "case_conversion" + mode: EnumCaseMode + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/transformations/model_transform_json_path_config.py b/src/omnibase_core/models/transformations/model_transform_json_path_config.py new file mode 100644 index 0000000000..8f8519bc88 --- /dev/null +++ b/src/omnibase_core/models/transformations/model_transform_json_path_config.py @@ -0,0 +1,35 @@ +""" +Configuration for JSONPath extraction. + +This module defines the configuration model for JSON_PATH transformations +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict, field_validator + + +class ModelTransformJsonPathConfig(BaseModel): + """ + Configuration for JSONPath extraction. + + Attributes: + config_type: Discriminator field for union type resolution. + path: The JSONPath expression to extract data (must start with "$"). + """ + + config_type: Literal["json_path"] = "json_path" + path: str + + model_config = ConfigDict(extra="forbid", frozen=True) + + @field_validator("path") + @classmethod + def validate_path(cls, v: str) -> str: + """Validate that path is non-empty and starts with '$'.""" + if not v or not v.strip(): + raise ValueError("path cannot be empty") + if not v.startswith("$"): + raise ValueError("path must start with '$'") + return v diff --git a/src/omnibase_core/models/transformations/model_transform_regex_config.py b/src/omnibase_core/models/transformations/model_transform_regex_config.py new file mode 100644 index 0000000000..e5115eae79 --- /dev/null +++ b/src/omnibase_core/models/transformations/model_transform_regex_config.py @@ -0,0 +1,31 @@ +""" +Configuration for regex transformation. + +This module defines the configuration model for REGEX transformations +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict, Field + +from omnibase_core.enums.enum_regex_flag import EnumRegexFlag + + +class ModelTransformRegexConfig(BaseModel): + """ + Configuration for regex transformation. + + Attributes: + config_type: Discriminator field for union type resolution. + pattern: The regex pattern to match. + replacement: The replacement string. Empty string means deletion. + flags: List of regex flags to apply. + """ + + config_type: Literal["regex"] = "regex" + pattern: str + replacement: str = "" + flags: list[EnumRegexFlag] = Field(default_factory=list) + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/transformations/model_transform_trim_config.py b/src/omnibase_core/models/transformations/model_transform_trim_config.py new file mode 100644 index 0000000000..689983dd0f --- /dev/null +++ b/src/omnibase_core/models/transformations/model_transform_trim_config.py @@ -0,0 +1,27 @@ +""" +Configuration for whitespace trimming. + +This module defines the configuration model for TRIM transformations +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict + +from omnibase_core.enums.enum_trim_mode import EnumTrimMode + + +class ModelTransformTrimConfig(BaseModel): + """ + Configuration for whitespace trimming. + + Attributes: + config_type: Discriminator field for union type resolution. + mode: The trim mode to apply. Defaults to BOTH. + """ + + config_type: Literal["trim"] = "trim" + mode: EnumTrimMode = EnumTrimMode.BOTH + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/transformations/model_transform_unicode_config.py b/src/omnibase_core/models/transformations/model_transform_unicode_config.py new file mode 100644 index 0000000000..35d2ab5236 --- /dev/null +++ b/src/omnibase_core/models/transformations/model_transform_unicode_config.py @@ -0,0 +1,27 @@ +""" +Configuration for unicode normalization. + +This module defines the configuration model for NORMALIZE_UNICODE transformations +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict + +from omnibase_core.enums.enum_unicode_form import EnumUnicodeForm + + +class ModelTransformUnicodeConfig(BaseModel): + """ + Configuration for unicode normalization. + + Attributes: + config_type: Discriminator field for union type resolution. + form: The unicode normalization form to apply. Defaults to NFC. + """ + + config_type: Literal["normalize_unicode"] = "normalize_unicode" + form: EnumUnicodeForm = EnumUnicodeForm.NFC + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/transformations/model_validation_step_config.py b/src/omnibase_core/models/transformations/model_validation_step_config.py new file mode 100644 index 0000000000..b3e6891267 --- /dev/null +++ b/src/omnibase_core/models/transformations/model_validation_step_config.py @@ -0,0 +1,30 @@ +""" +Configuration for VALIDATION step type. + +This module defines the configuration model for VALIDATION step types +in contract-driven NodeCompute v1.0. +""" + +from typing import Literal + +from pydantic import BaseModel, ConfigDict + + +class ModelValidationStepConfig(BaseModel): + """ + Configuration for VALIDATION step type. + + NOTE: This is NOT a transformation config. It configures the VALIDATION step type. + Schema validation is semantically distinct from data transformation. + + Attributes: + config_type: Discriminator field for union type resolution. + schema_ref: Reference to the schema to validate against. + fail_on_error: If True, validation failure aborts the pipeline. Defaults to True. + """ + + config_type: Literal["validation"] = "validation" + schema_ref: str + fail_on_error: bool = True + + model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/src/omnibase_core/models/transformations/types.py b/src/omnibase_core/models/transformations/types.py new file mode 100644 index 0000000000..d0cd4c5d7a --- /dev/null +++ b/src/omnibase_core/models/transformations/types.py @@ -0,0 +1,28 @@ +""" +Discriminated union types for transformation configurations. + +This module defines the ModelTransformationConfig union type that uses +Pydantic's discriminated union feature for type-safe transformation config handling. +""" + +from typing import Annotated, Union + +from pydantic import Field + +from .model_transform_case_config import ModelTransformCaseConfig +from .model_transform_json_path_config import ModelTransformJsonPathConfig +from .model_transform_regex_config import ModelTransformRegexConfig +from .model_transform_trim_config import ModelTransformTrimConfig +from .model_transform_unicode_config import ModelTransformUnicodeConfig + +# v1.0 Discriminated union - only 5 types (IDENTITY has no config) +ModelTransformationConfig = Annotated[ + Union[ + ModelTransformRegexConfig, + ModelTransformCaseConfig, + ModelTransformTrimConfig, + ModelTransformUnicodeConfig, + ModelTransformJsonPathConfig, + ], + Field(discriminator="config_type"), +] diff --git a/src/omnibase_core/utils/compute_executor.py b/src/omnibase_core/utils/compute_executor.py new file mode 100644 index 0000000000..584cfcfe41 --- /dev/null +++ b/src/omnibase_core/utils/compute_executor.py @@ -0,0 +1,669 @@ +""" +Pipeline executor for contract-driven NodeCompute v1.0. + +This module provides the core pipeline execution logic for contract-driven +compute nodes. It executes transformation pipelines defined in YAML contracts +with abort-on-first-failure semantics, ensuring deterministic and traceable +data processing. + +Thread Safety: + All functions in this module are pure and stateless - safe for concurrent use. + Each execution operates on its own data and context without modifying shared state. + +Pipeline Execution Model: + - Steps execute sequentially in definition order + - First failure aborts the entire pipeline + - Each step's output can be referenced by subsequent steps via path expressions + - Full execution metrics are captured for observability + +Path Expression Syntax (v1.0): + - $.input: Full input object + - $.input.: Direct child field of input + - $.input..: Nested field access + - $.steps..output: Output from a previous step + +Step Types Supported: + - TRANSFORMATION: Apply a transformation function to data + - MAPPING: Build output from multiple path expressions + - VALIDATION: Validate data against schema (v1.0: pass-through) + +Example: + >>> from omnibase_core.utils.compute_executor import execute_compute_pipeline + >>> from omnibase_core.models.compute import ModelComputeExecutionContext + >>> from uuid import uuid4 + >>> + >>> context = ModelComputeExecutionContext(operation_id=uuid4()) + >>> result = execute_compute_pipeline(contract, input_data, context) + >>> if result.success: + ... print(f"Pipeline completed in {result.processing_time_ms:.2f}ms") + +See Also: + - omnibase_core.utils.compute_transformations: Transformation functions + - omnibase_core.models.contracts.subcontracts: Contract models + - omnibase_core.mixins.mixin_compute_execution: Async wrapper mixin + - docs/guides/node-building/03_COMPUTE_NODE_TUTORIAL.md: Compute node tutorial +""" + +import logging +import time +import warnings +from typing import Any + +logger = logging.getLogger(__name__) + +# Track if validation warning has been emitted for session deduplication +# TODO(OMN-497): Refactor to context-based warning tracking instead of global state. +# This global mutable state is a known exception to the "pure and stateless" principle. +# It affects only warning output (not computation results) and pipeline execution +# remains deterministic. See: docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md +_validation_warning_emitted = False + +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.compute.model_compute_pipeline_result import ( + ModelComputePipelineResult, +) +from omnibase_core.models.compute.model_compute_step_metadata import ( + ModelComputeStepMetadata, +) +from omnibase_core.models.compute.model_compute_step_result import ( + ModelComputeStepResult, +) +from omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step import ( + ModelComputePipelineStep, +) +from omnibase_core.models.contracts.subcontracts.model_compute_subcontract import ( + ModelComputeSubcontract, +) +from omnibase_core.models.errors.model_onex_error import ModelOnexError +from omnibase_core.utils.compute_transformations import execute_transformation + + +def _get_error_type(error: ModelOnexError) -> str: + """ + Extract error type string from ModelOnexError. + + Converts the error code to a string representation suitable for + inclusion in pipeline results. Handles both enum-based and string + error codes gracefully. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + error: The ModelOnexError to extract the type from. + + Returns: + A string representation of the error type. Returns "compute_error" + if no error code is present. + + Note: + The default "compute_error" is a generic fallback for pipeline errors. + In v1.1+, this will be replaced with EnumComputeErrorType. + See: docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md + """ + if error.error_code is None: + # TODO(OMN-498): Replace with EnumComputeErrorType.COMPUTE_ERROR + # See: docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md + return "compute_error" + if hasattr(error.error_code, "value"): + return str(error.error_code.value) + return str(error.error_code) + + +# TODO(OMN-499): Migrate to shared utility omnibase_core.utils.compute_path_resolver +# The shared utility has been created with resolve_pipeline_path() which provides +# equivalent functionality. Replace this function with a thin wrapper or direct import: +# from omnibase_core.utils.compute_path_resolver import resolve_pipeline_path +# See: compute_path_resolver.py for unified path resolution logic with EBNF grammar docs +def resolve_mapping_path( + path: str, + input_data: Any, # Any: accepts dict, Pydantic models, or other objects with attributes + step_results: dict[str, ModelComputeStepResult], +) -> Any: # Any: return type depends on the path being resolved + """ + Resolve a v1.0 mapping path expression to its value. + + Navigates through the pipeline's input data or previous step results + to extract the value at the specified path. This enables steps to + reference and combine data from multiple sources. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Supported Path Formats: + - $.input: Returns the full input object + - $.input.: Direct child field of input + - $.input..: Nested field access (unlimited depth) + - $.steps.: Returns the step's output (shorthand, equivalent to .output) + - $.steps..output: Returns the step's output (explicit form) + + Path Resolution Behavior: + For input paths ($.input), full nested access is supported to any depth. + For step paths ($.steps), only the step's output is accessible in v1.0. + + Note: Both `$.steps.` and `$.steps..output` return the step's + output value. The shorthand form (`$.steps.`) is provided for + convenience and is the more common usage pattern. The explicit `.output` + suffix is supported for clarity and forward compatibility (v1.1+ may + expose additional step result fields like `.metadata` or `.duration_ms`). + + Private Attribute Security: + For security reasons, private attributes (those starting with "_") are + blocked from path traversal when accessing object attributes. This prevents + exposure of internal implementation details through path expressions. + Dictionary keys starting with "_" ARE accessible since dictionaries + represent user data, not internal state. + + Args: + path: The path expression to resolve. Must start with "$". + input_data: The original pipeline input (dict, Pydantic model, or object). + step_results: Dictionary of results from previously executed steps, + keyed by step name. + + Returns: + The resolved value. Type depends on the path target. + + Raises: + ModelOnexError: If the path is invalid or cannot be resolved: + - VALIDATION_ERROR: Path doesn't start with "$", invalid prefix, + or attempts to access private attributes + - OPERATION_FAILED: Key or step not found + + Example: + >>> step_results = {"normalize": ModelComputeStepResult(..., output="HELLO")} + >>> # Both forms are equivalent: + >>> resolve_mapping_path("$.steps.normalize", {}, step_results) + 'HELLO' + >>> resolve_mapping_path("$.steps.normalize.output", {}, step_results) + 'HELLO' + """ + if not path.startswith("$"): + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"Invalid path: must start with '$', got '{path}'", + ) + + if path == "$.input" or path == "$input": + return input_data + + if path.startswith("$.input."): + # Navigate into input data + remaining = path[8:] # Remove "$.input." + parts = remaining.split(".") + current = input_data + + for part in parts: + if not part: + continue + if isinstance(current, dict): + if part not in current: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Path '{path}' not found: key '{part}' missing in input", + ) + current = current[part] + # Block private attribute access for security + elif part.startswith("_"): + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"Path '{path}' invalid: cannot access private attribute '{part}'", + ) + elif hasattr(current, part): + current = getattr(current, part) + else: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Path '{path}' not found: cannot access '{part}'", + ) + return current + + if path.startswith("$.steps."): + # Navigate into step results + remaining = path[8:] # Remove "$.steps." + parts = remaining.split(".", 1) + step_name = parts[0] + + if step_name not in step_results: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Step '{step_name}' not found in executed steps", + ) + + result = step_results[step_name] + + if len(parts) == 1: + # Shorthand: $.steps. returns output (convenience form) + return result.output + + sub_path = parts[1] + if sub_path == "output": + # Explicit: $.steps..output returns output (explicit form) + return result.output + else: + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"Invalid step path: only '.output' supported, got '.{sub_path}'", + ) + + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"Invalid path prefix: '{path}'. Must be '$.input' or '$.steps.'", + ) + + +def execute_mapping_step( + step: ModelComputePipelineStep, + input_data: Any, # Any: accepts dict, Pydantic models, or other objects with attributes + step_results: dict[str, ModelComputeStepResult], +) -> dict[ + str, Any +]: # Returns dict with field mappings; values are Any based on resolved paths + """ + Execute a mapping step, building output from path expressions. + + Mapping steps allow constructing new data structures by combining + values from the pipeline input and previous step outputs. Each field + in the output is populated by resolving a path expression. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + step: The pipeline step configuration containing the mapping definition. + input_data: The original pipeline input for resolving $.input paths. + step_results: Results from previously executed steps for resolving $.steps paths. + + Returns: + A dictionary where keys are the output field names and values are + the resolved path expressions. + + Raises: + ModelOnexError: If mapping_config is missing (VALIDATION_ERROR) or + if any path expression fails to resolve. + + Example: + >>> # With step configured to map: {"name": "$.input.user.name", "result": "$.steps.transform.output"} + >>> execute_mapping_step(step, {"user": {"name": "Alice"}}, step_results) + {"name": "Alice", "result": "TRANSFORMED_VALUE"} + """ + if step.mapping_config is None: + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message="mapping_config required for mapping step", + ) + + result: dict[str, Any] = {} + for output_field, path_expr in step.mapping_config.field_mappings.items(): + result[output_field] = resolve_mapping_path(path_expr, input_data, step_results) + + return result + + +def execute_validation_step( + step: ModelComputePipelineStep, + data: Any, # Any: validation accepts any data type for schema checking + schema_registry: dict[str, Any] + | None = None, # Any: schema definitions vary in structure +) -> Any: # Any: returns input unchanged (v1.0 pass-through) + """ + Execute a validation step against a schema. + + Validates the input data against a schema reference. In v1.0, this is + a pass-through operation as full schema validation is deferred to v1.1. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + v1.0 Limitations: + This implementation currently passes data through without validation. + Full schema validation with JSON Schema support is planned for v1.1. + + Args: + step: The pipeline step configuration containing the validation definition. + data: The data to validate. + schema_registry: Optional registry of schema definitions (reserved for v1.1). + + Returns: + The input data, unchanged (v1.0 pass-through behavior). + + Raises: + ModelOnexError: If validation_config is missing (VALIDATION_ERROR). + + Note: + A warning is logged when this function is called, as validation is + not yet implemented. See docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md + for the v1.1 validation implementation plan. + """ + if step.validation_config is None: + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message="validation_config required for validation step", + ) + + # TODO(OMN-500, v1.1): Implement schema validation for validation steps + # Target: v1.1 release + # - Integrate with schema registry for schema resolution + # - Support JSON Schema validation + # - Add validation error messages with path information + # See: docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md + + # Emit UserWarning once per session for validation steps + global _validation_warning_emitted + if not _validation_warning_emitted: + warnings.warn( + "Validation steps are pass-through in v1.0. " + "Schema validation will be implemented in v1.1.", + UserWarning, + stacklevel=2, + ) + _validation_warning_emitted = True + + # Log debug-level message for each step (for troubleshooting) + logger.debug( + "Validation step '%s' using pass-through mode (v1.0)", + step.step_name, + ) + + # v1.0: Pass through data (schema validation deferred) + return data + + +def execute_pipeline_step( + step: ModelComputePipelineStep, + current_data: Any, # Any: data flows through pipeline with varying types per step + input_data: Any, # Any: original input preserved for mapping step references + step_results: dict[str, ModelComputeStepResult], +) -> ( + Any +): # Any: output type depends on step_type (transformation, mapping, or validation) + """ + Execute a single pipeline step and return the result. + + Dispatches to the appropriate step handler based on the step type. + Supports transformation, mapping, and validation step types. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + step: The pipeline step configuration to execute. + current_data: The current data in the pipeline (output of previous step). + input_data: The original pipeline input (for mapping step references). + step_results: Results from previously executed steps. + + Returns: + The step output. Type depends on the step type: + - TRANSFORMATION: Transformed data (type depends on transformation) + - MAPPING: Dictionary of mapped fields + - VALIDATION: Input data unchanged (v1.0) + + Raises: + ModelOnexError: If the step type is unknown (OPERATION_FAILED) or + if required configuration is missing (VALIDATION_ERROR). + + Note: + If step.enabled is False, returns current_data unchanged. + """ + if not step.enabled: + return current_data + + if step.step_type == EnumComputeStepType.TRANSFORMATION: + if step.transformation_type is None: + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message="transformation_type required for transformation step", + ) + return execute_transformation( + current_data, + step.transformation_type, + step.transformation_config, + ) + + elif step.step_type == EnumComputeStepType.MAPPING: + return execute_mapping_step(step, input_data, step_results) + + elif step.step_type == EnumComputeStepType.VALIDATION: + return execute_validation_step(step, current_data) + + else: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Unknown step type: {step.step_type}", + ) + + +# TODO(OMN-501): Enforce pipeline_timeout_ms - currently declared in contract but not enforced. +# Add asyncio.timeout or threading.Timer wrapper to abort pipeline if execution exceeds timeout. +# See: docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md +def execute_compute_pipeline( + contract: ModelComputeSubcontract, + input_data: Any, # Any: pipeline input can be any JSON-compatible or Pydantic model + context: ModelComputeExecutionContext, +) -> ModelComputePipelineResult: + """ + Execute a compute pipeline with abort-on-first-failure semantics. + + Processes input data through a series of transformation, mapping, and + validation steps defined in the contract. Execution stops at the first + failure, with full error context preserved in the result. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + Each invocation operates on its own copy of the data and results. + + Execution Model: + 1. Steps are executed in definition order + 2. Disabled steps are skipped + 3. Each step receives the output of the previous step as input + 4. First failure aborts the pipeline immediately + 5. All step results and timing metrics are captured + + Args: + contract: The compute subcontract defining the pipeline steps. + input_data: Input data to process (dict, Pydantic model, or JSON-compatible). + context: Execution context with operation_id and optional correlation_id + for distributed tracing. + + Returns: + ModelComputePipelineResult containing: + - success: Whether all steps completed successfully + - output: Final pipeline output (from last step), or None on failure + - processing_time_ms: Total execution time in milliseconds + - steps_executed: List of step names that were executed + - step_results: Dictionary of individual step results + - error_type, error_message, error_step: Error details on failure + + Example: + >>> from uuid import uuid4 + >>> context = ModelComputeExecutionContext( + ... operation_id=uuid4(), + ... correlation_id=uuid4(), + ... ) + >>> result = execute_compute_pipeline(contract, {"text": "hello"}, context) + >>> if result.success: + ... print(f"Output: {result.output}") + ... print(f"Completed in {result.processing_time_ms:.2f}ms") + ... else: + ... print(f"Failed at step '{result.error_step}': {result.error_message}") + """ + start_time = time.perf_counter() + step_results: dict[str, ModelComputeStepResult] = {} + steps_executed: list[str] = [] + current_data = input_data + + for step in contract.pipeline: + if not step.enabled: + logger.debug( + "Skipping disabled step '%s' (operation_id=%s, correlation_id=%s)", + step.step_name, + context.operation_id, + context.correlation_id, + ) + continue + + step_start = time.perf_counter() + + # Log step transition for observability + logger.debug( + "Executing step '%s' (type=%s, operation_id=%s, correlation_id=%s)", + step.step_name, + step.step_type.value if step.step_type else "unknown", + context.operation_id, + context.correlation_id, + ) + + try: + result_data = execute_pipeline_step( + step, + current_data, + input_data, + step_results, + ) + + step_duration = (time.perf_counter() - step_start) * 1000 + + step_result = ModelComputeStepResult( + step_name=step.step_name, + output=result_data, + success=True, + metadata=ModelComputeStepMetadata( + duration_ms=step_duration, + transformation_type=( + step.transformation_type.value + if step.transformation_type + else None + ), + ), + ) + + step_results[step.step_name] = step_result + steps_executed.append(step.step_name) + current_data = result_data + + logger.debug( + "Step '%s' completed successfully (duration_ms=%.2f, operation_id=%s)", + step.step_name, + step_duration, + context.operation_id, + ) + + except ModelOnexError as e: + step_duration = (time.perf_counter() - step_start) * 1000 + total_time = (time.perf_counter() - start_time) * 1000 + + # Log structured error for observability + error_code_str = ( + e.error_code.value + if e.error_code is not None and hasattr(e.error_code, "value") + else str(e.error_code) + ) + logger.warning( + "Pipeline step '%s' failed: %s (error_code=%s, operation_id=%s, " + "correlation_id=%s, step_type=%s)", + step.step_name, + e.message, + error_code_str, + context.operation_id, + context.correlation_id, + step.step_type.value if step.step_type else "unknown", + ) + + # Record failed step + step_result = ModelComputeStepResult( + step_name=step.step_name, + output=None, + success=False, + metadata=ModelComputeStepMetadata( + duration_ms=step_duration, + transformation_type=( + step.transformation_type.value + if step.transformation_type + else None + ), + ), + error_type=_get_error_type(e), + error_message=e.message, + ) + step_results[step.step_name] = step_result + steps_executed.append(step.step_name) + + # Abort pipeline + return ModelComputePipelineResult( + success=False, + output=None, + processing_time_ms=total_time, + steps_executed=steps_executed, + step_results=step_results, + error_type=_get_error_type(e), + error_message=e.message, + error_step=step.step_name, + ) + + except Exception as e: + step_duration = (time.perf_counter() - step_start) * 1000 + total_time = (time.perf_counter() - start_time) * 1000 + + # Log unexpected errors for observability + # TODO(OMN-502): Wire context.correlation_id to structured logging + # See: docs/architecture/CONTRACT_DRIVEN_NODECOMPUTE_V1_0.md + logger.exception( + "Unexpected error in pipeline step '%s': %s (type: %s, " + "operation_id: %s, correlation_id: %s)", + step.step_name, + str(e), + type(e).__name__, + context.operation_id, + context.correlation_id, + ) + + step_result = ModelComputeStepResult( + step_name=step.step_name, + output=None, + success=False, + metadata=ModelComputeStepMetadata( + duration_ms=step_duration, + transformation_type=( + step.transformation_type.value + if step.transformation_type + else None + ), + ), + error_type="unexpected_error", + error_message=str(e), + ) + step_results[step.step_name] = step_result + steps_executed.append(step.step_name) + + return ModelComputePipelineResult( + success=False, + output=None, + processing_time_ms=total_time, + steps_executed=steps_executed, + step_results=step_results, + error_type="unexpected_error", + error_message=str(e), + error_step=step.step_name, + ) + + total_time = (time.perf_counter() - start_time) * 1000 + + return ModelComputePipelineResult( + success=True, + output=current_data, + processing_time_ms=total_time, + steps_executed=steps_executed, + step_results=step_results, + ) + + +__all__ = [ + "resolve_mapping_path", + "execute_mapping_step", + "execute_validation_step", + "execute_pipeline_step", + "execute_compute_pipeline", +] diff --git a/src/omnibase_core/utils/compute_path_resolver.py b/src/omnibase_core/utils/compute_path_resolver.py new file mode 100644 index 0000000000..f5d4634ff7 --- /dev/null +++ b/src/omnibase_core/utils/compute_path_resolver.py @@ -0,0 +1,562 @@ +""" +Shared path resolution utilities for contract-driven NodeCompute v1.0+. + +This module provides unified path resolution logic for navigating through nested data +structures in compute pipelines. It consolidates the path traversal patterns used by +both the pipeline executor (resolve_mapping_path) and transformations (transform_json_path). + +Thread Safety: + All functions in this module are pure and stateless - safe for concurrent use. + Each function operates only on its input parameters and produces a new result + without modifying any shared state. + +Path Expression Grammar (EBNF): + This grammar describes the path expression syntax supported by the resolver: + + ```ebnf + (* Full path expression *) + path_expr = root_marker, path_body ; + + (* Root markers *) + root_marker = "$" | "" ; (* "$" is optional for simple paths *) + + (* Path body after root *) + path_body = "" | (".", segment, { ".", segment }) ; + + (* Path segments *) + segment = identifier ; (* No array indexing in v1.0 *) + + (* Identifiers *) + identifier = letter, { letter | digit | "_" } ; + letter = "a" | ... | "z" | "A" | ... | "Z" ; + digit = "0" | ... | "9" ; + + (* Extended grammar for pipeline paths *) + pipeline_path = "$", ("input" | "steps"), [".", path_tail] ; + path_tail = segment, { ".", segment } ; + ``` + +Path Expression Syntax (v1.0): + The resolver supports two path contexts with different prefixes: + + **Simple paths** (for JSON_PATH transformation): + - `$` or empty: Returns root object + - `$.field` or `field`: Direct field access + - `$.field.subfield` or `field.subfield`: Nested access (unlimited depth) + + **Pipeline paths** (for mapping step resolution): + - `$.input` or `$input`: Returns full pipeline input object + - `$.input.`: Direct child field of input + - `$.input..`: Nested field access (unlimited depth) + - `$.steps.`: Returns step's output (shorthand form) + - `$.steps..output`: Returns step's output (explicit form) + +Path Alias Conventions: + - `$input` is equivalent to `$.input` (convenience alias) + - `$.steps.` is equivalent to `$.steps..output` (shorthand) + + The shorthand `$.steps.` form is preferred for readability. The explicit + `.output` suffix is supported for clarity and forward compatibility (v1.1+ may + expose additional step result fields like `.metadata` or `.duration_ms`). + +Private Attribute Security: + For security reasons, private attributes (those starting with "_") are blocked + from path traversal when accessing object attributes. This prevents exposure of + internal implementation details through path expressions. + + Note: Dictionary keys starting with "_" ARE accessible since dictionaries + represent user data, not internal object state. + +v1.0 Limitations: + - No array indexing: `$.items[0]` not supported + - No wildcards: `$.items[*]` not supported + - No filters: `$.items[?(@.active)]` not supported + - Step paths only support `.output` access (no `.metadata`, `.duration_ms`) + + These features are planned for v1.2. See: docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md + +Example: + >>> from omnibase_core.utils.compute_path_resolver import resolve_path + >>> data = {"user": {"name": "Alice", "profile": {"age": 30}}} + >>> resolve_path("user.profile.age", data) + 30 + >>> resolve_path("$.user.name", data) + 'Alice' + +See Also: + - omnibase_core.utils.compute_executor: Pipeline execution (uses resolve_pipeline_path) + - omnibase_core.utils.compute_transformations: JSON_PATH transformation (uses resolve_path) + - docs/guides/node-building/03_COMPUTE_NODE_TUTORIAL.md: Compute node tutorial +""" + +from typing import Any + +from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode +from omnibase_core.models.errors.model_onex_error import ModelOnexError + + +class PathResolutionError(ModelOnexError): + """ + Error raised when path resolution fails. + + This error provides structured context about the path resolution failure, + including the original path, the failing segment, and available alternatives. + + Attributes: + path: The original path expression that failed to resolve + segment: The specific path segment where resolution failed (if applicable) + available_keys: List of available keys/attributes at the failure point + """ + + def __init__( + self, + message: str, + *, + error_code: EnumCoreErrorCode = EnumCoreErrorCode.OPERATION_FAILED, + path: str | None = None, + segment: str | None = None, + available_keys: list[str] | None = None, + ) -> None: + """ + Initialize a PathResolutionError. + + Args: + message: Human-readable error description + error_code: The error code category (default: OPERATION_FAILED) + path: The original path expression that failed + segment: The specific segment where resolution failed + available_keys: List of available keys/attributes at failure point + """ + context: dict[str, Any] = {} + if path is not None: + context["path"] = path + if segment is not None: + context["segment"] = segment + if available_keys is not None: + context["available_keys"] = available_keys + + super().__init__(message=message, error_code=error_code, context=context) + + +def _validate_path_start(path: str) -> None: + """ + Validate that a pipeline path starts with the required "$" prefix. + + Pipeline paths ($.input, $.steps) require the "$" prefix to distinguish + them from simple field paths. This validation ensures proper path format. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + path: The path expression to validate. + + Raises: + PathResolutionError: If path doesn't start with "$" (VALIDATION_ERROR). + + Example: + >>> _validate_path_start("$.input.field") # OK + >>> _validate_path_start("input.field") # raises PathResolutionError + """ + if not path.startswith("$"): + raise PathResolutionError( + message=f"Invalid path: must start with '$', got '{path}'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + ) + + +def _validate_private_attribute(part: str, path: str) -> None: + """ + Validate that a path segment doesn't access private attributes. + + Private attributes (those starting with "_") are blocked from path + traversal when accessing object attributes for security reasons. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + part: The path segment (attribute name) to validate. + path: The full path expression (for error context). + + Raises: + PathResolutionError: If part starts with "_" (VALIDATION_ERROR). + + Note: + This check is only applied when accessing object attributes, + not dictionary keys. Dictionary keys starting with "_" are allowed. + + Example: + >>> _validate_private_attribute("name", "$.user.name") # OK + >>> _validate_private_attribute("_private", "$.user._private") # raises + """ + if part.startswith("_"): + raise PathResolutionError( + message=f"Cannot access private attribute: '{part}'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + segment=part, + ) + + +def _traverse_path_segments( + current: Any, + parts: list[str], + path: str, + *, + check_private: bool = True, +) -> Any: + """ + Traverse through path segments to resolve nested values. + + Navigates through nested data structures (dicts, objects) following + the provided path segments. Handles both dictionary key access and + object attribute access. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + current: The current data context to traverse from. + parts: List of path segments to traverse. + path: The original full path (for error messages). + check_private: If True, block access to private attributes on objects. + Dictionary keys are always accessible regardless of this setting. + + Returns: + The value found at the end of the path traversal. + + Raises: + PathResolutionError: If any segment cannot be resolved: + - OPERATION_FAILED: Key or attribute not found + - VALIDATION_ERROR: Attempted private attribute access (if check_private) + + Example: + >>> data = {"user": {"name": "Alice"}} + >>> _traverse_path_segments(data, ["user", "name"], "$.user.name") + 'Alice' + """ + for part in parts: + if not part: + continue + + if isinstance(current, dict): + if part not in current: + raise PathResolutionError( + message=f"Path '{path}' not found: key '{part}' missing", + error_code=EnumCoreErrorCode.OPERATION_FAILED, + path=path, + segment=part, + available_keys=list(current.keys()) if current else [], + ) + current = current[part] + else: + # For non-dict objects, check for private attribute access + if check_private: + _validate_private_attribute(part, path) + + if hasattr(current, part): + current = getattr(current, part) + else: + raise PathResolutionError( + message=f"Path '{path}' not found: cannot access '{part}' on {type(current).__name__}", + error_code=EnumCoreErrorCode.OPERATION_FAILED, + path=path, + segment=part, + ) + + return current + + +def resolve_path( + path: str, + data: Any, + *, + check_private: bool = True, +) -> Any: + """ + Resolve a simple dot-notation path to its value. + + Navigates into nested data structures using dot-notation paths to extract + specific values. Supports both dictionary access and object attribute access. + This is the core path resolution function used by JSON_PATH transformation. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Path Formats: + - `$` or `""` (empty): Returns the root object + - `$.field` or `field`: Direct field access + - `$.field.subfield` or `field.subfield`: Nested field access + + Args: + path: The path expression to resolve. The leading "$." prefix is optional. + data: The input data structure to navigate (dict, object, or nested). + check_private: If True (default), block access to private attributes + (those starting with "_") on objects. Dictionary keys are always + accessible regardless of this setting. + + Returns: + The value found at the specified path. + + Raises: + PathResolutionError: If the path cannot be resolved: + - OPERATION_FAILED: Key or attribute not found + - VALIDATION_ERROR: Attempted private attribute access + + Example: + >>> data = {"user": {"name": "Alice", "profile": {"age": 30}}} + >>> resolve_path("user.profile.age", data) + 30 + >>> resolve_path("$.user.name", data) + 'Alice' + >>> resolve_path("$", data) == data + True + """ + # Handle root-level access + if not path or path == "$": + return data + + # Normalize path: remove leading "$." or "$" if present + if path.startswith("$."): + path = path[2:] + elif path.startswith("$"): + path = path[1:] + + # Split path and traverse + parts = path.split(".") + return _traverse_path_segments(data, parts, path, check_private=check_private) + + +def resolve_input_path( + path: str, + input_data: Any, +) -> Any: + """ + Resolve a pipeline input path expression ($.input prefix). + + Extracts values from the pipeline input data using the $.input path syntax. + This function handles the input-specific path resolution for mapping steps. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Path Formats: + - `$.input` or `$input`: Returns the full input object + - `$.input.`: Direct child field of input + - `$.input..`: Nested field access (unlimited depth) + + Args: + path: The path expression starting with "$.input" or "$input". + input_data: The pipeline input data (dict, Pydantic model, or object). + + Returns: + The resolved value from the input data. + + Raises: + PathResolutionError: If the path cannot be resolved or is malformed. + + Example: + >>> input_data = {"user": {"name": "Alice"}} + >>> resolve_input_path("$.input", input_data) + {'user': {'name': 'Alice'}} + >>> resolve_input_path("$.input.user.name", input_data) + 'Alice' + """ + # Handle both $.input and $input as aliases + if path == "$.input" or path == "$input": + return input_data + + # Extract remaining path after $.input. + if path.startswith("$.input."): + remaining = path[8:] # Remove "$.input." + elif path.startswith("$input."): + remaining = path[7:] # Remove "$input." + else: + raise PathResolutionError( + message=f"Invalid input path format: '{path}'. Expected '$.input' or '$.input.'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + ) + + parts = remaining.split(".") + return _traverse_path_segments(input_data, parts, path, check_private=True) + + +def resolve_step_path( + path: str, + step_results: dict[str, Any], +) -> Any: + """ + Resolve a pipeline step path expression ($.steps prefix). + + Extracts values from previous step results using the $.steps path syntax. + This function handles the step-specific path resolution for mapping steps. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Path Formats: + - `$.steps.`: Returns the step's output (shorthand form) + - `$.steps..output`: Returns the step's output (explicit form) + + Note: + Both path forms are equivalent and return the step's output value. + The shorthand form is preferred for readability. The explicit `.output` + suffix is supported for clarity and forward compatibility. + + v1.0 Limitations: + Only `.output` access is supported in v1.0. Future versions may expose + additional step result fields like `.metadata` or `.duration_ms`. + + Args: + path: The path expression starting with "$.steps.". + step_results: Dictionary of results from previously executed steps. + Each result should have an `.output` attribute or be the output directly. + + Returns: + The resolved step output value. + + Raises: + PathResolutionError: If the step is not found or path is malformed. + + Example: + >>> from dataclasses import dataclass + >>> @dataclass + ... class StepResult: + ... output: str + >>> step_results = {"transform": StepResult(output="HELLO")} + >>> resolve_step_path("$.steps.transform", step_results) + 'HELLO' + >>> resolve_step_path("$.steps.transform.output", step_results) + 'HELLO' + """ + if not path.startswith("$.steps."): + raise PathResolutionError( + message=f"Invalid step path format: '{path}'. Expected '$.steps.'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + ) + + # Extract step name and optional subpath + remaining = path[8:] # Remove "$.steps." + parts = remaining.split(".", 1) + step_name = parts[0] + + if not step_name: + raise PathResolutionError( + message=f"Invalid step path: missing step name in '{path}'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + ) + + if step_name not in step_results: + raise PathResolutionError( + message=f"Step '{step_name}' not found in executed steps", + error_code=EnumCoreErrorCode.OPERATION_FAILED, + path=path, + segment=step_name, + available_keys=list(step_results.keys()), + ) + + result = step_results[step_name] + + # Shorthand: $.steps. returns output directly + if len(parts) == 1: + # If result has .output attribute, return it; otherwise return result directly + if hasattr(result, "output"): + return result.output + return result + + # Explicit path after step name + sub_path = parts[1] + if sub_path == "output": + if hasattr(result, "output"): + return result.output + return result + else: + raise PathResolutionError( + message=f"Invalid step path: only '.output' supported in v1.0, got '.{sub_path}'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + segment=sub_path, + ) + + +def resolve_pipeline_path( + path: str, + input_data: Any, + step_results: dict[str, Any], +) -> Any: + """ + Resolve a pipeline path expression to its value. + + This is the main entry point for resolving paths in pipeline mapping steps. + It dispatches to the appropriate resolver based on the path prefix ($.input + or $.steps). + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Supported Path Formats: + - `$.input` or `$input`: Returns the full input object + - `$.input.`: Direct child field of input + - `$.input..`: Nested field access (unlimited depth) + - `$.steps.`: Returns the step's output (shorthand form) + - `$.steps..output`: Returns the step's output (explicit form) + + Path Alias Conventions: + - `$input` is equivalent to `$.input` + - `$.steps.` is equivalent to `$.steps..output` + + Args: + path: The path expression to resolve. Must start with "$". + input_data: The original pipeline input (dict, Pydantic model, or object). + step_results: Dictionary of results from previously executed steps. + + Returns: + The resolved value. Type depends on the path target. + + Raises: + PathResolutionError: If the path is invalid or cannot be resolved. + + Example: + >>> from dataclasses import dataclass + >>> @dataclass + ... class StepResult: + ... output: str + >>> input_data = {"user": {"name": "Alice"}} + >>> step_results = {"normalize": StepResult(output="HELLO")} + >>> resolve_pipeline_path("$.input.user.name", input_data, step_results) + 'Alice' + >>> resolve_pipeline_path("$.steps.normalize", input_data, step_results) + 'HELLO' + """ + _validate_path_start(path) + + # Handle input paths (including $input alias) + if path in ("$.input", "$input") or path.startswith(("$.input.", "$input.")): + return resolve_input_path(path, input_data) + + # Handle step paths + if path.startswith("$.steps."): + return resolve_step_path(path, step_results) + + # Invalid prefix + raise PathResolutionError( + message=f"Invalid path prefix: '{path}'. Must be '$.input', '$input', or '$.steps.'", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + path=path, + ) + + +__all__ = [ + # Error type + "PathResolutionError", + # Core resolution functions + "resolve_path", + "resolve_input_path", + "resolve_step_path", + "resolve_pipeline_path", +] diff --git a/src/omnibase_core/utils/compute_transformations.py b/src/omnibase_core/utils/compute_transformations.py new file mode 100644 index 0000000000..d2cadc7a71 --- /dev/null +++ b/src/omnibase_core/utils/compute_transformations.py @@ -0,0 +1,586 @@ +""" +Pure transformation functions for contract-driven NodeCompute v1.0. + +This module provides a collection of pure transformation functions for processing +data in compute pipelines. Each function follows the pattern (data, config) -> result +with no side effects and deterministic output. + +Thread Safety: + All functions in this module are pure and stateless - safe for concurrent use. + Each function operates only on its input parameters and produces a new result + without modifying any shared state. + +Supported Transformations: + - IDENTITY: Pass-through transformation (no change) + - REGEX: Regular expression substitution + - CASE_CONVERSION: Uppercase, lowercase, title case + - TRIM: Whitespace trimming (left, right, both) + - NORMALIZE_UNICODE: Unicode normalization (NFC, NFD, NFKC, NFKD) + - JSON_PATH: Simple dot-notation path extraction + +Example: + >>> from omnibase_core.utils.compute_transformations import execute_transformation + >>> from omnibase_core.enums import EnumTransformationType + >>> from omnibase_core.models.transformations import ModelTransformCaseConfig + >>> from omnibase_core.enums import EnumCaseMode + >>> + >>> config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + >>> result = execute_transformation("hello", EnumTransformationType.CASE_CONVERSION, config) + >>> # result == "HELLO" + +See Also: + - omnibase_core.utils.compute_executor: Pipeline execution + - omnibase_core.models.transformations: Transformation config models + - docs/guides/node-building/03_COMPUTE_NODE_TUTORIAL.md: Compute node tutorial +""" + +import re +import unicodedata +from collections.abc import Callable +from typing import Any, Union + +# Type alias for JSON-compatible data structures +# Used for documenting expected data shapes in transformation functions +JsonValue = Union[str, int, float, bool, None] +JsonType = Union[dict[str, "JsonType"], list["JsonType"], JsonValue] + +# Type alias for transformation handler functions +# All handlers follow the signature (data, config) -> result for uniform registry usage +# We use Callable[..., Any] for flexibility with different config types +TransformationHandler = Callable[..., Any] + +from omnibase_core.enums.enum_case_mode import EnumCaseMode +from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode +from omnibase_core.enums.enum_regex_flag import EnumRegexFlag +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.enums.enum_trim_mode import EnumTrimMode +from omnibase_core.enums.enum_unicode_form import EnumUnicodeForm +from omnibase_core.models.errors.model_onex_error import ModelOnexError +from omnibase_core.models.transformations.model_transform_case_config import ( + ModelTransformCaseConfig, +) +from omnibase_core.models.transformations.model_transform_json_path_config import ( + ModelTransformJsonPathConfig, +) +from omnibase_core.models.transformations.model_transform_regex_config import ( + ModelTransformRegexConfig, +) +from omnibase_core.models.transformations.model_transform_trim_config import ( + ModelTransformTrimConfig, +) +from omnibase_core.models.transformations.model_transform_unicode_config import ( + ModelTransformUnicodeConfig, +) +from omnibase_core.models.transformations.types import ModelTransformationConfig + + +def _validate_string_input(value: Any, transform_name: str) -> str: + """ + Validate that input is a string type for string transformation functions. + + This is a DRY helper function used by string transformations (REGEX, + CASE_CONVERSION, TRIM, NORMALIZE_UNICODE) to validate input type + before processing. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + value: The input value to validate. + transform_name: Name of the transformation for error messaging + (e.g., "REGEX", "CASE_CONVERSION"). + + Returns: + The input value unchanged, typed as str. + + Raises: + ModelOnexError: If value is not a string (VALIDATION_ERROR). + + Example: + >>> data = _validate_string_input("hello", "CASE_CONVERSION") + >>> data + 'hello' + >>> _validate_string_input(123, "CASE_CONVERSION") # raises ModelOnexError + """ + if not isinstance(value, str): + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"{transform_name} transformation requires string input, got {type(value).__name__}", + context={ + "transform_type": transform_name, + "input_type": type(value).__name__, + "expected_type": "str", + }, + ) + return value + + +# TODO(OMN-503): Create TransformationError for more specific error handling. +# Currently uses ModelOnexError which is generic. A dedicated TransformationError would: +# - Enable more precise error handling in pipeline execution +# - Allow callers to distinguish transformation failures from other error types +# - Support structured transformation-specific error context (e.g., step_name, input_type) +# See: docs/architecture/NODECOMPUTE_VERSIONING_ROADMAP.md + + +def transform_identity( + data: Any, # Any: intentionally polymorphic - accepts any input type unchanged + config: ( + ModelTransformationConfig | None + ), # Aligned with other handlers for uniform registry usage +) -> Any: # Any: output type mirrors input type + """ + Identity transformation - returns data unchanged. + + This is a no-op transformation that passes data through without modification. + Useful as a placeholder or for testing pipeline mechanics. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Note: + The signature uses `config: ModelTransformationConfig | None` to align with + other TransformationHandler functions which all take `(data, config)`. This + maintains uniform `handler(data, config)` call pattern in the registry and + enables safer type checking when handlers are stored in the registry. + + The config parameter should be None for IDENTITY transformation - no + configuration is required or used. This is enforced at the contract level by + ModelComputePipelineStep validation, which rejects any IDENTITY step that has + transformation_config set. + + Args: + data: Any input data to pass through unchanged. + config: Should be None. IDENTITY transformation requires no configuration. + This parameter exists for uniform registry handler signature, + allowing the registry to call all handlers with `handler(data, config)`. + The type allows ModelTransformationConfig for registry compatibility, + but IDENTITY steps should always pass None. + + Returns: + The input data, unchanged. + + Example: + >>> result = transform_identity({"key": "value"}, None) + >>> result == {"key": "value"} + True + """ + # config parameter is intentionally unused - exists for registry uniformity + del config # Explicitly mark as unused to satisfy linters + return data + + +def transform_regex(data: str, config: ModelTransformRegexConfig) -> str: + """ + Apply regex substitution to string data. + + Performs a regular expression search-and-replace operation on the input string + using the pattern and replacement defined in the configuration. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + data: The input string to transform. + config: Configuration containing: + - pattern: The regex pattern to match + - replacement: The replacement string + - flags: Optional list of regex flags (IGNORECASE, MULTILINE, DOTALL) + + Returns: + The transformed string with all pattern matches replaced. + + Raises: + ModelOnexError: If data is not a string (VALIDATION_ERROR) or + if the regex pattern is invalid (OPERATION_FAILED). + + Example: + >>> from omnibase_core.models.transformations import ModelTransformRegexConfig + >>> config = ModelTransformRegexConfig(pattern=r"\\d+", replacement="NUM") + >>> transform_regex("Order 123 has 456 items", config) + 'Order NUM has NUM items' + """ + _validate_string_input(data, "REGEX") + + # Convert EnumRegexFlag to Python re flags + flags = 0 + for flag in config.flags: + if flag == EnumRegexFlag.IGNORECASE: + flags |= re.IGNORECASE + elif flag == EnumRegexFlag.MULTILINE: + flags |= re.MULTILINE + elif flag == EnumRegexFlag.DOTALL: + flags |= re.DOTALL + + try: + return re.sub(config.pattern, config.replacement, data, flags=flags) + except re.error as e: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Invalid regex pattern: {e}", + context={ + "transform_type": "REGEX", + "pattern": config.pattern, + "replacement": config.replacement, + "regex_error": str(e), + }, + ) from e + + +def transform_case(data: str, config: ModelTransformCaseConfig) -> str: + """ + Apply case transformation to string data. + + Converts the input string to the specified case format. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + data: The input string to transform. + config: Configuration containing the target case mode: + - UPPER: Convert to uppercase + - LOWER: Convert to lowercase + - TITLE: Convert to title case + + Returns: + The string converted to the specified case. + + Raises: + ModelOnexError: If data is not a string (VALIDATION_ERROR) or + if an unknown case mode is specified (OPERATION_FAILED). + + Example: + >>> from omnibase_core.models.transformations import ModelTransformCaseConfig + >>> from omnibase_core.enums import EnumCaseMode + >>> config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + >>> transform_case("hello world", config) + 'HELLO WORLD' + """ + _validate_string_input(data, "CASE_CONVERSION") + + if config.mode == EnumCaseMode.UPPER: + return data.upper() + elif config.mode == EnumCaseMode.LOWER: + return data.lower() + elif config.mode == EnumCaseMode.TITLE: + return data.title() + else: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Unknown case mode: {config.mode}", + context={ + "transform_type": "CASE_CONVERSION", + "mode": str(config.mode), + }, + ) + + +def transform_trim(data: str, config: ModelTransformTrimConfig) -> str: + """ + Trim whitespace from string data. + + Removes leading and/or trailing whitespace from the input string + based on the specified trim mode. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + data: The input string to trim. + config: Configuration containing the trim mode: + - BOTH: Remove whitespace from both ends + - LEFT: Remove leading whitespace only + - RIGHT: Remove trailing whitespace only + + Returns: + The trimmed string. + + Raises: + ModelOnexError: If data is not a string (VALIDATION_ERROR) or + if an unknown trim mode is specified (OPERATION_FAILED). + + Example: + >>> from omnibase_core.models.transformations import ModelTransformTrimConfig + >>> from omnibase_core.enums import EnumTrimMode + >>> config = ModelTransformTrimConfig(mode=EnumTrimMode.BOTH) + >>> transform_trim(" hello world ", config) + 'hello world' + """ + _validate_string_input(data, "TRIM") + + if config.mode == EnumTrimMode.BOTH: + return data.strip() + elif config.mode == EnumTrimMode.LEFT: + return data.lstrip() + elif config.mode == EnumTrimMode.RIGHT: + return data.rstrip() + else: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Unknown trim mode: {config.mode}", + context={ + "transform_type": "TRIM", + "mode": str(config.mode), + }, + ) + + +def transform_unicode(data: str, config: ModelTransformUnicodeConfig) -> str: + """ + Normalize unicode in string data. + + Applies Unicode normalization to ensure consistent character representation. + This is important for comparing strings that may contain characters with + multiple valid Unicode representations. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + data: The input string to normalize. + config: Configuration containing the normalization form: + - NFC: Canonical Decomposition, followed by Canonical Composition + - NFD: Canonical Decomposition + - NFKC: Compatibility Decomposition, followed by Canonical Composition + - NFKD: Compatibility Decomposition + + Returns: + The Unicode-normalized string. + + Raises: + ModelOnexError: If data is not a string (VALIDATION_ERROR). + + Example: + >>> from omnibase_core.models.transformations import ModelTransformUnicodeConfig + >>> from omnibase_core.enums import EnumUnicodeForm + >>> config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFC) + >>> # Normalize a string with combining characters + >>> transform_unicode("cafe\\u0301", config) # e + combining acute + 'café' # Single precomposed character (e with acute, U+00E9) + """ + _validate_string_input(data, "NORMALIZE_UNICODE") + + return unicodedata.normalize(config.form.value, data) + + +# TODO(OMN-499): Consider using shared utility omnibase_core.utils.compute_path_resolver +# The shared utility has resolve_path() which provides equivalent functionality. +# This function could be replaced with a thin wrapper that extracts config.path: +# from omnibase_core.utils.compute_path_resolver import resolve_path +# def transform_json_path(data, config): return resolve_path(config.path, data) +# See: compute_path_resolver.py for unified path resolution logic with EBNF grammar docs +def transform_json_path( + data: Any, # Any: accepts dict, Pydantic models, or other objects with attributes + config: ModelTransformJsonPathConfig, +) -> Any: # Any: return type depends on the path being resolved + """ + Extract data using simple JSONPath-like path notation. + + Navigates into nested data structures using dot-notation paths to extract + specific values. Supports both dictionary access and object attribute access. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + v1.0 Limitations: + This implementation supports only simple dot-notation paths. Array indexing, + wildcards, filters, and other advanced JSONPath features are deferred to v1.2. + + Private Attribute Security: + For security reasons, private attributes (those starting with "_") are + blocked from path traversal when accessing object attributes. This prevents + exposure of internal implementation details through path expressions. + Dictionary keys starting with "_" ARE accessible since dictionaries + represent user data, not internal state. + + Args: + data: The input data structure to navigate (dict, object, or nested structure). + config: Configuration containing the path to extract: + - "$": Root-level access (returns entire data) + - "$.field": Direct field access + - "$.field.subfield": Nested field access + Note: Path must start with "$" (validated by ModelTransformJsonPathConfig). + + Returns: + The value found at the specified path. + + Raises: + ModelOnexError: If the path cannot be resolved (key missing, attribute not found) + or if attempting to access private attributes (those starting with "_") + on objects (not dictionaries). + + Example: + >>> from omnibase_core.models.transformations import ModelTransformJsonPathConfig + >>> data = {"user": {"name": "Alice", "profile": {"age": 30}}} + >>> config = ModelTransformJsonPathConfig(path="$.user.profile.age") + >>> transform_json_path(data, config) + 30 + """ + path = config.path + + # Handle root-level access + if not path or path == "$": + return data + + # Remove leading $ if present + if path.startswith("$."): + path = path[2:] + elif path.startswith("$"): + path = path[1:] + + # Split path and traverse + parts = path.split(".") + current = data + + for part in parts: + if not part: + continue + + if isinstance(current, dict): + if part not in current: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Path '{config.path}' not found: key '{part}' missing", + context={ + "transform_type": "JSON_PATH", + "path": config.path, + "missing_key": part, + "available_keys": list(current.keys()) if current else [], + }, + ) + current = current[part] + # Block private attribute access for security + elif part.startswith("_"): + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"Cannot access private attribute: '{part}'", + context={ + "transform_type": "JSON_PATH", + "path": config.path, + "private_attribute": part, + }, + ) + elif hasattr(current, part): + current = getattr(current, part) + else: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Path '{config.path}' not found: cannot access '{part}' on {type(current).__name__}", + context={ + "transform_type": "JSON_PATH", + "path": config.path, + "missing_attribute": part, + "object_type": type(current).__name__, + }, + ) + + return current + + +# Transformation registry mapping type to handler +# TransformationHandler used because handlers have varying signatures +TRANSFORMATION_REGISTRY: dict[EnumTransformationType, TransformationHandler] = { + EnumTransformationType.IDENTITY: transform_identity, + EnumTransformationType.REGEX: transform_regex, + EnumTransformationType.CASE_CONVERSION: transform_case, + EnumTransformationType.TRIM: transform_trim, + EnumTransformationType.NORMALIZE_UNICODE: transform_unicode, + EnumTransformationType.JSON_PATH: transform_json_path, +} + + +def execute_transformation( + data: Any, # Any: input varies based on transformation_type (str for regex/case, any for json_path) + transformation_type: EnumTransformationType, + config: ModelTransformationConfig | None, +) -> Any: # Any: output type depends on transformation_type + """ + Execute a single transformation on input data. + + This is the main entry point for executing transformations. It dispatches + to the appropriate transformation function based on the transformation type. + + Thread Safety: + This function is pure and stateless - safe for concurrent use. + + Args: + data: Input data to transform. The expected type depends on the transformation: + - IDENTITY: Any type (passed through unchanged) + - REGEX, CASE_CONVERSION, TRIM, NORMALIZE_UNICODE: str + - JSON_PATH: dict, object, or nested structure + transformation_type: The type of transformation to apply, from EnumTransformationType. + config: Configuration for the transformation. Required for all types except IDENTITY. + Must match the expected config type for the transformation. + + Returns: + The transformed data. Return type depends on the transformation type. + + Raises: + ModelOnexError: If transformation fails due to: + - VALIDATION_ERROR: Wrong input type or missing required config + - OPERATION_FAILED: Unknown transformation type or transformation failure + + Example: + >>> from omnibase_core.enums import EnumTransformationType, EnumCaseMode + >>> from omnibase_core.models.transformations import ModelTransformCaseConfig + >>> + >>> config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + >>> result = execute_transformation( + ... "hello world", + ... EnumTransformationType.CASE_CONVERSION, + ... config + ... ) + >>> result + 'HELLO WORLD' + """ + handler = TRANSFORMATION_REGISTRY.get(transformation_type) + if handler is None: + raise ModelOnexError( + error_code=EnumCoreErrorCode.OPERATION_FAILED, + message=f"Unknown transformation type: {transformation_type}", + context={ + "transform_type": str(transformation_type), + "available_types": [t.value for t in TRANSFORMATION_REGISTRY], + }, + ) + + # Validate config requirements: + # - IDENTITY: config must be None (no configuration needed) + # - All others: config is required + # Contract-level validation (ModelComputePipelineStep) ensures IDENTITY steps + # never have transformation_config set, so config will be None here. + if transformation_type != EnumTransformationType.IDENTITY and config is None: + raise ModelOnexError( + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + message=f"Configuration required for {transformation_type} transformation", + context={ + "transform_type": ( + transformation_type.value + if hasattr(transformation_type, "value") + else str(transformation_type) + ), + "input_type": type(data).__name__, + }, + ) + + # Uniform handler call - all handlers take (data, config) + # For IDENTITY, config is None; for others, config is the specific config type + return handler(data, config) + + +__all__ = [ + # Type aliases + "JsonValue", + "JsonType", + "TransformationHandler", + # Transformation functions + "transform_identity", + "transform_regex", + "transform_case", + "transform_trim", + "transform_unicode", + "transform_json_path", + "execute_transformation", + "TRANSFORMATION_REGISTRY", +] diff --git a/tests/integration/conftest.py b/tests/integration/conftest.py new file mode 100644 index 0000000000..01b1a7592d --- /dev/null +++ b/tests/integration/conftest.py @@ -0,0 +1,60 @@ +"""Shared fixtures for integration tests. + +Provides reusable fixtures for: +- ModelComputeExecutionContext creation +- Common test utilities for pipeline integration tests + +These fixtures reduce test boilerplate and ensure consistent test data. + +Note: + Integration tests using these fixtures should be marked with: + - @pytest.mark.integration: For test classification + - @pytest.mark.timeout(60): For CI protection against hangs + + The integration marker is already registered in pyproject.toml. +""" + +from collections.abc import Callable +from uuid import uuid4 + +import pytest + +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) + +# Type alias for the context factory callable +ComputeContextFactory = Callable[[], ModelComputeExecutionContext] + + +@pytest.fixture +def compute_execution_context_factory() -> ComputeContextFactory: + """Factory fixture that creates a new execution context each time called. + + This fixture is used by TestComputePipelineIntegration to create unique + execution contexts for each pipeline invocation. It provides: + - Unique operation_id for each context + - Unique correlation_id for tracing + - Fresh context without accumulated state + + Useful when you need multiple distinct contexts in a single test, such as + verifying that pipeline executions are independent. + + Returns: + ComputeContextFactory: A callable that returns a new + ModelComputeExecutionContext with unique IDs each time it's called. + + Example: + def test_pipeline(self, compute_execution_context_factory): + context1 = compute_execution_context_factory() + context2 = compute_execution_context_factory() + assert context1.operation_id != context2.operation_id + """ + + def _create_context() -> ModelComputeExecutionContext: + return ModelComputeExecutionContext( + operation_id=uuid4(), + correlation_id=uuid4(), + ) + + return _create_context diff --git a/tests/integration/test_compute_pipeline_integration.py b/tests/integration/test_compute_pipeline_integration.py new file mode 100644 index 0000000000..b77804c74d --- /dev/null +++ b/tests/integration/test_compute_pipeline_integration.py @@ -0,0 +1,1781 @@ +""" +Integration tests for contract-driven NodeCompute pipeline execution. + +Tests complete pipeline scenarios including multi-step execution, +error propagation, and complex data flows. + +These tests verify: +1. Full pipeline execution with multiple chained steps +2. Error propagation and abort-on-first-failure semantics +3. Mapping steps referencing multiple prior step outputs +4. Disabled step handling +5. Large data handling performance baseline +""" + +import concurrent.futures +from uuid import uuid4 + +import pytest + +from omnibase_core.enums.enum_case_mode import EnumCaseMode +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_regex_flag import EnumRegexFlag +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.enums.enum_trim_mode import EnumTrimMode +from omnibase_core.enums.enum_unicode_form import EnumUnicodeForm +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step import ( + ModelComputePipelineStep, +) +from omnibase_core.models.contracts.subcontracts.model_compute_subcontract import ( + ModelComputeSubcontract, +) +from omnibase_core.models.transformations.model_mapping_config import ( + ModelMappingConfig, +) +from omnibase_core.models.transformations.model_transform_case_config import ( + ModelTransformCaseConfig, +) +from omnibase_core.models.transformations.model_transform_regex_config import ( + ModelTransformRegexConfig, +) +from omnibase_core.models.transformations.model_transform_trim_config import ( + ModelTransformTrimConfig, +) +from omnibase_core.models.transformations.model_transform_unicode_config import ( + ModelTransformUnicodeConfig, +) +from omnibase_core.models.transformations.model_validation_step_config import ( + ModelValidationStepConfig, +) +from omnibase_core.utils.compute_executor import execute_compute_pipeline +from tests.integration.conftest import ComputeContextFactory + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineIntegration: + """Integration tests for complete pipeline scenarios. + + Note: 60-second timeout protects against pipeline execution hangs. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_multi_step_pipeline_execution( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline with multiple transformation steps chained together. + + Verifies that: + - Multiple steps execute in sequence + - Each step receives the output of the previous step + - Final output reflects all transformations + - All steps are tracked in steps_executed + """ + # Create a pipeline: TRIM -> CASE_CONVERSION -> IDENTITY + contract = ModelComputeSubcontract( + operation_name="multi_step_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="trim_input", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="identity_pass", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data=" hello world ", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO WORLD" + assert len(result.steps_executed) == 3 + assert result.steps_executed == ["trim_input", "uppercase", "identity_pass"] + assert result.error_step is None + assert result.error_message is None + + def test_error_propagation_aborts_pipeline( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that failure in middle step aborts the pipeline. + + Verifies abort-on-first-failure semantics: + - Step 1 (IDENTITY) succeeds with integer input + - Step 2 (TRIM) fails because it requires string input + - Step 3 never executes + - Pipeline reports failure with correct error step + """ + # Create a pipeline where step 2 will fail (integer input to TRIM) + contract = ModelComputeSubcontract( + operation_name="error_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="step1_identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + # This will fail because step1 output is not a string + ModelComputePipelineStep( + step_name="step2_trim_fails", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ModelComputePipelineStep( + step_name="step3_never_reached", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + # Pass integer to cause TRIM to fail (TRIM requires string input) + result = execute_compute_pipeline( + contract=contract, + input_data=12345, + context=compute_execution_context_factory(), + ) + + assert result.success is False + assert result.error_step == "step2_trim_fails" + # step2_trim_fails is added to steps_executed even though it failed + assert "step2_trim_fails" in result.steps_executed + # step3 should never be executed + assert "step3_never_reached" not in result.steps_executed + assert result.error_message is not None + assert ( + "string" in result.error_message.lower() + or "str" in result.error_message.lower() + ) + + def test_mapping_references_multiple_prior_steps( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test mapping step that references outputs from multiple prior steps. + + Verifies that: + - Mapping can reference $.input for original input + - Mapping can reference $.steps..output for step outputs + - Multiple fields can be constructed from different sources + """ + contract = ModelComputeSubcontract( + operation_name="mapping_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="uppercase_name", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + # This step creates a mapping from prior step and input + ModelComputePipelineStep( + step_name="combine_results", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "original": "$.input", + "transformed": "$.steps.uppercase_name.output", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="hello", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert isinstance(result.output, dict) + assert result.output["original"] == "hello" + assert result.output["transformed"] == "HELLO" + + def test_disabled_steps_are_skipped( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that disabled steps are properly skipped. + + Verifies that: + - Disabled steps don't execute + - Disabled steps don't appear in steps_executed + - Pipeline continues past disabled steps + - Output reflects only enabled transformations + """ + contract = ModelComputeSubcontract( + operation_name="disabled_step_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="step1", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="step2_disabled", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.LOWER + ), + enabled=False, + ), + ModelComputePipelineStep( + step_name="step3", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="Hello", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO" # Not "hello" - step2 was skipped + assert "step2_disabled" not in result.steps_executed + assert result.steps_executed == ["step1", "step3"] + + def test_complex_pipeline_with_regex_and_unicode( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test complex pipeline with regex and unicode normalization. + + Verifies that: + - Regex transformations work correctly + - Unicode normalization integrates with other steps + - Complex multi-step pipelines execute correctly + """ + contract = ModelComputeSubcontract( + operation_name="complex_pipeline", + operation_version="1.0.0", + pipeline=[ + # Step 1: Normalize multiple spaces to single space + ModelComputePipelineStep( + step_name="normalize_spaces", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.REGEX, + transformation_config=ModelTransformRegexConfig( + pattern=r"\s+", + replacement=" ", + ), + ), + # Step 2: Trim whitespace + ModelComputePipelineStep( + step_name="trim", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + # Step 3: Convert to uppercase + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data=" hello world test ", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO WORLD TEST" + assert len(result.steps_executed) == 3 + + def test_mapping_with_nested_input_access( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test mapping step accessing nested fields in input. + + Verifies that: + - Mapping can access nested fields via $.input.field.subfield + - Complex input structures are properly traversed + """ + contract = ModelComputeSubcontract( + operation_name="nested_mapping_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="extract_fields", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "user_name": "$.input.user.name", + "user_email": "$.input.user.email", + "full_input": "$.input", + } + ), + ), + ], + ) + + input_data = { + "user": { + "name": "John Doe", + "email": "john@example.com", + }, + "metadata": { + "created": "2024-01-01", + }, + } + + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output["user_name"] == "John Doe" + assert result.output["user_email"] == "john@example.com" + assert result.output["full_input"] == input_data + + def test_chained_transformations_with_intermediate_mapping( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline with transformations and intermediate mapping. + + Verifies complex data flow: + 1. Transform input to uppercase + 2. Transform input to lowercase (on original input via mapping) + 3. Combine both in final mapping + """ + contract = ModelComputeSubcontract( + operation_name="chained_with_mapping", + operation_version="1.0.0", + pipeline=[ + # Step 1: Uppercase transformation + ModelComputePipelineStep( + step_name="to_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + # Step 2: Create intermediate mapping preserving original + ModelComputePipelineStep( + step_name="preserve_both", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "upper_version": "$.steps.to_upper.output", + "original": "$.input", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="Hello World", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output["upper_version"] == "HELLO WORLD" + assert result.output["original"] == "Hello World" + + def test_pipeline_timing_is_tracked( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that pipeline timing information is properly tracked. + + Verifies that: + - Total processing time is recorded + - Individual step durations are recorded + - All timing values are non-negative + """ + contract = ModelComputeSubcontract( + operation_name="timing_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="step1", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="step2", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data=" hello ", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.processing_time_ms >= 0 + + # Check individual step timings + for step_name in ["step1", "step2"]: + step_result = result.step_results[step_name] + assert step_result.metadata.duration_ms >= 0 + + def test_empty_pipeline_returns_input_unchanged( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that empty pipeline returns input unchanged.""" + contract = ModelComputeSubcontract( + operation_name="empty_pipeline", + operation_version="1.0.0", + pipeline=[], + ) + + input_data = {"key": "value", "nested": {"data": 123}} + + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == input_data + assert result.steps_executed == [] + + def test_regex_with_flags( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test regex transformation with IGNORECASE flag. + + Verifies that: + - IGNORECASE flag replaces all case variations + - Exactly 3 replacements occur (HELLO, hello, HeLLo) + """ + contract = ModelComputeSubcontract( + operation_name="regex_flags_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="case_insensitive_replace", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.REGEX, + transformation_config=ModelTransformRegexConfig( + pattern=r"hello", + replacement="hi", + flags=[EnumRegexFlag.IGNORECASE], + ), + ), + ], + ) + + input_data = "HELLO World, hello friend, HeLLo there" + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + + assert result.success is True + # Verify exact output - all 3 case variations replaced + assert result.output == "hi World, hi friend, hi there" + # Verify count: input has 3 "hello" variants, output has 3 "hi" + assert input_data.lower().count("hello") == 3 + assert result.output.count("hi") == 3 + + def test_unicode_normalization_in_pipeline( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test unicode normalization as part of a pipeline. + + Verifies that: + - Unicode normalization works correctly + - Can be combined with other transformations + """ + contract = ModelComputeSubcontract( + operation_name="unicode_pipeline", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="normalize", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.NORMALIZE_UNICODE, + transformation_config=ModelTransformUnicodeConfig( + form=EnumUnicodeForm.NFC + ), + ), + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + # Use decomposed form (e + combining accent) + decomposed = "cafe\u0301" # cafe with combining acute accent on e + + result = execute_compute_pipeline( + contract=contract, + input_data=decomposed, + context=compute_execution_context_factory(), + ) + + assert result.success is True + # Should be normalized and uppercased + assert result.output == "CAF\xc9" # CAFE with composed E-acute + + @pytest.mark.slow + def test_large_data_performance_baseline( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline performance with larger data sets. + + Verifies that: + - Pipeline handles large inputs without issues + - Processing completes in reasonable time + - Output is correct for large data + """ + # Create a simple pipeline + contract = ModelComputeSubcontract( + operation_name="performance_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="trim", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + # Generate large input (100KB of text) + large_input = " " + "a" * 100000 + " " + + result = execute_compute_pipeline( + contract=contract, + input_data=large_input, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert len(result.output) == 100000 + assert result.output == "A" * 100000 + # Performance baseline: should complete in reasonable time (under 1 second) + assert result.processing_time_ms < 1000 + + @pytest.mark.slow + def test_many_steps_pipeline( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline with many steps. + + Verifies that: + - Pipeline can handle many sequential steps + - All steps are tracked correctly + - Performance remains reasonable + """ + # Create a pipeline with many identity steps + steps = [] + for i in range(50): + steps.append( + ModelComputePipelineStep( + step_name=f"step_{i:02d}", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + ) + + # Add a final transformation to verify data flows through + steps.append( + ModelComputePipelineStep( + step_name="final_uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig(mode=EnumCaseMode.UPPER), + ) + ) + + contract = ModelComputeSubcontract( + operation_name="many_steps_test", + operation_version="1.0.0", + pipeline=steps, + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="hello world", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO WORLD" + assert len(result.steps_executed) == 51 + # Performance: should complete quickly despite many steps + assert result.processing_time_ms < 500 + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineErrorScenarios: + """Integration tests for error handling scenarios. + + Note: 60-second timeout protects against pipeline execution hangs. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_invalid_regex_pattern_fails_gracefully( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that invalid regex pattern causes proper failure.""" + contract = ModelComputeSubcontract( + operation_name="invalid_regex", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="bad_regex", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.REGEX, + transformation_config=ModelTransformRegexConfig( + pattern=r"[invalid", # Unclosed bracket + replacement="x", + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="test input", + context=compute_execution_context_factory(), + ) + + assert result.success is False + assert result.error_step == "bad_regex" + assert result.error_message is not None + + def test_missing_mapping_path_fails( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that referencing non-existent path in mapping fails.""" + contract = ModelComputeSubcontract( + operation_name="missing_path", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="bad_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "missing": "$.input.nonexistent.field", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data={"actual": "data"}, + context=compute_execution_context_factory(), + ) + + assert result.success is False + assert result.error_step == "bad_mapping" + assert result.error_message is not None + + def test_referencing_nonexistent_step_fails( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that referencing non-existent step in mapping fails.""" + contract = ModelComputeSubcontract( + operation_name="missing_step", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="bad_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "missing": "$.steps.nonexistent_step.output", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="test", + context=compute_execution_context_factory(), + ) + + assert result.success is False + assert result.error_step == "bad_mapping" + assert ( + "nonexistent_step" in result.error_message + or "not found" in result.error_message.lower() + ) + + def test_type_mismatch_in_middle_step( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that type mismatch in middle of pipeline aborts correctly.""" + contract = ModelComputeSubcontract( + operation_name="type_mismatch", + operation_version="1.0.0", + pipeline=[ + # Step 1: Create a mapping (outputs dict) + ModelComputePipelineStep( + step_name="create_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "value": "$.input", + } + ), + ), + # Step 2: Try to uppercase (expects string, gets dict) + ModelComputePipelineStep( + step_name="uppercase_fails", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + # Step 3: Should never be reached + ModelComputePipelineStep( + step_name="never_reached", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="test input", + context=compute_execution_context_factory(), + ) + + assert result.success is False + assert result.error_step == "uppercase_fails" + assert "never_reached" not in result.steps_executed + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineContextTracking: + """Integration tests for context and correlation ID tracking. + + Note: 60-second timeout protects against pipeline execution hangs. + """ + + def test_context_ids_are_preserved( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that operation and correlation IDs are available in context.""" + operation_id = uuid4() + correlation_id = uuid4() + + context = ModelComputeExecutionContext( + operation_id=operation_id, + correlation_id=correlation_id, + node_id="test-node-001", + ) + + contract = ModelComputeSubcontract( + operation_name="context_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="test", + context=context, + ) + + assert result.success is True + # Context should be passed through (though we can't directly inspect it + # from the result, we verify the pipeline executed successfully with it) + + def test_context_without_correlation_id( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that context works without optional correlation ID.""" + context = ModelComputeExecutionContext( + operation_id=uuid4(), + # correlation_id is optional + ) + + contract = ModelComputeSubcontract( + operation_name="minimal_context_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="test", + context=context, + ) + + assert result.success is True + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineSingleStepScenarios: + """Integration tests for single-step pipeline edge cases. + + Note: 60-second timeout protects against pipeline execution hangs. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_single_transformation_step( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline with exactly one transformation step. + + Verifies that: + - Single step executes correctly + - Output reflects the transformation + - Steps executed list contains exactly one step + """ + contract = ModelComputeSubcontract( + operation_name="single_step_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="only_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="hello", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO" + assert len(result.steps_executed) == 1 + assert result.steps_executed == ["only_step"] + assert "only_step" in result.step_results + + def test_single_mapping_step( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline with exactly one mapping step. + + Verifies that: + - Mapping step can access input directly + - Single mapping produces correct output structure + """ + contract = ModelComputeSubcontract( + operation_name="single_mapping_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="only_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "wrapped_input": "$.input", + "user_name": "$.input.user.name", + } + ), + ), + ], + ) + + input_data = {"user": {"name": "Alice", "age": 30}} + + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output["wrapped_input"] == input_data + assert result.output["user_name"] == "Alice" + assert len(result.steps_executed) == 1 + + def test_single_step_failure( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline with single failing step. + + Verifies that: + - Error is properly captured + - Error step is identified + - Steps executed contains the failed step + """ + contract = ModelComputeSubcontract( + operation_name="single_fail_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="failing_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ], + ) + + # Integer input will fail TRIM (requires string) + result = execute_compute_pipeline( + contract=contract, + input_data=12345, + context=compute_execution_context_factory(), + ) + + assert result.success is False + assert result.error_step == "failing_step" + assert "failing_step" in result.steps_executed + assert result.error_message is not None + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineStepResultAccumulation: + """Integration tests for step result accumulation and field mapping across steps. + + Note: 60-second timeout protects against pipeline execution hangs. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_step_results_accumulate_correctly( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that all step results are accumulated and accessible. + + Verifies that: + - Each step result is stored with correct step_name key + - Step outputs are preserved correctly + - Success flags are accurate per step + - Timing metadata is captured for each step + """ + contract = ModelComputeSubcontract( + operation_name="accumulation_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="step_trim", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ModelComputePipelineStep( + step_name="step_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="step_identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data=" hello world ", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert len(result.step_results) == 3 + + # Verify step_trim result + assert "step_trim" in result.step_results + trim_result = result.step_results["step_trim"] + assert trim_result.success is True + assert trim_result.output == "hello world" + assert trim_result.metadata.duration_ms >= 0 + + # Verify step_upper result + assert "step_upper" in result.step_results + upper_result = result.step_results["step_upper"] + assert upper_result.success is True + assert upper_result.output == "HELLO WORLD" + assert upper_result.metadata.duration_ms >= 0 + + # Verify step_identity result + assert "step_identity" in result.step_results + identity_result = result.step_results["step_identity"] + assert identity_result.success is True + assert identity_result.output == "HELLO WORLD" + assert identity_result.metadata.duration_ms >= 0 + + def test_mapping_references_all_prior_step_outputs( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test mapping step that references outputs from ALL prior steps. + + Verifies that: + - Mapping can access any prior step's output + - Original input remains accessible + - Complex aggregation of multiple step outputs works + """ + contract = ModelComputeSubcontract( + operation_name="multi_reference_mapping", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="upper_version", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="lower_version", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.LOWER + ), + # Note: This operates on upper_version output + ), + ModelComputePipelineStep( + step_name="aggregate_all", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "original": "$.input", + "uppercase": "$.steps.upper_version.output", + "lowercase": "$.steps.lower_version.output", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="Hello World", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert isinstance(result.output, dict) + assert result.output["original"] == "Hello World" + assert result.output["uppercase"] == "HELLO WORLD" + assert result.output["lowercase"] == "hello world" + + def test_sequential_mappings_build_complex_structures( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test multiple mapping steps building complex nested structures. + + Verifies that: + - Mapping output can be used by subsequent steps + - Nested structures are properly constructed + - Deep nesting paths work correctly + """ + contract = ModelComputeSubcontract( + operation_name="nested_mapping_build", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="first_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "level1_data": "$.input", + } + ), + ), + ModelComputePipelineStep( + step_name="second_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "level2_wrapper": "$.steps.first_mapping.output", + "original_preserved": "$.input", + } + ), + ), + ], + ) + + input_data = {"name": "test", "value": 42} + + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output["level2_wrapper"]["level1_data"] == input_data + assert result.output["original_preserved"] == input_data + + def test_transformation_metadata_tracking( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test that transformation type metadata is correctly tracked per step. + + Verifies that: + - transformation_type is recorded in step metadata + - Different transformation types are distinguished + """ + contract = ModelComputeSubcontract( + operation_name="metadata_tracking_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="trim_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ModelComputePipelineStep( + step_name="case_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="mapping_step", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={"result": "$.input"} + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data=" hello ", + context=compute_execution_context_factory(), + ) + + assert result.success is True + + # Verify transformation_type metadata (enum values are lowercase) + assert result.step_results["trim_step"].metadata.transformation_type == "trim" + assert ( + result.step_results["case_step"].metadata.transformation_type + == "case_conversion" + ) + # Mapping steps don't have transformation_type + assert result.step_results["mapping_step"].metadata.transformation_type is None + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineAllStepTypes: + """Integration tests covering all supported step types in various combinations. + + Note: 60-second timeout protects against pipeline execution hangs. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_all_transformation_types_in_pipeline( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline using all major transformation types. + + Verifies that: + - IDENTITY, TRIM, CASE_CONVERSION, REGEX, NORMALIZE_UNICODE all work together + - Transformations chain correctly + """ + contract = ModelComputeSubcontract( + operation_name="all_transforms_test", + operation_version="1.0.0", + pipeline=[ + # Step 1: Normalize unicode + ModelComputePipelineStep( + step_name="normalize", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.NORMALIZE_UNICODE, + transformation_config=ModelTransformUnicodeConfig( + form=EnumUnicodeForm.NFC + ), + ), + # Step 2: Regex - normalize whitespace + ModelComputePipelineStep( + step_name="regex_normalize", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.REGEX, + transformation_config=ModelTransformRegexConfig( + pattern=r"\s+", + replacement=" ", + ), + ), + # Step 3: Trim + ModelComputePipelineStep( + step_name="trim", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + # Step 4: Case conversion + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + # Step 5: Identity (pass-through) + ModelComputePipelineStep( + step_name="final_identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + # Input with multiple spaces and trailing whitespace + result = execute_compute_pipeline( + contract=contract, + input_data=" hello world ", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO WORLD" + assert len(result.steps_executed) == 5 + + def test_mixed_step_types_transform_and_mapping( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline mixing transformation and mapping steps. + + Verifies correct data flow between different step types. + """ + contract = ModelComputeSubcontract( + operation_name="mixed_types_test", + operation_version="1.0.0", + pipeline=[ + # Transform: uppercase + ModelComputePipelineStep( + step_name="transform_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + # Mapping: capture transformed and original + ModelComputePipelineStep( + step_name="capture_mapping", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "transformed": "$.steps.transform_upper.output", + "original": "$.input", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="hello", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output["transformed"] == "HELLO" + assert result.output["original"] == "hello" + + def test_validation_step_passthrough( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test validation step in v1.0 (pass-through mode). + + Verifies that: + - Validation step doesn't modify data in v1.0 + - Pipeline continues after validation + """ + contract = ModelComputeSubcontract( + operation_name="validation_passthrough_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="validate_input", + step_type=EnumComputeStepType.VALIDATION, + validation_config=ModelValidationStepConfig( + schema_ref="schemas/test_schema.json", + ), + ), + ModelComputePipelineStep( + step_name="final_transform", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data="hello", + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == "HELLO" + assert "validate_input" in result.steps_executed + + +@pytest.mark.integration +@pytest.mark.timeout(120) +class TestComputePipelineConcurrency: + """Integration tests for concurrent pipeline execution scenarios. + + Note: 120-second timeout allows for concurrent execution overhead. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_concurrent_pipeline_executions( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test multiple pipeline executions with different inputs concurrently. + + Verifies that: + - Multiple executions don't interfere with each other + - Each execution produces correct output for its input + - No shared state corruption occurs + """ + contract = ModelComputeSubcontract( + operation_name="concurrent_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + # Define inputs and expected outputs + test_cases = [ + ("hello", "HELLO"), + ("world", "WORLD"), + ("test", "TEST"), + ("data", "DATA"), + ("pipeline", "PIPELINE"), + ] + + def execute_pipeline( + input_output_pair: tuple[str, str], + ) -> tuple[bool, str, str]: + input_data, expected = input_output_pair + context = compute_execution_context_factory() + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=context, + ) + return (result.success, result.output, expected) + + # Execute concurrently + with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: + futures = [executor.submit(execute_pipeline, tc) for tc in test_cases] + results = [f.result() for f in concurrent.futures.as_completed(futures)] + + # Verify all executions succeeded with correct output + for success, actual, expected in results: + assert success is True + assert actual == expected + + def test_concurrent_different_pipelines( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test concurrent execution of different pipeline contracts. + + Verifies that: + - Different contracts can execute concurrently + - Each contract produces correct results + """ + contract_upper = ModelComputeSubcontract( + operation_name="upper_pipeline", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="uppercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + + contract_lower = ModelComputeSubcontract( + operation_name="lower_pipeline", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="lowercase", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.LOWER + ), + ), + ], + ) + + test_cases = [ + (contract_upper, "hello", "HELLO"), + (contract_lower, "HELLO", "hello"), + (contract_upper, "world", "WORLD"), + (contract_lower, "WORLD", "world"), + ] + + def execute_with_contract( + args: tuple[ModelComputeSubcontract, str, str], + ) -> tuple[bool, str, str]: + contract, input_data, expected = args + context = ModelComputeExecutionContext( + operation_id=uuid4(), + correlation_id=uuid4(), + ) + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=context, + ) + return (result.success, result.output, expected) + + with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor: + futures = [executor.submit(execute_with_contract, tc) for tc in test_cases] + results = [f.result() for f in concurrent.futures.as_completed(futures)] + + for success, actual, expected in results: + assert success is True + assert actual == expected + + +@pytest.mark.integration +@pytest.mark.timeout(60) +class TestComputePipelineEdgeCases: + """Integration tests for pipeline edge cases and boundary conditions. + + Note: 60-second timeout protects against pipeline execution hangs. + Uses compute_execution_context_factory fixture for context creation. + """ + + def test_all_disabled_steps_returns_input( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline where all steps are disabled. + + Verifies that: + - Output equals input when all steps disabled + - No steps appear in steps_executed + """ + contract = ModelComputeSubcontract( + operation_name="all_disabled_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="disabled1", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + enabled=False, + ), + ModelComputePipelineStep( + step_name="disabled2", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + enabled=False, + ), + ], + ) + + input_data = " hello " + + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == input_data # Unchanged + assert result.steps_executed == [] # No steps executed + + def test_special_characters_in_input( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline handles special characters correctly. + + Verifies that: + - Unicode characters pass through correctly + - Special regex characters don't cause issues + - Newlines and tabs are handled + """ + contract = ModelComputeSubcontract( + operation_name="special_chars_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + # Test various special characters + special_inputs = [ + "hello\nworld", # Newline + "hello\tworld", # Tab + "hello 世界", # Unicode + "test [regex] (chars) {special}", # Regex chars + "line1\r\nline2", # Windows newline + "", # Empty string + ] + + for input_data in special_inputs: + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + assert result.success is True + assert result.output == input_data + + def test_deeply_nested_input_data( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline handles deeply nested input structures. + + Verifies that: + - Deep nesting doesn't cause stack overflow + - Mapping can access deeply nested fields + """ + # Create deeply nested structure + deep_input: dict = { + "level0": {"level1": {"level2": {"level3": {"value": "found"}}}} + } + + contract = ModelComputeSubcontract( + operation_name="deep_nesting_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="extract_deep", + step_type=EnumComputeStepType.MAPPING, + mapping_config=ModelMappingConfig( + field_mappings={ + "deep_value": "$.input.level0.level1.level2.level3.value", + "full_input": "$.input", + } + ), + ), + ], + ) + + result = execute_compute_pipeline( + contract=contract, + input_data=deep_input, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output["deep_value"] == "found" + assert result.output["full_input"] == deep_input + + def test_null_and_empty_values_in_input( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline handles None and empty values. + + Verifies that: + - None values pass through correctly + - Empty strings and lists are handled + - Empty dicts are handled + """ + contract = ModelComputeSubcontract( + operation_name="null_empty_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + # Test various null/empty inputs + test_inputs: list[None | str | list | dict] = [ + None, + "", + [], + {}, + {"key": None}, + {"key": ""}, + {"key": []}, + ] + + for input_data in test_inputs: + result = execute_compute_pipeline( + contract=contract, + input_data=input_data, + context=compute_execution_context_factory(), + ) + assert result.success is True + assert result.output == input_data + + def test_large_list_input( + self, compute_execution_context_factory: ComputeContextFactory + ) -> None: + """Test pipeline handles large list inputs. + + Verifies that: + - Large lists pass through without performance degradation + - Identity transformation handles lists correctly + """ + contract = ModelComputeSubcontract( + operation_name="large_list_test", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + + # Create large list + large_list = list(range(10000)) + + result = execute_compute_pipeline( + contract=contract, + input_data=large_list, + context=compute_execution_context_factory(), + ) + + assert result.success is True + assert result.output == large_list + assert len(result.output) == 10000 + # Should complete reasonably fast + assert result.processing_time_ms < 1000 + + +if __name__ == "__main__": + pytest.main([__file__, "-v"]) diff --git a/tests/unit/container/test_container_service_resolver.py b/tests/unit/container/test_container_service_resolver.py index a8b6cf6478..48b1323e86 100644 --- a/tests/unit/container/test_container_service_resolver.py +++ b/tests/unit/container/test_container_service_resolver.py @@ -23,6 +23,7 @@ from omnibase_core.models.container.model_service import ModelService +@pytest.mark.unit class TestRegistryMapBuilding: """Test registry map construction from container.""" @@ -119,6 +120,7 @@ def test_build_registry_map_returns_callables(self): assert callable(service_method), f"Service {service_name} not callable" +@pytest.mark.unit class TestServiceResolution: """Test service resolution logic.""" @@ -199,6 +201,7 @@ def test_resolve_event_bus_by_protocol(self): assert service.protocol_name == "ProtocolEventBus" +@pytest.mark.unit class TestErrorHandling: """Test error handling in service resolution.""" @@ -245,6 +248,7 @@ def test_raises_error_for_invalid_protocol(self): assert exc_info.value.error_code == EnumCoreErrorCode.REGISTRY_RESOLUTION_FAILED +@pytest.mark.unit class TestMethodBinding: """Test bind_get_service_method functionality.""" @@ -281,6 +285,7 @@ def test_bound_method_preserves_container_context(self): assert container.get_service.__self__ is container +@pytest.mark.unit class TestServiceResolutionEdgeCases: """Test edge cases in service resolution.""" @@ -327,6 +332,7 @@ def test_multiple_service_resolutions(self): assert len({svc.service_name for svc in services}) == 3 +@pytest.mark.unit class TestProtocolTypeHandling: """Test protocol type handling variations.""" diff --git a/tests/unit/container/test_service_registry.py b/tests/unit/container/test_service_registry.py index fda3d03d97..46d50a02ee 100644 --- a/tests/unit/container/test_service_registry.py +++ b/tests/unit/container/test_service_registry.py @@ -34,6 +34,7 @@ def execute(self) -> str: return f"Executed {self.name} (count: {self.execution_count})" +@pytest.mark.unit class TestServiceRegistry: """Test suite for ServiceRegistry.""" @@ -393,6 +394,7 @@ async def test_registration_access_tracking( assert registration_after.access_count > initial_access_count +@pytest.mark.unit class TestContainerIntegration: """Test container integration with ServiceRegistry.""" diff --git a/tests/unit/contracts/test_model_effect_retry_config.py b/tests/unit/contracts/test_model_effect_retry_config.py index b8cbecc15f..b1578e62dc 100644 --- a/tests/unit/contracts/test_model_effect_retry_config.py +++ b/tests/unit/contracts/test_model_effect_retry_config.py @@ -214,10 +214,10 @@ def test_circuit_breaker_threshold_minimum_value(self): config = ModelEffectRetryConfig(circuit_breaker_threshold=1) assert config.circuit_breaker_threshold == 1 - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelEffectRetryConfig(circuit_breaker_threshold=0) - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelEffectRetryConfig(circuit_breaker_threshold=-1) def test_circuit_breaker_timeout_s_minimum_value(self): @@ -225,10 +225,10 @@ def test_circuit_breaker_timeout_s_minimum_value(self): config = ModelEffectRetryConfig(circuit_breaker_timeout_s=1) assert config.circuit_breaker_timeout_s == 1 - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelEffectRetryConfig(circuit_breaker_timeout_s=0) - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelEffectRetryConfig(circuit_breaker_timeout_s=-1) @@ -404,7 +404,7 @@ def test_validate_assignment(self): assert config.max_attempts == 5 # Invalid assignment should fail - with pytest.raises(Exception): + with pytest.raises(ValidationError): config.max_attempts = 0 diff --git a/tests/unit/core/test_model_base_factory.py b/tests/unit/core/test_model_base_factory.py index ab7ffbcdfe..068c35906c 100644 --- a/tests/unit/core/test_model_base_factory.py +++ b/tests/unit/core/test_model_base_factory.py @@ -6,7 +6,7 @@ """ import pytest -from pydantic import BaseModel, Field +from pydantic import BaseModel, Field, ValidationError from omnibase_core.models.base.model_factory import ModelBaseFactory @@ -229,7 +229,7 @@ def test_create_multiple_instances(self): def test_create_with_validation_error(self): """Test that create raises validation error for invalid data.""" factory = ConcreteProductFactory() - with pytest.raises(Exception): # Pydantic ValidationError + with pytest.raises(ValidationError): factory.create(name="Test", price="invalid_price") # type: ignore[arg-type] def test_factory_state_independence(self): diff --git a/tests/unit/enums/test_compute_enums.py b/tests/unit/enums/test_compute_enums.py new file mode 100644 index 0000000000..1fae6aff27 --- /dev/null +++ b/tests/unit/enums/test_compute_enums.py @@ -0,0 +1,140 @@ +"""Unit tests for Contract-Driven NodeCompute v1.0 enums.""" + +import pytest + +from omnibase_core.enums.enum_case_mode import EnumCaseMode +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_regex_flag import EnumRegexFlag +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.enums.enum_trim_mode import EnumTrimMode +from omnibase_core.enums.enum_unicode_form import EnumUnicodeForm + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestEnumComputeStepType: + """Tests for EnumComputeStepType.""" + + def test_values_exist(self) -> None: + """Test all expected values exist.""" + assert EnumComputeStepType.VALIDATION == "validation" + assert EnumComputeStepType.TRANSFORMATION == "transformation" + assert EnumComputeStepType.MAPPING == "mapping" + + def test_is_str_enum(self) -> None: + """Test enum values are strings for JSON serialization.""" + for member in EnumComputeStepType: + assert isinstance(member.value, str) + + def test_member_count(self) -> None: + """Test only v1.0 members exist (no CONDITIONAL/PARALLEL).""" + assert len(EnumComputeStepType) == 3 + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestEnumTransformationType: + """Tests for EnumTransformationType.""" + + def test_values_exist(self) -> None: + """Test all v1.0 values exist.""" + assert EnumTransformationType.IDENTITY == "identity" + assert EnumTransformationType.REGEX == "regex" + assert EnumTransformationType.CASE_CONVERSION == "case_conversion" + assert EnumTransformationType.TRIM == "trim" + assert EnumTransformationType.NORMALIZE_UNICODE == "normalize_unicode" + assert EnumTransformationType.JSON_PATH == "json_path" + + def test_is_str_enum(self) -> None: + """Test enum values are strings for JSON serialization.""" + for member in EnumTransformationType: + assert isinstance(member.value, str) + + def test_member_count(self) -> None: + """Test only v1.0 members exist (6 transformations).""" + assert len(EnumTransformationType) == 6 + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestEnumCaseMode: + """Tests for EnumCaseMode.""" + + def test_values_exist(self) -> None: + """Test all values exist.""" + assert EnumCaseMode.UPPER == "uppercase" + assert EnumCaseMode.LOWER == "lowercase" + assert EnumCaseMode.TITLE == "titlecase" + + def test_is_str_enum(self) -> None: + """Test enum values are strings.""" + for member in EnumCaseMode: + assert isinstance(member.value, str) + + def test_member_count(self) -> None: + """Test expected number of members (3 case modes).""" + assert len(EnumCaseMode) == 3 + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestEnumRegexFlag: + """Tests for EnumRegexFlag.""" + + def test_values_exist(self) -> None: + """Test all values exist.""" + assert EnumRegexFlag.IGNORECASE == "IGNORECASE" + assert EnumRegexFlag.MULTILINE == "MULTILINE" + assert EnumRegexFlag.DOTALL == "DOTALL" + + def test_is_str_enum(self) -> None: + """Test enum values are strings.""" + for member in EnumRegexFlag: + assert isinstance(member.value, str) + + def test_member_count(self) -> None: + """Test expected number of members (3 regex flags).""" + assert len(EnumRegexFlag) == 3 + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestEnumUnicodeForm: + """Tests for EnumUnicodeForm.""" + + def test_values_exist(self) -> None: + """Test all Unicode normalization forms exist.""" + assert EnumUnicodeForm.NFC == "NFC" + assert EnumUnicodeForm.NFD == "NFD" + assert EnumUnicodeForm.NFKC == "NFKC" + assert EnumUnicodeForm.NFKD == "NFKD" + + def test_is_str_enum(self) -> None: + """Test enum values are strings.""" + for member in EnumUnicodeForm: + assert isinstance(member.value, str) + + def test_member_count(self) -> None: + """Test expected number of members (4 Unicode forms).""" + assert len(EnumUnicodeForm) == 4 + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestEnumTrimMode: + """Tests for EnumTrimMode.""" + + def test_values_exist(self) -> None: + """Test all values exist.""" + assert EnumTrimMode.BOTH == "both" + assert EnumTrimMode.LEFT == "left" + assert EnumTrimMode.RIGHT == "right" + + def test_is_str_enum(self) -> None: + """Test enum values are strings.""" + for member in EnumTrimMode: + assert isinstance(member.value, str) + + def test_member_count(self) -> None: + """Test expected number of members (3 trim modes).""" + assert len(EnumTrimMode) == 3 diff --git a/tests/unit/enums/test_enum_node_kind.py b/tests/unit/enums/test_enum_node_kind.py index 87be0362c7..838a8d145b 100644 --- a/tests/unit/enums/test_enum_node_kind.py +++ b/tests/unit/enums/test_enum_node_kind.py @@ -473,26 +473,26 @@ def check_type_to_kind_mapping(node_type: EnumNodeType) -> None: check_type_to_kind_mapping() def test_unknown_node_type_raises_error(self) -> None: - """Verify that UNKNOWN node type raises ModelOnexError when get_node_kind is called. + """Verify that UNKNOWN node type raises ValueError when get_node_kind is called. DESIGN DECISION: UNKNOWN intentionally has NO kind mapping because: - UNKNOWN semantically means "we don't know what this is" - Silently defaulting to COMPUTE would hide bugs in node classification - Callers must explicitly handle the UNKNOWN case with proper error handling - This forces explicit error handling rather than silent failures + + NOTE: Uses ValueError instead of ModelOnexError to avoid enum->model import + (architectural violation: enums must not import models). """ import pytest from omnibase_core.enums.enum_node_type import EnumNodeType - from omnibase_core.models.errors.model_onex_error import ModelOnexError - with pytest.raises(ModelOnexError) as exc_info: + with pytest.raises(ValueError) as exc_info: EnumNodeType.get_node_kind(EnumNodeType.UNKNOWN) # Verify the error contains useful context - assert "UNKNOWN" in str(exc_info.value) or "UNKNOWN" in str( - exc_info.value.context - ) + assert "UNKNOWN" in str(exc_info.value) if __name__ == "__main__": diff --git a/tests/unit/enums/test_enum_node_type_kind_mapping.py b/tests/unit/enums/test_enum_node_type_kind_mapping.py index 894e91e1e0..d94ede6536 100644 --- a/tests/unit/enums/test_enum_node_type_kind_mapping.py +++ b/tests/unit/enums/test_enum_node_type_kind_mapping.py @@ -483,7 +483,7 @@ class TestEnumNodeTypeKindMappingEdgeCases: def test_unknown_type_intentionally_unmapped(self) -> None: """ - Verify that UNKNOWN node type raises ModelOnexError when get_node_kind is called. + Verify that UNKNOWN node type raises ValueError when get_node_kind is called. DESIGN DECISION - UNKNOWN intentionally has NO kind mapping because: - UNKNOWN semantically means "we don't know what this is" @@ -491,24 +491,22 @@ def test_unknown_type_intentionally_unmapped(self) -> None: - Callers must explicitly handle the UNKNOWN case with proper error handling - This forces explicit error handling rather than silent failures + NOTE: Uses ValueError instead of ModelOnexError to avoid enum->model import + (architectural violation: enums must not import models). + This test ensures the design decision is preserved across refactoring. """ # Skip this test if get_node_kind doesn't exist yet (pre-refactor state) if not hasattr(EnumNodeType, "get_node_kind"): pytest.skip("EnumNodeType.get_node_kind() not yet implemented") - from omnibase_core.models.errors.model_onex_error import ModelOnexError - - # UNKNOWN must raise ModelOnexError, NOT return a default kind - with pytest.raises(ModelOnexError) as exc_info: + # UNKNOWN must raise ValueError, NOT return a default kind + with pytest.raises(ValueError) as exc_info: EnumNodeType.get_node_kind(EnumNodeType.UNKNOWN) # Verify the error message contains useful context - error = exc_info.value - assert "UNKNOWN" in str(error) or ( - error.context and "UNKNOWN" in str(error.context.get("node_type", "")) - ), ( - f"Error message should mention UNKNOWN. Got: {error}, context: {error.context}" + assert "UNKNOWN" in str(exc_info.value), ( + f"Error message should mention UNKNOWN. Got: {exc_info.value}" ) def test_unknown_is_not_in_kind_map(self) -> None: diff --git a/tests/unit/errors/test_compute_pipeline_error.py b/tests/unit/errors/test_compute_pipeline_error.py new file mode 100644 index 0000000000..5e519ce2b8 --- /dev/null +++ b/tests/unit/errors/test_compute_pipeline_error.py @@ -0,0 +1,209 @@ +""" +Tests for ComputePipelineError (OMN-465). + +TDD tests for compute pipeline error class used in contract-driven NodeCompute v1.0. +""" + +from uuid import UUID, uuid4 + +import pytest + + +@pytest.mark.unit +@pytest.mark.timeout(10) +class TestComputePipelineError: + """Tests for ComputePipelineError.""" + + def test_compute_pipeline_error_creation(self) -> None: + """Test creating ComputePipelineError with minimal args.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError("Pipeline step failed") + + # ModelOnexError includes error code in string representation + assert "[ONEX_CORE_165_PROCESSING_ERROR]" in str(error) + assert "Pipeline step failed" in str(error) + # Check error-specific attributes before isinstance check + assert error.correlation_id is not None + assert isinstance(error.correlation_id, UUID) + assert isinstance(error, Exception) + + def test_compute_pipeline_error_with_step_name(self) -> None: + """Test ComputePipelineError with step_name.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError( + "Transformation failed", + step_name="normalize_case", + ) + + assert error.step_name == "normalize_case" + + def test_compute_pipeline_error_with_operation(self) -> None: + """Test ComputePipelineError with operation context.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError( + "Operation failed", + step_name="transform", + operation="transformation", + ) + + assert error.step_name == "transform" + assert error.operation == "transformation" + + def test_compute_pipeline_error_with_correlation_id(self) -> None: + """Test ComputePipelineError preserves correlation_id.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + corr_id = uuid4() + error = ComputePipelineError( + "Pipeline error", + step_name="validate", + correlation_id=corr_id, + ) + + assert error.correlation_id == corr_id + + def test_compute_pipeline_error_serialization(self) -> None: + """Test ComputePipelineError can be serialized.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError( + "Pipeline error", + step_name="transform_data", + operation="mapping", + ) + + # Should have model_dump method from ModelOnexError + error_dict = error.model_dump() + + assert error_dict["message"] == "Pipeline error" + assert "correlation_id" in error_dict + assert "context" in error_dict + assert error_dict["context"].get("step_name") == "transform_data" + assert error_dict["context"].get("operation") == "mapping" + + def test_compute_pipeline_error_with_custom_error_code(self) -> None: + """Test ComputePipelineError with custom error code.""" + from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError( + "Validation failed", + step_name="validate_input", + error_code=EnumCoreErrorCode.VALIDATION_ERROR, + ) + + assert error.error_code == EnumCoreErrorCode.VALIDATION_ERROR + assert "[ONEX_CORE_006_VALIDATION_ERROR]" in str(error) + + def test_compute_pipeline_error_with_additional_context(self) -> None: + """Test ComputePipelineError supports additional structured context.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError( + "Type mismatch in transformation", + step_name="case_conversion", + operation="transformation", + input_type="int", + expected_type="str", + ) + + error_dict = error.model_dump() + + # Additional context fields are nested in 'context' dict + assert "context" in error_dict + context = error_dict["context"] + assert context.get("input_type") == "int" + assert context.get("expected_type") == "str" + assert context.get("step_name") == "case_conversion" + assert context.get("operation") == "transformation" + + +@pytest.mark.unit +@pytest.mark.timeout(10) +class TestComputePipelineErrorImport: + """Tests for ComputePipelineError import from errors module.""" + + def test_import_from_errors_module(self) -> None: + """Test ComputePipelineError can be imported from errors module.""" + from omnibase_core.errors import ComputePipelineError + + error = ComputePipelineError( + "Test error", + step_name="test_step", + ) + + assert error.step_name == "test_step" + assert "[ONEX_CORE_165_PROCESSING_ERROR]" in str(error) + + def test_compute_pipeline_error_inherits_model_onex_error(self) -> None: + """Test ComputePipelineError inherits from ModelOnexError.""" + from omnibase_core.errors import ComputePipelineError + from omnibase_core.models.errors.model_onex_error import ModelOnexError + + error = ComputePipelineError("Test error") + + assert isinstance(error, ModelOnexError) + assert isinstance(error, Exception) + + +@pytest.mark.unit +@pytest.mark.timeout(10) +class TestComputePipelineErrorInvariants: + """Tests for ComputePipelineError invariants.""" + + def test_error_includes_correlation_id(self) -> None: + """Test ComputePipelineError always includes correlation_id.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError("test") + + assert hasattr(error, "correlation_id") + assert error.correlation_id is not None + assert isinstance(error.correlation_id, UUID) + + def test_no_raw_stack_traces_in_serialization(self) -> None: + """Test error serialization doesn't include raw stack traces.""" + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError("test error", step_name="test") + error_dict = error.model_dump() + + # Should not have __traceback__ or raw stack trace + assert "__traceback__" not in error_dict + assert "traceback" not in error_dict + + # Context fields should be structured, not raw exception data + for _, value in error_dict.items(): + assert not isinstance(value, type(error)) # No nested exceptions + + def test_default_error_code_is_processing_error(self) -> None: + """Test default error code is PROCESSING_ERROR.""" + from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode + from omnibase_core.errors.exception_compute_pipeline_error import ( + ComputePipelineError, + ) + + error = ComputePipelineError("test") + + assert error.error_code == EnumCoreErrorCode.PROCESSING_ERROR diff --git a/tests/unit/mixins/test_mixin_completion_data.py b/tests/unit/mixins/test_mixin_completion_data.py index 7cfe20a154..923c077d0f 100644 --- a/tests/unit/mixins/test_mixin_completion_data.py +++ b/tests/unit/mixins/test_mixin_completion_data.py @@ -3,6 +3,7 @@ """ import pytest +from pydantic import ValidationError from omnibase_core.mixins.mixin_completion_data import MixinCompletionData @@ -153,17 +154,17 @@ def test_mixin_completion_data_to_event_kwargs_excludes_none(self): assert "tags" not in kwargs def test_mixin_completion_data_immutable(self): - """Test that MixinCompletionData is immutable.""" + """Test that MixinCompletionData is immutable (frozen model).""" completion = MixinCompletionData( message="Test", success=True, code=200, tags=["test"] ) - # Should not be able to modify fields (Pydantic ValidationError for frozen models) - with pytest.raises(Exception): # ValidationError for frozen models - completion.message = "Modified" + # Should not be able to modify fields - frozen model raises ValidationError + with pytest.raises(ValidationError, match="frozen"): + completion.message = "Modified" # type: ignore[misc] - with pytest.raises(Exception): # ValidationError for frozen models - completion.success = False + with pytest.raises(ValidationError, match="frozen"): + completion.success = False # type: ignore[misc] def test_mixin_completion_data_strict_types(self): """Test that MixinCompletionData enforces strict types.""" diff --git a/tests/unit/models/compute/conftest.py b/tests/unit/models/compute/conftest.py new file mode 100644 index 0000000000..65e0a9e30b --- /dev/null +++ b/tests/unit/models/compute/conftest.py @@ -0,0 +1,189 @@ +"""Shared fixtures for compute-related tests. + +Provides reusable fixtures for: +- ModelComputeSubcontract creation +- ModelComputeExecutionContext creation +- ModelComputePipelineStep creation +- Common transformation configs + +These fixtures reduce test boilerplate and ensure consistent test data. +""" + +from uuid import uuid4 + +import pytest + +from omnibase_core.enums.enum_case_mode import EnumCaseMode +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.enums.enum_trim_mode import EnumTrimMode +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step import ( + ModelComputePipelineStep, +) +from omnibase_core.models.contracts.subcontracts.model_compute_subcontract import ( + ModelComputeSubcontract, +) +from omnibase_core.models.transformations.model_transform_case_config import ( + ModelTransformCaseConfig, +) +from omnibase_core.models.transformations.model_transform_trim_config import ( + ModelTransformTrimConfig, +) + + +@pytest.fixture +def sample_operation_id(): + """Generate a sample operation ID for tests.""" + return uuid4() + + +@pytest.fixture +def sample_correlation_id(): + """Generate a sample correlation ID for tests.""" + return uuid4() + + +@pytest.fixture +def sample_execution_context(sample_operation_id): + """Create a sample ModelComputeExecutionContext for tests. + + Returns: + ModelComputeExecutionContext with operation_id set + """ + return ModelComputeExecutionContext(operation_id=sample_operation_id) + + +@pytest.fixture +def sample_execution_context_full(sample_operation_id, sample_correlation_id): + """Create a ModelComputeExecutionContext with all fields populated. + + Returns: + ModelComputeExecutionContext with operation_id, correlation_id, and node_id + """ + return ModelComputeExecutionContext( + operation_id=sample_operation_id, + correlation_id=sample_correlation_id, + node_id="test-node-123", + ) + + +@pytest.fixture +def uppercase_transform_config(): + """Create uppercase transformation config.""" + return ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + + +@pytest.fixture +def lowercase_transform_config(): + """Create lowercase transformation config.""" + return ModelTransformCaseConfig(mode=EnumCaseMode.LOWER) + + +@pytest.fixture +def trim_both_config(): + """Create trim both sides config.""" + return ModelTransformTrimConfig(mode=EnumTrimMode.BOTH) + + +@pytest.fixture +def identity_pipeline_step(): + """Create an identity transformation pipeline step. + + Returns: + ModelComputePipelineStep configured for identity transformation + """ + return ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + + +@pytest.fixture +def uppercase_pipeline_step(uppercase_transform_config): + """Create an uppercase transformation pipeline step. + + Returns: + ModelComputePipelineStep configured for uppercase case conversion + """ + return ModelComputePipelineStep( + step_name="to_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=uppercase_transform_config, + ) + + +@pytest.fixture +def trim_pipeline_step(trim_both_config): + """Create a trim transformation pipeline step. + + Returns: + ModelComputePipelineStep configured for trimming whitespace + """ + return ModelComputePipelineStep( + step_name="trim", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=trim_both_config, + ) + + +@pytest.fixture +def empty_compute_subcontract(): + """Create a ModelComputeSubcontract with an empty pipeline. + + Returns: + ModelComputeSubcontract with no pipeline steps + """ + return ModelComputeSubcontract( + operation_name="empty_op", + operation_version="1.0.0", + pipeline=[], + ) + + +@pytest.fixture +def simple_compute_subcontract(uppercase_pipeline_step): + """Create a simple ModelComputeSubcontract with one pipeline step. + + Returns: + ModelComputeSubcontract with single uppercase transformation + """ + return ModelComputeSubcontract( + operation_name="text_upper", + operation_version="1.0.0", + pipeline=[uppercase_pipeline_step], + ) + + +@pytest.fixture +def multi_step_compute_subcontract(trim_pipeline_step, uppercase_pipeline_step): + """Create a ModelComputeSubcontract with multiple pipeline steps. + + Returns: + ModelComputeSubcontract with trim then uppercase transformations + """ + return ModelComputeSubcontract( + operation_name="text_normalize", + operation_version="1.0.0", + pipeline=[trim_pipeline_step, uppercase_pipeline_step], + ) + + +@pytest.fixture +def compute_subcontract_with_timeout(uppercase_pipeline_step): + """Create a ModelComputeSubcontract with pipeline timeout configured. + + Returns: + ModelComputeSubcontract with 5 second timeout + """ + return ModelComputeSubcontract( + operation_name="timed_op", + operation_version="1.0.0", + pipeline=[uppercase_pipeline_step], + pipeline_timeout_ms=5000, + ) diff --git a/tests/unit/models/compute/test_compute_models.py b/tests/unit/models/compute/test_compute_models.py new file mode 100644 index 0000000000..aebb6aba5e --- /dev/null +++ b/tests/unit/models/compute/test_compute_models.py @@ -0,0 +1,363 @@ +"""Unit tests for Contract-Driven NodeCompute v1.0 compute models.""" + +from uuid import uuid4 + +import pytest +from pydantic import ValidationError + +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.compute.model_compute_pipeline_result import ( + ModelComputePipelineResult, +) +from omnibase_core.models.compute.model_compute_step_metadata import ( + ModelComputeStepMetadata, +) +from omnibase_core.models.compute.model_compute_step_result import ( + ModelComputeStepResult, +) +from omnibase_core.models.model_compute_output import ModelComputeOutput + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelComputeExecutionContext: + """Tests for ModelComputeExecutionContext.""" + + def test_create_minimal(self) -> None: + """Test creating context with minimal fields.""" + op_id = uuid4() + context = ModelComputeExecutionContext(operation_id=op_id) + assert context.operation_id == op_id + assert context.correlation_id is None + assert context.node_id is None + + def test_create_full(self) -> None: + """Test creating context with all fields.""" + op_id = uuid4() + corr_id = uuid4() + context = ModelComputeExecutionContext( + operation_id=op_id, + correlation_id=corr_id, + node_id="test-node-123", + ) + assert context.operation_id == op_id + assert context.correlation_id == corr_id + assert context.node_id == "test-node-123" + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + context = ModelComputeExecutionContext(operation_id=uuid4()) + with pytest.raises(ValidationError, match="frozen"): + context.node_id = "modified" # type: ignore[misc] + + def test_extra_fields_rejected(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelComputeExecutionContext( + operation_id=uuid4(), + extra_field="invalid", # type: ignore[call-arg] + ) + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelComputeStepMetadata: + """Tests for ModelComputeStepMetadata.""" + + def test_create_minimal(self) -> None: + """Test creating metadata with minimal fields.""" + metadata = ModelComputeStepMetadata(duration_ms=10.5) + assert metadata.duration_ms == 10.5 + assert metadata.transformation_type is None + + def test_create_full(self) -> None: + """Test creating metadata with all fields.""" + metadata = ModelComputeStepMetadata( + duration_ms=25.3, + transformation_type="case_conversion", + ) + assert metadata.duration_ms == 25.3 + assert metadata.transformation_type == "case_conversion" + + def test_duration_ms_zero_allowed(self) -> None: + """Test that zero duration is allowed.""" + metadata = ModelComputeStepMetadata(duration_ms=0.0) + assert metadata.duration_ms == 0.0 + + def test_duration_ms_negative_raises_error(self) -> None: + """Test that negative duration raises validation error.""" + with pytest.raises(ValidationError, match="greater than or equal to 0"): + ModelComputeStepMetadata(duration_ms=-1.0) + + def test_duration_ms_large_negative_raises_error(self) -> None: + """Test that large negative duration raises validation error.""" + with pytest.raises(ValidationError, match="greater than or equal to 0"): + ModelComputeStepMetadata(duration_ms=-999.99) + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + metadata = ModelComputeStepMetadata(duration_ms=10.0) + with pytest.raises(ValidationError, match="frozen"): + metadata.duration_ms = 20.0 # type: ignore[misc] + + def test_extra_fields_rejected(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelComputeStepMetadata( + duration_ms=10.0, + extra_field="invalid", # type: ignore[call-arg] + ) + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelComputeStepResult: + """Tests for ModelComputeStepResult.""" + + def test_create_success(self) -> None: + """Test creating successful step result.""" + metadata = ModelComputeStepMetadata(duration_ms=5.0) + result = ModelComputeStepResult( + step_name="transform_case", + output="HELLO WORLD", + success=True, + metadata=metadata, + ) + assert result.step_name == "transform_case" + assert result.output == "HELLO WORLD" + assert result.success is True + assert result.error_type is None + assert result.error_message is None + + def test_create_failure(self) -> None: + """Test creating failed step result.""" + metadata = ModelComputeStepMetadata(duration_ms=2.0) + result = ModelComputeStepResult( + step_name="validate_input", + output=None, + success=False, + metadata=metadata, + error_type="validation_error", + error_message="Input schema validation failed", + ) + assert result.success is False + assert result.error_type == "validation_error" + assert result.error_message == "Input schema validation failed" + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + metadata = ModelComputeStepMetadata(duration_ms=5.0) + result = ModelComputeStepResult( + step_name="test", + output="data", + metadata=metadata, + ) + with pytest.raises(ValidationError, match="frozen"): + result.success = False # type: ignore[misc] + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelComputePipelineResult: + """Tests for ModelComputePipelineResult.""" + + def test_create_success(self) -> None: + """Test creating successful pipeline result.""" + metadata = ModelComputeStepMetadata(duration_ms=5.0) + step_result = ModelComputeStepResult( + step_name="step1", + output="result", + metadata=metadata, + ) + + result = ModelComputePipelineResult( + success=True, + output="final result", + processing_time_ms=15.5, + steps_executed=["step1"], + step_results={"step1": step_result}, + ) + assert result.success is True + assert result.output == "final result" + assert result.processing_time_ms == 15.5 + assert result.steps_executed == ["step1"] + assert "step1" in result.step_results + assert result.error_type is None + assert result.error_step is None + + def test_create_failure(self) -> None: + """Test creating failed pipeline result.""" + metadata = ModelComputeStepMetadata(duration_ms=3.0) + step_result = ModelComputeStepResult( + step_name="failing_step", + output=None, + success=False, + metadata=metadata, + error_type="compute_error", + error_message="Invalid input type", + ) + + result = ModelComputePipelineResult( + success=False, + output=None, + processing_time_ms=3.5, + steps_executed=["failing_step"], + step_results={"failing_step": step_result}, + error_type="compute_error", + error_message="Invalid input type", + error_step="failing_step", + ) + assert result.success is False + assert result.error_step == "failing_step" + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + result = ModelComputePipelineResult( + success=True, + output="data", + processing_time_ms=10.0, + steps_executed=[], + step_results={}, + ) + with pytest.raises(ValidationError, match="frozen"): + result.success = False # type: ignore[misc] + + def test_processing_time_ms_zero_allowed(self) -> None: + """Test that zero processing time is allowed.""" + result = ModelComputePipelineResult( + success=True, + output="data", + processing_time_ms=0.0, + steps_executed=[], + step_results={}, + ) + assert result.processing_time_ms == 0.0 + + def test_processing_time_ms_negative_raises_error(self) -> None: + """Test that negative processing time raises validation error.""" + with pytest.raises(ValidationError, match="greater than or equal to 0"): + ModelComputePipelineResult( + success=True, + output="data", + processing_time_ms=-1.0, + steps_executed=[], + step_results={}, + ) + + def test_processing_time_ms_large_negative_raises_error(self) -> None: + """Test that large negative processing time raises validation error.""" + with pytest.raises(ValidationError, match="greater than or equal to 0"): + ModelComputePipelineResult( + success=True, + output="data", + processing_time_ms=-999.99, + steps_executed=[], + step_results={}, + ) + + def test_extra_fields_rejected(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelComputePipelineResult( + success=True, + output="data", + processing_time_ms=10.0, + steps_executed=[], + step_results={}, + extra_field="invalid", # type: ignore[call-arg] + ) + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelComputeOutput: + """Tests for ModelComputeOutput.""" + + def test_create_minimal(self) -> None: + """Test creating output with minimal fields.""" + op_id = uuid4() + output = ModelComputeOutput( + result="test_result", + operation_id=op_id, + computation_type="test_computation", + processing_time_ms=10.5, + ) + assert output.result == "test_result" + assert output.operation_id == op_id + assert output.computation_type == "test_computation" + assert output.processing_time_ms == 10.5 + assert output.cache_hit is False + assert output.parallel_execution_used is False + assert output.metadata == {} + + def test_create_full(self) -> None: + """Test creating output with all fields.""" + op_id = uuid4() + output = ModelComputeOutput( + result={"transformed": "data"}, + operation_id=op_id, + computation_type="complex_transform", + processing_time_ms=25.3, + cache_hit=True, + parallel_execution_used=True, + metadata={"source": "test"}, + ) + assert output.result == {"transformed": "data"} + assert output.cache_hit is True + assert output.parallel_execution_used is True + assert output.metadata == {"source": "test"} + + def test_processing_time_ms_zero_allowed(self) -> None: + """Test that zero processing time is allowed (e.g., cache hit).""" + output = ModelComputeOutput( + result="cached", + operation_id=uuid4(), + computation_type="test", + processing_time_ms=0.0, + cache_hit=True, + ) + assert output.processing_time_ms == 0.0 + + def test_processing_time_ms_negative_raises_error(self) -> None: + """Test that negative processing time raises validation error.""" + with pytest.raises(ValidationError, match="greater than or equal to 0"): + ModelComputeOutput( + result="test", + operation_id=uuid4(), + computation_type="test", + processing_time_ms=-1.0, + ) + + def test_processing_time_ms_large_negative_raises_error(self) -> None: + """Test that large negative processing time raises validation error.""" + with pytest.raises(ValidationError, match="greater than or equal to 0"): + ModelComputeOutput( + result="test", + operation_id=uuid4(), + computation_type="test", + processing_time_ms=-999.99, + ) + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + output = ModelComputeOutput( + result="test", + operation_id=uuid4(), + computation_type="test", + processing_time_ms=10.0, + ) + with pytest.raises(ValidationError, match="frozen"): + output.result = "modified" # type: ignore[misc] + + def test_extra_fields_rejected(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelComputeOutput( + result="test", + operation_id=uuid4(), + computation_type="test", + processing_time_ms=10.0, + extra_field="invalid", # type: ignore[call-arg] + ) diff --git a/tests/unit/models/configuration/test_model_compute_cache_config.py b/tests/unit/models/configuration/test_model_compute_cache_config.py index 835c798e2d..e9fd2d7f4f 100644 --- a/tests/unit/models/configuration/test_model_compute_cache_config.py +++ b/tests/unit/models/configuration/test_model_compute_cache_config.py @@ -13,6 +13,7 @@ ) +@pytest.mark.unit class TestModelComputeCacheConfigInitialization: """Test ModelComputeCacheConfig initialization.""" @@ -47,6 +48,7 @@ def test_config_inheritance(self): assert isinstance(config, BaseModel) +@pytest.mark.unit class TestModelComputeCacheConfigValidation: """Test ModelComputeCacheConfig field validation.""" @@ -127,6 +129,7 @@ def test_enable_stats_validation(self): assert config.enable_stats is False +@pytest.mark.unit class TestModelComputeCacheConfigSerialization: """Test ModelComputeCacheConfig serialization.""" @@ -184,6 +187,7 @@ def test_config_roundtrip(self): assert restored.enable_stats == original.enable_stats +@pytest.mark.unit class TestModelComputeCacheConfigMethods: """Test ModelComputeCacheConfig methods.""" @@ -250,6 +254,7 @@ def test_validate_memory_requirements_large_cache(self): assert memory["max_memory_mb"] == 11.72 # 20% overhead +@pytest.mark.unit class TestModelComputeCacheConfigEdgeCases: """Test cache config edge cases.""" @@ -286,6 +291,7 @@ def test_ttl_none_for_no_expiration(self): assert config.get_effective_ttl_seconds() == 0 +@pytest.mark.unit class TestModelComputeCacheConfigAttributes: """Test cache config attributes and metadata.""" @@ -338,6 +344,7 @@ def test_config_str_repr(self): assert "ModelComputeCacheConfig" in repr_str +@pytest.mark.unit class TestModelComputeCacheConfigProductionScenarios: """Test production deployment scenarios.""" diff --git a/tests/unit/models/container/test_model_service.py b/tests/unit/models/container/test_model_service.py index 9dc25caa21..ce6ef78f05 100644 --- a/tests/unit/models/container/test_model_service.py +++ b/tests/unit/models/container/test_model_service.py @@ -3,6 +3,7 @@ from uuid import uuid4 import pytest +from pydantic import ValidationError from omnibase_core.models.container.model_service import ModelService @@ -55,7 +56,7 @@ def test_frozen_model(self): service_type="compute", ) - with pytest.raises(Exception): # ValidationError for frozen model + with pytest.raises(ValidationError, match="frozen"): service.service_name = "modified" # type: ignore[misc] def test_service_types(self): diff --git a/tests/unit/models/contracts/subcontracts/test_model_compute_subcontract.py b/tests/unit/models/contracts/subcontracts/test_model_compute_subcontract.py new file mode 100644 index 0000000000..3ca3863388 --- /dev/null +++ b/tests/unit/models/contracts/subcontracts/test_model_compute_subcontract.py @@ -0,0 +1,294 @@ +"""Unit tests for ModelComputeSubcontract validation.""" + +import pytest +from pydantic import ValidationError + +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step import ( + ModelComputePipelineStep, +) +from omnibase_core.models.contracts.subcontracts.model_compute_subcontract import ( + ModelComputeSubcontract, +) + + +@pytest.mark.unit +class TestModelComputeSubcontractCreation: + """Tests for ModelComputeSubcontract creation.""" + + def test_create_minimal(self) -> None: + """Test creating subcontract with minimal fields.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + ) + assert subcontract.operation_name == "test_op" + assert subcontract.operation_version == "1.0.0" + assert subcontract.pipeline == [] + assert subcontract.pipeline_timeout_ms is None + assert subcontract.version == "1.0.0" + + def test_create_with_pipeline(self) -> None: + """Test creating subcontract with pipeline steps.""" + step = ModelComputePipelineStep( + step_name="transform", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[step], + ) + assert len(subcontract.pipeline) == 1 + assert subcontract.pipeline[0].step_name == "transform" + + def test_create_with_timeout(self) -> None: + """Test creating subcontract with pipeline timeout.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=5000, + ) + assert subcontract.pipeline_timeout_ms == 5000 + + +@pytest.mark.unit +class TestModelComputeSubcontractTimeoutValidation: + """Tests for pipeline_timeout_ms validation.""" + + def test_timeout_none_allowed(self) -> None: + """Test that None timeout is allowed (no timeout).""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=None, + ) + assert subcontract.pipeline_timeout_ms is None + + def test_timeout_positive_allowed(self) -> None: + """Test that positive timeout values are allowed.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=1000, + ) + assert subcontract.pipeline_timeout_ms == 1000 + + def test_timeout_large_value_allowed(self) -> None: + """Test that large timeout values are allowed (up to 1 hour).""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=600000, # 10 minutes + ) + assert subcontract.pipeline_timeout_ms == 600000 + + def test_timeout_max_allowed(self) -> None: + """Test that maximum timeout (1 hour) is allowed.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=3600000, # 1 hour (max allowed) + ) + assert subcontract.pipeline_timeout_ms == 3600000 + + def test_timeout_exceeds_max_raises_error(self) -> None: + """Test that timeout exceeding 1 hour raises validation error.""" + with pytest.raises(ValidationError, match="less than or equal to 3600000"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=3600001, # Just over 1 hour + ) + + def test_timeout_one_allowed(self) -> None: + """Test that timeout of 1ms is allowed.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=1, + ) + assert subcontract.pipeline_timeout_ms == 1 + + def test_timeout_zero_raises_error(self) -> None: + """Test that zero timeout raises validation error.""" + with pytest.raises(ValidationError, match="greater than 0"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=0, + ) + + def test_timeout_negative_raises_error(self) -> None: + """Test that negative timeout raises validation error.""" + with pytest.raises(ValidationError, match="greater than 0"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=-1000, + ) + + def test_timeout_large_negative_raises_error(self) -> None: + """Test that large negative timeout raises validation error.""" + with pytest.raises(ValidationError, match="greater than 0"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + pipeline_timeout_ms=-99999, + ) + + +@pytest.mark.unit +class TestModelComputeSubcontractFrozen: + """Tests for model immutability.""" + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + ) + with pytest.raises(ValidationError): + subcontract.operation_name = "modified" # type: ignore[misc] + + def test_extra_fields_rejected(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + extra_field="invalid", # type: ignore[call-arg] + ) + + +@pytest.mark.unit +class TestModelComputeSubcontractUniqueStepNames: + """Tests for unique step name validation.""" + + def test_unique_step_names_allowed(self) -> None: + """Test that unique step names are accepted.""" + step1 = ModelComputePipelineStep( + step_name="step_one", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + step2 = ModelComputePipelineStep( + step_name="step_two", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[step1, step2], + ) + assert len(subcontract.pipeline) == 2 + assert subcontract.pipeline[0].step_name == "step_one" + assert subcontract.pipeline[1].step_name == "step_two" + + def test_duplicate_step_names_raises_error(self) -> None: + """Test that duplicate step names raise validation error.""" + step1 = ModelComputePipelineStep( + step_name="duplicate_name", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + step2 = ModelComputePipelineStep( + step_name="duplicate_name", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + with pytest.raises(ValidationError, match="step names must be unique"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[step1, step2], + ) + + def test_multiple_duplicates_raises_error(self) -> None: + """Test that multiple duplicate step names raise validation error.""" + steps = [ + ModelComputePipelineStep( + step_name="dup1", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ModelComputePipelineStep( + step_name="dup1", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ModelComputePipelineStep( + step_name="dup2", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ModelComputePipelineStep( + step_name="dup2", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ] + with pytest.raises(ValidationError, match="step names must be unique"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=steps, + ) + + def test_empty_pipeline_allowed(self) -> None: + """Test that empty pipeline is allowed.""" + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[], + ) + assert subcontract.pipeline == [] + + def test_single_step_allowed(self) -> None: + """Test that single step pipeline is allowed.""" + step = ModelComputePipelineStep( + step_name="only_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + subcontract = ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[step], + ) + assert len(subcontract.pipeline) == 1 + + def test_error_message_includes_duplicate_names(self) -> None: + """Test that error message includes the duplicate step names.""" + step1 = ModelComputePipelineStep( + step_name="my_duplicate", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + step2 = ModelComputePipelineStep( + step_name="my_duplicate", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ) + with pytest.raises(ValidationError, match="my_duplicate"): + ModelComputeSubcontract( + operation_name="test_op", + operation_version="1.0.0", + pipeline=[step1, step2], + ) diff --git a/tests/unit/models/core/conftest.py b/tests/unit/models/core/conftest.py new file mode 100644 index 0000000000..f8f05becea --- /dev/null +++ b/tests/unit/models/core/conftest.py @@ -0,0 +1,55 @@ +""" +Shared fixtures for models/core tests. + +Provides reusable fixtures for: +- CLI command definition creation +- Default version instances +""" + +import pytest + +from omnibase_core.models.core.model_cli_command_definition import ( + ModelCliCommandDefinition, +) +from omnibase_core.models.core.model_event_type import ModelEventType +from omnibase_core.models.core.model_node_reference import ModelNodeReference +from omnibase_core.models.primitives.model_semver import ModelSemVer + + +@pytest.fixture +def default_version(): + """Provide default SemVer version for tests.""" + return ModelSemVer(major=1, minor=0, patch=0) + + +@pytest.fixture +def create_test_command(default_version): + """Factory fixture to create test command definitions. + + Returns a factory function that creates ModelCliCommandDefinition instances. + + Usage: + def test_example(create_test_command): + command = create_test_command("my_cmd", "node_name") + assert command.command_name == "my_cmd" + """ + + def _create( + command_name: str, node_name: str, category: str = "general" + ) -> ModelCliCommandDefinition: + return ModelCliCommandDefinition( + version=default_version, + command_name=command_name, + target_node=ModelNodeReference.create_local(node_name), + action=command_name, + description=f"Test command {command_name}", + event_type=ModelEventType( + schema_version=default_version, + event_name="NODE_START", + namespace="onex", + description="Test event", + ), + category=category, + ) + + return _create diff --git a/tests/unit/models/core/test_model_container.py b/tests/unit/models/core/test_model_container.py index d87718451e..68904b4d62 100644 --- a/tests/unit/models/core/test_model_container.py +++ b/tests/unit/models/core/test_model_container.py @@ -180,7 +180,7 @@ def test_map_value_failure(self): container_type="string_container", ) - with pytest.raises(Exception) as exc_info: + with pytest.raises(OnexError) as exc_info: container.map_value(lambda x: int(x)) assert "Failed to map container value" in str(exc_info.value) @@ -200,7 +200,7 @@ def test_validate_with_failure(self): """Test validation failure.""" container = ModelContainer.create(value=-5, container_type="int_container") - with pytest.raises(Exception) as exc_info: + with pytest.raises(OnexError) as exc_info: container.validate_with(lambda x: x > 0, "Number must be positive") assert "Number must be positive" in str(exc_info.value) @@ -217,7 +217,7 @@ def test_validate_with_exception_in_validator(self): def failing_validator(x: str) -> bool: raise ValueError("Validator crashed") - with pytest.raises(Exception) as exc_info: + with pytest.raises(OnexError) as exc_info: container.validate_with(failing_validator) assert "Validation error:" in str(exc_info.value) diff --git a/tests/unit/models/health/test_model_health_check.py b/tests/unit/models/health/test_model_health_check.py index 0d5c51efc6..be43be29f1 100644 --- a/tests/unit/models/health/test_model_health_check.py +++ b/tests/unit/models/health/test_model_health_check.py @@ -1,7 +1,7 @@ """Tests for ModelHealthCheck.""" import pytest -from pydantic import HttpUrl +from pydantic import HttpUrl, ValidationError from omnibase_core.enums.enum_health_check_type import EnumHealthCheckType from omnibase_core.models.errors.model_onex_error import ModelOnexError @@ -161,13 +161,13 @@ def test_status_code_validation(self): ) # Invalid status codes - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthCheck( check_type=EnumHealthCheckType.HTTP_GET, endpoint_path="/health", expected_status_code=99, ) - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthCheck( check_type=EnumHealthCheckType.HTTP_GET, endpoint_path="/health", @@ -189,13 +189,13 @@ def test_timeout_validation(self): ) # Invalid timeouts - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthCheck( check_type=EnumHealthCheckType.HTTP_GET, endpoint_path="/health", timeout_seconds=0, ) - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthCheck( check_type=EnumHealthCheckType.HTTP_GET, endpoint_path="/health", diff --git a/tests/unit/models/health/test_model_health_metric.py b/tests/unit/models/health/test_model_health_metric.py index 62f9b24083..358f2b3e3b 100644 --- a/tests/unit/models/health/test_model_health_metric.py +++ b/tests/unit/models/health/test_model_health_metric.py @@ -1,6 +1,7 @@ """Tests for ModelHealthMetric.""" import pytest +from pydantic import ValidationError from omnibase_core.models.health.model_health_metric import ModelHealthMetric @@ -55,7 +56,7 @@ def test_trend_pattern_validation(self): ) # Invalid trend - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthMetric( metric_name="test", current_value=50.0, unit="%", trend="invalid" ) diff --git a/tests/unit/models/health/test_model_health_status.py b/tests/unit/models/health/test_model_health_status.py index 33022758bd..15cba927be 100644 --- a/tests/unit/models/health/test_model_health_status.py +++ b/tests/unit/models/health/test_model_health_status.py @@ -4,6 +4,7 @@ from uuid import uuid4 import pytest +from pydantic import ValidationError from omnibase_core.models.health.model_health_issue import ModelHealthIssue from omnibase_core.models.health.model_health_metric import ModelHealthMetric @@ -70,7 +71,7 @@ def test_status_pattern_validation(self): ModelHealthStatus(status="unknown", health_score=0.0) # Invalid status - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthStatus(status="invalid_status", health_score=1.0) def test_health_score_range_validation(self): @@ -81,9 +82,9 @@ def test_health_score_range_validation(self): ModelHealthStatus(status="healthy", health_score=1.0) # Invalid scores - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthStatus(status="healthy", health_score=-0.1) - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthStatus(status="healthy", health_score=1.1) def test_check_duration_validation(self): @@ -93,7 +94,7 @@ def test_check_duration_validation(self): ModelHealthStatus(status="healthy", health_score=1.0, check_duration_ms=1000) # Invalid durations - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthStatus(status="healthy", health_score=1.0, check_duration_ms=-1) def test_uptime_validation(self): @@ -103,7 +104,7 @@ def test_uptime_validation(self): ModelHealthStatus(status="healthy", health_score=1.0, uptime_seconds=3600) # Invalid uptimes - with pytest.raises(Exception): + with pytest.raises(ValidationError): ModelHealthStatus(status="healthy", health_score=1.0, uptime_seconds=-1) diff --git a/tests/unit/models/nodes/node_services/test_model_service_compute_health.py b/tests/unit/models/nodes/node_services/test_model_service_compute_health.py index 91f2aa9413..9b4b94d57c 100644 --- a/tests/unit/models/nodes/node_services/test_model_service_compute_health.py +++ b/tests/unit/models/nodes/node_services/test_model_service_compute_health.py @@ -128,6 +128,8 @@ def tool_invocation_event(node_id, correlation_id): ) +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeHealthStatus: """Test cases for health status retrieval and metrics.""" @@ -235,6 +237,8 @@ def test_get_service_health_success_rate(self, service_compute): assert health["success_rate"] == 1.0 +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeHealthMonitoring: """Test cases for health monitoring loop.""" @@ -364,6 +368,8 @@ def failing_get_health(): assert "Health check failed" in error_logs[0] +@pytest.mark.unit +@pytest.mark.timeout(60) class TestModelServiceComputeGracefulShutdown: """Test cases for graceful shutdown.""" @@ -646,6 +652,8 @@ async def mock_wait(timeout_ms): assert cleanup_called is True +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeHealthEdgeCases: """Test edge cases for health monitoring and shutdown.""" diff --git a/tests/unit/models/nodes/node_services/test_model_service_compute_integration.py b/tests/unit/models/nodes/node_services/test_model_service_compute_integration.py index f7c2005d30..2f2dd96596 100644 --- a/tests/unit/models/nodes/node_services/test_model_service_compute_integration.py +++ b/tests/unit/models/nodes/node_services/test_model_service_compute_integration.py @@ -70,6 +70,8 @@ def cleanup_event_handlers(self): """Mock cleanup.""" +@pytest.mark.unit +@pytest.mark.timeout(60) class TestModelServiceComputeIntegration: """Integration tests for ModelServiceCompute with all mixins.""" @@ -596,6 +598,8 @@ async def test_method_accessibility_from_all_mixins(self, service_compute): assert "status" in health +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeMRODetails: """Detailed MRO and diamond problem tests.""" diff --git a/tests/unit/models/nodes/node_services/test_model_service_compute_invocation.py b/tests/unit/models/nodes/node_services/test_model_service_compute_invocation.py index b091bc3f2c..684b861502 100644 --- a/tests/unit/models/nodes/node_services/test_model_service_compute_invocation.py +++ b/tests/unit/models/nodes/node_services/test_model_service_compute_invocation.py @@ -98,8 +98,13 @@ def tool_invocation_event(service_compute): ) +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeBasicInvocation: - """Test basic tool invocation handling.""" + """Test basic tool invocation handling. + + Note: 30-second timeout protects against service invocation hangs. + """ @pytest.mark.asyncio async def test_basic_tool_invocation_handling( @@ -221,6 +226,8 @@ async def test_target_node_validation_by_service_name( ) +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeTargetValidation: """Test target node validation logic.""" @@ -260,6 +267,8 @@ async def test_wrong_target_node_ignored(self, service_compute, mock_event_bus): assert service_compute._successful_invocations == 0 +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeInputStateConversion: """Test event-to-input-state conversion.""" @@ -378,6 +387,8 @@ async def test_convert_event_to_input_state_with_complex_params( assert input_state.tags == ["tag1", "tag2", "tag3"] +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeToolExecution: """Test tool execution via node.run().""" @@ -488,6 +499,8 @@ async def slow_run(input_state): assert response_event.execution_time_ms > 0 +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeResultSerialization: """Test result serialization.""" @@ -619,6 +632,8 @@ async def test_serialize_result_none( assert "valid result" in response_event.error.lower() +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeResponseEmission: """Test success and error response emission.""" @@ -678,6 +693,8 @@ async def test_emit_tool_response_error( assert service_compute._failed_invocations == 1 +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeCorrelationTracking: """Test correlation ID tracking.""" @@ -705,6 +722,8 @@ async def test_correlation_id_tracking( assert response_event.correlation_id == expected_correlation_id +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeActiveInvocations: """Test active invocation tracking.""" @@ -757,6 +776,8 @@ async def tracked_run(input_state): ) +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeConcurrentInvocations: """Test concurrent invocation handling.""" @@ -810,6 +831,8 @@ async def slow_run(input_state): assert mock_event_bus.publish.call_count == num_invocations +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputePureFunctionSemantics: """Test compute-specific pure function semantics.""" @@ -852,6 +875,8 @@ async def test_pure_function_semantics_validation( assert response.result == {"output": 84} +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeDeterministicOutput: """Test deterministic output verification.""" diff --git a/tests/unit/models/nodes/node_services/test_model_service_compute_lifecycle.py b/tests/unit/models/nodes/node_services/test_model_service_compute_lifecycle.py index 6278eb69f9..09877672ed 100644 --- a/tests/unit/models/nodes/node_services/test_model_service_compute_lifecycle.py +++ b/tests/unit/models/nodes/node_services/test_model_service_compute_lifecycle.py @@ -91,6 +91,8 @@ def tool_invocation_event(compute_node): ) +@pytest.mark.unit +@pytest.mark.timeout(60) class TestModelServiceComputeLifecycle: """Test ModelServiceCompute lifecycle operations.""" @@ -537,6 +539,8 @@ async def test_service_restart_cycles(self, compute_node): await asyncio.sleep(0.05) +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeToolInvocation: """Test tool invocation handling in service mode.""" @@ -600,6 +604,8 @@ async def test_handle_tool_invocation_wrong_target( assert compute_node._successful_invocations == 0 +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeHealthMonitoring: """Test health monitoring functionality.""" @@ -672,6 +678,8 @@ def test_get_service_health_with_invocations(self, compute_node): assert health["success_rate"] == 0.9 # 90/100 +@pytest.mark.unit +@pytest.mark.timeout(30) class TestModelServiceComputeShutdownHandling: """Test shutdown handling and callbacks.""" diff --git a/tests/unit/models/security/test_model_permission.py b/tests/unit/models/security/test_model_permission.py index 92680b2248..1992ebb7ba 100644 --- a/tests/unit/models/security/test_model_permission.py +++ b/tests/unit/models/security/test_model_permission.py @@ -23,6 +23,7 @@ from uuid import UUID import pytest +from pydantic import ValidationError # Add src to path for direct imports src_path = Path(__file__).parent.parent.parent.parent.parent / "src" @@ -150,7 +151,7 @@ def test_name_pattern_validation_valid(self): def test_name_pattern_validation_invalid(self): """Test invalid name patterns.""" - with pytest.raises(Exception): # Pydantic ValidationError + with pytest.raises(ValidationError): ModelPermission( version=DEFAULT_VERSION, name="123_invalid", diff --git a/tests/unit/models/transformations/test_transformation_configs.py b/tests/unit/models/transformations/test_transformation_configs.py new file mode 100644 index 0000000000..7f8b3401d2 --- /dev/null +++ b/tests/unit/models/transformations/test_transformation_configs.py @@ -0,0 +1,223 @@ +"""Unit tests for Contract-Driven NodeCompute v1.0 transformation configs.""" + +import pytest +from pydantic import ValidationError + +from omnibase_core.enums.enum_case_mode import EnumCaseMode +from omnibase_core.enums.enum_regex_flag import EnumRegexFlag +from omnibase_core.enums.enum_trim_mode import EnumTrimMode +from omnibase_core.enums.enum_unicode_form import EnumUnicodeForm +from omnibase_core.models.transformations.model_mapping_config import ModelMappingConfig +from omnibase_core.models.transformations.model_transform_case_config import ( + ModelTransformCaseConfig, +) +from omnibase_core.models.transformations.model_transform_json_path_config import ( + ModelTransformJsonPathConfig, +) +from omnibase_core.models.transformations.model_transform_regex_config import ( + ModelTransformRegexConfig, +) +from omnibase_core.models.transformations.model_transform_trim_config import ( + ModelTransformTrimConfig, +) +from omnibase_core.models.transformations.model_transform_unicode_config import ( + ModelTransformUnicodeConfig, +) +from omnibase_core.models.transformations.model_validation_step_config import ( + ModelValidationStepConfig, +) + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelTransformRegexConfig: + """Tests for ModelTransformRegexConfig.""" + + def test_create_minimal(self) -> None: + """Test creating config with minimal fields.""" + config = ModelTransformRegexConfig(pattern=r"\s+") + assert config.config_type == "regex" + assert config.pattern == r"\s+" + assert config.replacement == "" + assert config.flags == [] + + def test_create_full(self) -> None: + """Test creating config with all fields.""" + config = ModelTransformRegexConfig( + pattern=r"[a-z]+", + replacement="X", + flags=[EnumRegexFlag.IGNORECASE, EnumRegexFlag.MULTILINE], + ) + assert config.replacement == "X" + assert len(config.flags) == 2 + + def test_rejects_extra_fields(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelTransformRegexConfig(pattern=r"\d+", extra_field="bad") # type: ignore[call-arg] + + def test_is_frozen(self) -> None: + """Test that model is immutable.""" + config = ModelTransformRegexConfig(pattern=r"\d+") + with pytest.raises(ValidationError, match="frozen"): + config.pattern = r"\w+" # type: ignore[misc] + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelTransformCaseConfig: + """Tests for ModelTransformCaseConfig.""" + + def test_create_upper(self) -> None: + """Test creating uppercase config.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + assert config.config_type == "case_conversion" + assert config.mode == EnumCaseMode.UPPER + + def test_create_lower(self) -> None: + """Test creating lowercase config.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.LOWER) + assert config.mode == EnumCaseMode.LOWER + + def test_create_title(self) -> None: + """Test creating titlecase config.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.TITLE) + assert config.mode == EnumCaseMode.TITLE + + def test_rejects_extra_fields(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelTransformCaseConfig(mode=EnumCaseMode.UPPER, extra="bad") # type: ignore[call-arg] + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelTransformTrimConfig: + """Tests for ModelTransformTrimConfig.""" + + def test_default_mode(self) -> None: + """Test default mode is BOTH.""" + config = ModelTransformTrimConfig() + assert config.config_type == "trim" + assert config.mode == EnumTrimMode.BOTH + + def test_create_left(self) -> None: + """Test creating left trim config.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.LEFT) + assert config.mode == EnumTrimMode.LEFT + + def test_create_right(self) -> None: + """Test creating right trim config.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.RIGHT) + assert config.mode == EnumTrimMode.RIGHT + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelTransformUnicodeConfig: + """Tests for ModelTransformUnicodeConfig.""" + + def test_default_form(self) -> None: + """Test default form is NFC.""" + config = ModelTransformUnicodeConfig() + assert config.config_type == "normalize_unicode" + assert config.form == EnumUnicodeForm.NFC + + def test_create_nfd(self) -> None: + """Test creating NFD config.""" + config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFD) + assert config.form == EnumUnicodeForm.NFD + + def test_create_nfkc(self) -> None: + """Test creating NFKC config.""" + config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFKC) + assert config.form == EnumUnicodeForm.NFKC + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelTransformJsonPathConfig: + """Tests for ModelTransformJsonPathConfig.""" + + def test_create(self) -> None: + """Test creating JSONPath config.""" + config = ModelTransformJsonPathConfig(path="$.data.items") + assert config.config_type == "json_path" + assert config.path == "$.data.items" + + def test_path_root_only(self) -> None: + """Test that root-only path is valid.""" + config = ModelTransformJsonPathConfig(path="$") + assert config.path == "$" + + def test_path_empty_raises_error(self) -> None: + """Test that empty path raises validation error.""" + with pytest.raises(ValueError, match="path cannot be empty"): + ModelTransformJsonPathConfig(path="") + + def test_path_whitespace_only_raises_error(self) -> None: + """Test that whitespace-only path raises validation error.""" + with pytest.raises(ValueError, match="path cannot be empty"): + ModelTransformJsonPathConfig(path=" ") + + def test_path_without_dollar_raises_error(self) -> None: + """Test that path not starting with $ raises validation error.""" + with pytest.raises(ValueError, match="path must start with"): + ModelTransformJsonPathConfig(path="data.items") + + def test_path_with_dot_but_no_dollar_raises_error(self) -> None: + """Test that path starting with dot raises validation error.""" + with pytest.raises(ValueError, match="path must start with"): + ModelTransformJsonPathConfig(path=".data") + + def test_rejects_extra_fields(self) -> None: + """Test that extra fields are rejected.""" + with pytest.raises(ValidationError): + ModelTransformJsonPathConfig(path="$.x", extra="bad") # type: ignore[call-arg] + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelMappingConfig: + """Tests for ModelMappingConfig.""" + + def test_create(self) -> None: + """Test creating mapping config.""" + config = ModelMappingConfig( + field_mappings={ + "result": "$.steps.transform.output", + "original": "$.input.text", + } + ) + assert config.config_type == "mapping" + assert len(config.field_mappings) == 2 + assert config.field_mappings["result"] == "$.steps.transform.output" + + def test_empty_mappings_raises_error(self) -> None: + """Test that empty field_mappings raises validation error.""" + import pytest + + with pytest.raises(ValueError, match="field_mappings cannot be empty"): + ModelMappingConfig(field_mappings={}) + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestModelValidationStepConfig: + """Tests for ModelValidationStepConfig.""" + + def test_create_minimal(self) -> None: + """Test creating config with minimal fields.""" + config = ModelValidationStepConfig(schema_ref="input_schema") + assert config.config_type == "validation" + assert config.schema_ref == "input_schema" + assert config.fail_on_error is True # default + + def test_create_full(self) -> None: + """Test creating config with all fields.""" + config = ModelValidationStepConfig( + schema_ref="my_schema_v1", + fail_on_error=False, + ) + assert config.schema_ref == "my_schema_v1" + assert config.fail_on_error is False diff --git a/tests/unit/nodes/test_retry_count_tracking.py b/tests/unit/nodes/test_retry_count_tracking.py index 44ed3c743b..bff7ac8df0 100644 --- a/tests/unit/nodes/test_retry_count_tracking.py +++ b/tests/unit/nodes/test_retry_count_tracking.py @@ -28,6 +28,7 @@ def service_effect(mock_container): return ModelServiceEffect(mock_container) +@pytest.mark.unit class TestRetryCountTracking: """Test that retry_count is correctly tracked and returned.""" diff --git a/tests/unit/nodes/test_rollback_failure_handling.py b/tests/unit/nodes/test_rollback_failure_handling.py index 1751418547..41f96e0e1b 100644 --- a/tests/unit/nodes/test_rollback_failure_handling.py +++ b/tests/unit/nodes/test_rollback_failure_handling.py @@ -20,6 +20,7 @@ from omnibase_core.nodes.node_effect import NodeEffect +@pytest.mark.unit class TestModelEffectTransactionRollbackFailures: """Test ModelEffectTransaction rollback failure handling.""" @@ -154,6 +155,7 @@ def rollback_fail(): assert errors[0].__cause__ is original_error +@pytest.mark.unit class TestNodeEffectRollbackFailures: """Test NodeEffect rollback failure handling and callbacks.""" diff --git a/tests/unit/test_init_exports.py b/tests/unit/test_init_exports.py index 7121f6ab08..0392540e25 100644 --- a/tests/unit/test_init_exports.py +++ b/tests/unit/test_init_exports.py @@ -15,6 +15,7 @@ import pytest +@pytest.mark.unit class TestInitExports: """Test __init__.py public API exports.""" @@ -58,6 +59,7 @@ def test_init_exports_are_accessible(self): assert export_obj is not None, f"Export {export_name} is None" +@pytest.mark.unit class TestInitImportPerformance: """Test __init__.py import performance and lazy loading.""" @@ -117,6 +119,7 @@ def test_init_lazy_loading_delays_validation_imports(self): ) +@pytest.mark.unit class TestInitCircularImports: """Test circular import prevention in __init__.py.""" @@ -148,6 +151,7 @@ def test_init_getattr_handles_missing_attributes(self): assert "has no attribute 'NonExistentAttribute'" in str(exc_info.value) +@pytest.mark.unit class TestInitValidationExports: """Test validation tool exports from __init__.py.""" @@ -201,6 +205,7 @@ def test_init_validate_union_usage_export(self): assert callable(validate_union_usage) +@pytest.mark.unit class TestInitErrorExports: """Test error class exports from __init__.py.""" @@ -222,6 +227,7 @@ def test_init_model_onex_error_export(self): assert issubclass(ModelOnexError, Exception) +@pytest.mark.unit class TestInitLazyLoadingBehavior: """Test lazy loading behavior for different attributes.""" @@ -271,6 +277,7 @@ def test_init_multiple_lazy_loads_return_same_object(self): assert Error1 is Error2 +@pytest.mark.unit class TestInitModuleDocstring: """Test __init__.py module documentation.""" diff --git a/tests/unit/utils/test_compute_path_resolver.py b/tests/unit/utils/test_compute_path_resolver.py new file mode 100644 index 0000000000..371493c239 --- /dev/null +++ b/tests/unit/utils/test_compute_path_resolver.py @@ -0,0 +1,567 @@ +""" +Unit tests for compute_path_resolver shared utility. + +Tests the path resolution functions extracted for v1.1 to reduce duplication +between compute_executor.py and compute_transformations.py. + +Test Categories: + - Simple path resolution (resolve_path) + - Input path resolution (resolve_input_path) + - Step path resolution (resolve_step_path) + - Pipeline path resolution (resolve_pipeline_path) + - Private attribute security + - Error handling and edge cases +""" + +from dataclasses import dataclass +from typing import Any + +import pytest + +from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode +from omnibase_core.utils.compute_path_resolver import ( + PathResolutionError, + resolve_input_path, + resolve_path, + resolve_pipeline_path, + resolve_step_path, +) + + +# Test fixtures and helpers +@dataclass +class MockStepResult: + """Mock step result with output attribute for testing.""" + + output: Any + step_name: str = "mock_step" + + +@dataclass +class MockObject: + """Mock object with various attributes for testing.""" + + name: str + value: int + _private: str = "secret" + + +class NestedObject: + """Object with nested structure for testing attribute access.""" + + def __init__(self) -> None: + self.user = MockObject(name="Alice", value=42) + self.items = [1, 2, 3] + + +# ============================================================================= +# Test resolve_path (simple dot-notation paths) +# ============================================================================= + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestResolvePath: + """Tests for resolve_path function (simple dot-notation paths).""" + + def test_root_access_empty_string(self) -> None: + """Test empty string returns root object.""" + data = {"name": "test"} + assert resolve_path("", data) == data + + def test_root_access_dollar_sign(self) -> None: + """Test $ returns root object.""" + data = {"name": "test"} + assert resolve_path("$", data) == data + + def test_simple_field_access(self) -> None: + """Test simple field access without $ prefix.""" + data = {"name": "Alice", "age": 30} + assert resolve_path("name", data) == "Alice" + assert resolve_path("age", data) == 30 + + def test_field_access_with_dollar_prefix(self) -> None: + """Test field access with $. prefix.""" + data = {"name": "Bob", "active": True} + assert resolve_path("$.name", data) == "Bob" + assert resolve_path("$.active", data) is True + + def test_nested_field_access(self) -> None: + """Test nested field access.""" + data = {"user": {"name": "Charlie", "profile": {"age": 25}}} + assert resolve_path("user.name", data) == "Charlie" + assert resolve_path("$.user.profile.age", data) == 25 + + def test_deeply_nested_access(self) -> None: + """Test deeply nested field access.""" + data = {"a": {"b": {"c": {"d": {"e": "deep"}}}}} + assert resolve_path("a.b.c.d.e", data) == "deep" + assert resolve_path("$.a.b.c.d.e", data) == "deep" + + def test_object_attribute_access(self) -> None: + """Test accessing attributes on objects (not dicts).""" + obj = MockObject(name="Test", value=100) + assert resolve_path("name", obj) == "Test" + assert resolve_path("value", obj) == 100 + + def test_nested_object_attribute_access(self) -> None: + """Test nested object attribute access.""" + container = NestedObject() + assert resolve_path("user.name", container) == "Alice" + assert resolve_path("user.value", container) == 42 + + def test_mixed_dict_and_object_access(self) -> None: + """Test mixed dictionary and object attribute access.""" + data = {"container": NestedObject()} + assert resolve_path("container.user.name", data) == "Alice" + + def test_missing_key_raises_error(self) -> None: + """Test missing dictionary key raises PathResolutionError.""" + data = {"name": "test"} + with pytest.raises(PathResolutionError) as exc_info: + resolve_path("missing", data) + assert exc_info.value.error_code == EnumCoreErrorCode.OPERATION_FAILED + assert "missing" in str(exc_info.value.message) + + def test_missing_attribute_raises_error(self) -> None: + """Test missing object attribute raises PathResolutionError.""" + obj = MockObject(name="Test", value=1) + with pytest.raises(PathResolutionError) as exc_info: + resolve_path("nonexistent", obj) + assert exc_info.value.error_code == EnumCoreErrorCode.OPERATION_FAILED + + def test_private_attribute_blocked_by_default(self) -> None: + """Test private attribute access is blocked by default.""" + obj = MockObject(name="Test", value=1, _private="secret") + with pytest.raises(PathResolutionError) as exc_info: + resolve_path("_private", obj) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + assert "private" in str(exc_info.value.message).lower() + + def test_private_attribute_allowed_when_disabled(self) -> None: + """Test private attribute access when check_private=False.""" + obj = MockObject(name="Test", value=1, _private="secret") + result = resolve_path("_private", obj, check_private=False) + assert result == "secret" + + def test_underscore_dict_key_accessible(self) -> None: + """Test dictionary keys starting with _ are accessible.""" + data = {"_private_key": "allowed"} + assert resolve_path("_private_key", data) == "allowed" + + def test_none_value_accessible(self) -> None: + """Test None values can be accessed.""" + data = {"nullable": None} + assert resolve_path("nullable", data) is None + + def test_empty_string_value_accessible(self) -> None: + """Test empty string values can be accessed.""" + data = {"empty": ""} + assert resolve_path("empty", data) == "" + + def test_list_value_accessible(self) -> None: + """Test list values can be accessed.""" + data = {"items": [1, 2, 3]} + assert resolve_path("items", data) == [1, 2, 3] + + def test_boolean_values(self) -> None: + """Test boolean values can be accessed.""" + data = {"active": True, "deleted": False} + assert resolve_path("active", data) is True + assert resolve_path("deleted", data) is False + + +# ============================================================================= +# Test resolve_input_path ($.input prefix) +# ============================================================================= + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestResolveInputPath: + """Tests for resolve_input_path function ($.input prefix paths).""" + + def test_full_input_access(self) -> None: + """Test $.input returns full input object.""" + input_data = {"text": "hello", "count": 5} + assert resolve_input_path("$.input", input_data) == input_data + + def test_full_input_access_shorthand(self) -> None: + """Test $input shorthand returns full input object.""" + input_data = {"text": "hello"} + assert resolve_input_path("$input", input_data) == input_data + + def test_input_field_access(self) -> None: + """Test $.input. access.""" + input_data = {"text": "hello", "count": 5} + assert resolve_input_path("$.input.text", input_data) == "hello" + assert resolve_input_path("$.input.count", input_data) == 5 + + def test_input_field_access_shorthand(self) -> None: + """Test $input. shorthand access.""" + input_data = {"text": "hello"} + assert resolve_input_path("$input.text", input_data) == "hello" + + def test_nested_input_field_access(self) -> None: + """Test $.input.. access.""" + input_data = {"user": {"name": "John", "id": 123}} + assert resolve_input_path("$.input.user.name", input_data) == "John" + assert resolve_input_path("$.input.user.id", input_data) == 123 + + def test_deeply_nested_input_access(self) -> None: + """Test deeply nested input field access.""" + input_data = {"a": {"b": {"c": {"d": "deep"}}}} + assert resolve_input_path("$.input.a.b.c.d", input_data) == "deep" + + def test_missing_input_field_raises_error(self) -> None: + """Test missing input field raises error.""" + input_data = {"name": "test"} + with pytest.raises(PathResolutionError) as exc_info: + resolve_input_path("$.input.missing", input_data) + assert exc_info.value.error_code == EnumCoreErrorCode.OPERATION_FAILED + + def test_private_attribute_blocked_on_input_objects(self) -> None: + """Test private attribute access blocked on input objects.""" + input_data = MockObject(name="Test", value=1, _private="secret") + with pytest.raises(PathResolutionError) as exc_info: + resolve_input_path("$.input._private", input_data) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + + def test_underscore_dict_key_accessible_in_input(self) -> None: + """Test underscore dict keys are accessible in input.""" + input_data = {"_metadata": {"version": 1}} + assert resolve_input_path("$.input._metadata.version", input_data) == 1 + + def test_invalid_input_path_format_raises_error(self) -> None: + """Test invalid input path format raises error.""" + with pytest.raises(PathResolutionError) as exc_info: + resolve_input_path("$.other.field", {}) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + + +# ============================================================================= +# Test resolve_step_path ($.steps prefix) +# ============================================================================= + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestResolveStepPath: + """Tests for resolve_step_path function ($.steps prefix paths).""" + + def test_step_output_shorthand(self) -> None: + """Test $.steps. returns output (shorthand).""" + step_results = {"transform": MockStepResult(output="HELLO")} + assert resolve_step_path("$.steps.transform", step_results) == "HELLO" + + def test_step_output_explicit(self) -> None: + """Test $.steps..output returns output (explicit).""" + step_results = {"transform": MockStepResult(output="HELLO")} + assert resolve_step_path("$.steps.transform.output", step_results) == "HELLO" + + def test_step_output_shorthand_and_explicit_equivalent(self) -> None: + """Test shorthand and explicit forms return same value.""" + step_results = {"normalize": MockStepResult(output={"data": "value"})} + shorthand = resolve_step_path("$.steps.normalize", step_results) + explicit = resolve_step_path("$.steps.normalize.output", step_results) + assert shorthand == explicit + + def test_multiple_steps(self) -> None: + """Test resolving from multiple step results.""" + step_results = { + "step1": MockStepResult(output="first"), + "step2": MockStepResult(output="second"), + "step3": MockStepResult(output="third"), + } + assert resolve_step_path("$.steps.step1", step_results) == "first" + assert resolve_step_path("$.steps.step2", step_results) == "second" + assert resolve_step_path("$.steps.step3.output", step_results) == "third" + + def test_step_with_dict_output(self) -> None: + """Test step result with dictionary output.""" + step_results = {"transform": MockStepResult(output={"key": "value"})} + result = resolve_step_path("$.steps.transform", step_results) + assert result == {"key": "value"} + + def test_step_with_list_output(self) -> None: + """Test step result with list output.""" + step_results = {"collect": MockStepResult(output=[1, 2, 3])} + result = resolve_step_path("$.steps.collect", step_results) + assert result == [1, 2, 3] + + def test_step_with_none_output(self) -> None: + """Test step result with None output.""" + step_results = {"nullable": MockStepResult(output=None)} + result = resolve_step_path("$.steps.nullable", step_results) + assert result is None + + def test_missing_step_raises_error(self) -> None: + """Test missing step raises error.""" + step_results = {"existing": MockStepResult(output="value")} + with pytest.raises(PathResolutionError) as exc_info: + resolve_step_path("$.steps.nonexistent", step_results) + assert exc_info.value.error_code == EnumCoreErrorCode.OPERATION_FAILED + assert "nonexistent" in str(exc_info.value.message) + + def test_invalid_step_subpath_raises_error(self) -> None: + """Test invalid step subpath (not .output) raises error.""" + step_results = {"transform": MockStepResult(output="HELLO")} + with pytest.raises(PathResolutionError) as exc_info: + resolve_step_path("$.steps.transform.invalid", step_results) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + assert "only '.output' supported" in str(exc_info.value.message) + + def test_invalid_step_path_format_raises_error(self) -> None: + """Test invalid step path format raises error.""" + with pytest.raises(PathResolutionError) as exc_info: + resolve_step_path("$.input.field", {}) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + + def test_empty_step_name_raises_error(self) -> None: + """Test empty step name raises error.""" + with pytest.raises(PathResolutionError) as exc_info: + resolve_step_path("$.steps.", {}) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + + def test_step_result_without_output_attribute(self) -> None: + """Test step result that's just a value (no .output attribute).""" + # Some implementations may store raw values instead of objects + step_results: dict[str, Any] = {"direct": "raw_value"} + result = resolve_step_path("$.steps.direct", step_results) + assert result == "raw_value" + + +# ============================================================================= +# Test resolve_pipeline_path (unified resolver) +# ============================================================================= + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestResolvePipelinePath: + """Tests for resolve_pipeline_path function (unified pipeline resolver).""" + + def test_routes_to_input_path(self) -> None: + """Test input paths are routed correctly.""" + input_data = {"text": "hello"} + step_results: dict[str, Any] = {} + result = resolve_pipeline_path("$.input.text", input_data, step_results) + assert result == "hello" + + def test_routes_to_step_path(self) -> None: + """Test step paths are routed correctly.""" + input_data: dict[str, Any] = {} + step_results = {"transform": MockStepResult(output="HELLO")} + result = resolve_pipeline_path("$.steps.transform", input_data, step_results) + assert result == "HELLO" + + def test_full_input_via_pipeline_resolver(self) -> None: + """Test $.input via pipeline resolver.""" + input_data = {"data": "value"} + result = resolve_pipeline_path("$.input", input_data, {}) + assert result == input_data + + def test_input_shorthand_via_pipeline_resolver(self) -> None: + """Test $input shorthand via pipeline resolver.""" + input_data = {"data": "value"} + result = resolve_pipeline_path("$input", input_data, {}) + assert result == input_data + + def test_nested_input_via_pipeline_resolver(self) -> None: + """Test nested $.input paths via pipeline resolver.""" + input_data = {"user": {"name": "Alice"}} + result = resolve_pipeline_path("$.input.user.name", input_data, {}) + assert result == "Alice" + + def test_step_output_via_pipeline_resolver(self) -> None: + """Test $.steps paths via pipeline resolver.""" + step_results = {"process": MockStepResult(output=42)} + result = resolve_pipeline_path("$.steps.process.output", {}, step_results) + assert result == 42 + + def test_path_without_dollar_prefix_raises_error(self) -> None: + """Test path without $ prefix raises error.""" + with pytest.raises(PathResolutionError) as exc_info: + resolve_pipeline_path("input.field", {}, {}) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + assert "must start with '$'" in str(exc_info.value.message) + + def test_invalid_prefix_raises_error(self) -> None: + """Test invalid path prefix raises error.""" + with pytest.raises(PathResolutionError) as exc_info: + resolve_pipeline_path("$.other.path", {}, {}) + assert exc_info.value.error_code == EnumCoreErrorCode.VALIDATION_ERROR + assert "Invalid path prefix" in str(exc_info.value.message) + + +# ============================================================================= +# Test PathResolutionError +# ============================================================================= + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestPathResolutionError: + """Tests for PathResolutionError exception class.""" + + def test_basic_error_creation(self) -> None: + """Test basic error creation.""" + error = PathResolutionError( + message="Test error", + error_code=EnumCoreErrorCode.OPERATION_FAILED, + ) + assert "Test error" in str(error.message) + assert error.error_code == EnumCoreErrorCode.OPERATION_FAILED + + def test_error_with_path_context(self) -> None: + """Test error with path in context.""" + error = PathResolutionError( + message="Path not found", + path="$.input.missing", + ) + assert error.context is not None + # Custom context is stored in additional_context.context by ModelOnexError + additional = error.context.get("additional_context", {}) + inner_context = additional.get("context", {}) + assert inner_context.get("path") == "$.input.missing" + + def test_error_with_segment_context(self) -> None: + """Test error with segment in context.""" + error = PathResolutionError( + message="Missing key", + path="$.input.user.name", + segment="name", + ) + assert error.context is not None + # Custom context is stored in additional_context.context by ModelOnexError + additional = error.context.get("additional_context", {}) + inner_context = additional.get("context", {}) + assert inner_context.get("segment") == "name" + + def test_error_with_available_keys_context(self) -> None: + """Test error with available_keys in context.""" + error = PathResolutionError( + message="Key not found", + path="$.input.missing", + segment="missing", + available_keys=["name", "age", "email"], + ) + assert error.context is not None + # Custom context is stored in additional_context.context by ModelOnexError + additional = error.context.get("additional_context", {}) + inner_context = additional.get("context", {}) + assert inner_context.get("available_keys") == ["name", "age", "email"] + + def test_error_is_model_onex_error_subclass(self) -> None: + """Test PathResolutionError is a ModelOnexError subclass.""" + from omnibase_core.models.errors.model_onex_error import ModelOnexError + + error = PathResolutionError(message="Test") + assert isinstance(error, ModelOnexError) + + +# ============================================================================= +# Test edge cases and security +# ============================================================================= + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestEdgeCasesAndSecurity: + """Tests for edge cases and security considerations.""" + + def test_empty_dict_traversal(self) -> None: + """Test traversing into empty dict.""" + data: dict[str, Any] = {} + with pytest.raises(PathResolutionError): + resolve_path("missing", data) + + def test_nested_empty_dict(self) -> None: + """Test nested empty dict access.""" + data: dict[str, Any] = {"outer": {}} + with pytest.raises(PathResolutionError): + resolve_path("outer.inner", data) + + def test_path_with_consecutive_dots(self) -> None: + """Test path with consecutive dots (empty segments are skipped).""" + data = {"outer": {"inner": "value"}} + # Consecutive dots in middle of path (e.g., "outer..inner") result in + # empty segments that are skipped, equivalent to "outer.inner" + result = resolve_path("outer..inner", data) + assert result == "value" + + # Multiple consecutive dots (3+ dots) + result = resolve_path("outer...inner", data) + assert result == "value" + + # Trailing consecutive dots + data_simple = {"field": "value"} + result = resolve_path("field..", data_simple) + assert result == "value" + + # Leading consecutive dots (empty segments at start are skipped) + result = resolve_path("..outer.inner", data) + assert result == "value" + + # Mixed: leading, middle, and trailing consecutive dots + result = resolve_path("..outer..inner..", data) + assert result == "value" + + def test_path_with_trailing_dot(self) -> None: + """Test path with trailing dot.""" + data = {"field": "value"} + # Trailing dot results in empty final segment, skipped + result = resolve_path("field.", data) + assert result == "value" + + def test_unicode_keys(self) -> None: + """Test paths with unicode keys.""" + data = {"user_name": "test", "full_name": "Test User"} + assert resolve_path("user_name", data) == "test" + assert resolve_path("full_name", data) == "Test User" + + def test_numeric_string_keys(self) -> None: + """Test paths with numeric string keys.""" + data = {"123": "numeric_key", "0": "zero"} + assert resolve_path("123", data) == "numeric_key" + assert resolve_path("0", data) == "zero" + + def test_special_characters_in_keys(self) -> None: + """Test dictionary keys with special characters.""" + data = {"key-with-dash": "dash", "key_with_underscore": "underscore"} + # These should work as they're dict keys + assert resolve_path("key-with-dash", data) == "dash" + assert resolve_path("key_with_underscore", data) == "underscore" + + def test_very_long_path(self) -> None: + """Test resolution of very long paths.""" + # Build a deeply nested structure + depth = 50 + data: dict[str, Any] = {"value": "found"} + for i in range(depth - 1, -1, -1): + data = {f"level{i}": data} + + # Build the path + path = ".".join(f"level{i}" for i in range(depth)) + ".value" + assert resolve_path(path, data) == "found" + + def test_step_names_with_underscores(self) -> None: + """Test step names containing underscores.""" + step_results = { + "my_step": MockStepResult(output="value1"), + "another_longer_step_name": MockStepResult(output="value2"), + } + assert resolve_step_path("$.steps.my_step", step_results) == "value1" + assert ( + resolve_step_path("$.steps.another_longer_step_name", step_results) + == "value2" + ) + + def test_step_names_with_numbers(self) -> None: + """Test step names containing numbers.""" + step_results = { + "step1": MockStepResult(output="first"), + "process_v2": MockStepResult(output="second"), + } + assert resolve_step_path("$.steps.step1", step_results) == "first" + assert resolve_step_path("$.steps.process_v2", step_results) == "second" diff --git a/tests/unit/utils/test_compute_pipeline.py b/tests/unit/utils/test_compute_pipeline.py new file mode 100644 index 0000000000..066204d1d6 --- /dev/null +++ b/tests/unit/utils/test_compute_pipeline.py @@ -0,0 +1,597 @@ +"""Unit tests for Contract-Driven NodeCompute v1.0 transformations and executor.""" + +from uuid import uuid4 + +import pytest + +from omnibase_core.enums.enum_case_mode import EnumCaseMode +from omnibase_core.enums.enum_compute_step_type import EnumComputeStepType +from omnibase_core.enums.enum_regex_flag import EnumRegexFlag +from omnibase_core.enums.enum_transformation_type import EnumTransformationType +from omnibase_core.enums.enum_trim_mode import EnumTrimMode +from omnibase_core.enums.enum_unicode_form import EnumUnicodeForm +from omnibase_core.models.compute.model_compute_execution_context import ( + ModelComputeExecutionContext, +) +from omnibase_core.models.compute.model_compute_step_metadata import ( + ModelComputeStepMetadata, +) +from omnibase_core.models.compute.model_compute_step_result import ( + ModelComputeStepResult, +) +from omnibase_core.models.contracts.subcontracts.model_compute_pipeline_step import ( + ModelComputePipelineStep, +) +from omnibase_core.models.contracts.subcontracts.model_compute_subcontract import ( + ModelComputeSubcontract, +) +from omnibase_core.models.errors.model_onex_error import ModelOnexError +from omnibase_core.models.transformations.model_mapping_config import ModelMappingConfig +from omnibase_core.models.transformations.model_transform_case_config import ( + ModelTransformCaseConfig, +) +from omnibase_core.models.transformations.model_transform_json_path_config import ( + ModelTransformJsonPathConfig, +) +from omnibase_core.models.transformations.model_transform_regex_config import ( + ModelTransformRegexConfig, +) +from omnibase_core.models.transformations.model_transform_trim_config import ( + ModelTransformTrimConfig, +) +from omnibase_core.models.transformations.model_transform_unicode_config import ( + ModelTransformUnicodeConfig, +) +from omnibase_core.models.transformations.model_validation_step_config import ( + ModelValidationStepConfig, +) +from omnibase_core.utils.compute_executor import ( + execute_compute_pipeline, + resolve_mapping_path, +) +from omnibase_core.utils.compute_transformations import ( + execute_transformation, + transform_case, + transform_identity, + transform_json_path, + transform_regex, + transform_trim, + transform_unicode, +) + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestTransformIdentity: + """Tests for identity transformation.""" + + def test_returns_same_value(self) -> None: + """Test identity returns input unchanged.""" + assert transform_identity("hello", None) == "hello" + assert transform_identity(123, None) == 123 + assert transform_identity({"key": "value"}, None) == {"key": "value"} + assert transform_identity(None, None) is None + + def test_with_list(self) -> None: + """Test identity with list input.""" + data = [1, 2, 3] + assert transform_identity(data, None) == [1, 2, 3] + + def test_with_nested_structure(self) -> None: + """Test identity with nested data structure.""" + data = {"outer": {"inner": [1, 2, {"deep": True}]}} + assert transform_identity(data, None) == data + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestTransformRegex: + """Tests for regex transformation.""" + + def test_simple_replacement(self) -> None: + """Test simple regex replacement.""" + config = ModelTransformRegexConfig(pattern=r"\s+", replacement=" ") + result = transform_regex("hello world", config) + assert result == "hello world" + + def test_with_ignorecase(self) -> None: + """Test regex with IGNORECASE flag.""" + config = ModelTransformRegexConfig( + pattern=r"hello", + replacement="hi", + flags=[EnumRegexFlag.IGNORECASE], + ) + result = transform_regex("HELLO world", config) + assert result == "hi world" + + def test_with_multiline(self) -> None: + """Test regex with MULTILINE flag.""" + config = ModelTransformRegexConfig( + pattern=r"^line", + replacement="row", + flags=[EnumRegexFlag.MULTILINE], + ) + result = transform_regex("line1\nline2", config) + assert result == "row1\nrow2" + + def test_with_dotall(self) -> None: + """Test regex with DOTALL flag.""" + config = ModelTransformRegexConfig( + pattern=r"a.b", + replacement="X", + flags=[EnumRegexFlag.DOTALL], + ) + result = transform_regex("a\nb", config) + assert result == "X" + + def test_invalid_input_type(self) -> None: + """Test regex rejects non-string input.""" + config = ModelTransformRegexConfig(pattern=r"\d+") + with pytest.raises(ModelOnexError): + transform_regex(123, config) + + def test_invalid_regex_pattern(self) -> None: + """Test regex raises error on invalid pattern.""" + config = ModelTransformRegexConfig(pattern=r"[invalid", replacement="x") + with pytest.raises(ModelOnexError): + transform_regex("test", config) + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestTransformCase: + """Tests for case transformation.""" + + def test_uppercase(self) -> None: + """Test uppercase transformation.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + assert transform_case("hello world", config) == "HELLO WORLD" + + def test_lowercase(self) -> None: + """Test lowercase transformation.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.LOWER) + assert transform_case("HELLO WORLD", config) == "hello world" + + def test_titlecase(self) -> None: + """Test titlecase transformation.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.TITLE) + assert transform_case("hello world", config) == "Hello World" + + def test_invalid_input_type(self) -> None: + """Test case rejects non-string input.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + with pytest.raises(ModelOnexError): + transform_case(123, config) + + def test_empty_string(self) -> None: + """Test case transformation with empty string.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + assert transform_case("", config) == "" + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestTransformTrim: + """Tests for trim transformation.""" + + def test_trim_both(self) -> None: + """Test trimming both sides.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.BOTH) + assert transform_trim(" hello ", config) == "hello" + + def test_trim_left(self) -> None: + """Test trimming left side only.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.LEFT) + assert transform_trim(" hello ", config) == "hello " + + def test_trim_right(self) -> None: + """Test trimming right side only.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.RIGHT) + assert transform_trim(" hello ", config) == " hello" + + def test_trim_invalid_input_type(self) -> None: + """Test trim rejects non-string input.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.BOTH) + with pytest.raises(ModelOnexError): + transform_trim(123, config) + + def test_trim_no_whitespace(self) -> None: + """Test trim with no leading/trailing whitespace.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.BOTH) + assert transform_trim("hello", config) == "hello" + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestTransformUnicode: + """Tests for unicode normalization.""" + + def test_nfc_normalization(self) -> None: + """Test NFC normalization.""" + config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFC) + # e can be composed (NFC) or decomposed (NFD) + decomposed = "e\u0301" # e + combining acute accent + result = transform_unicode(decomposed, config) + assert result == "\xe9" # composed form (e-acute) + + def test_nfd_normalization(self) -> None: + """Test NFD normalization.""" + config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFD) + composed = "\xe9" # composed form (e-acute) + result = transform_unicode(composed, config) + assert len(result) == 2 # decomposed into e + combining accent + + def test_unicode_invalid_input_type(self) -> None: + """Test unicode rejects non-string input.""" + config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFC) + with pytest.raises(ModelOnexError): + transform_unicode(123, config) + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestTransformJsonPath: + """Tests for JSONPath extraction.""" + + def test_simple_field(self) -> None: + """Test extracting simple field.""" + config = ModelTransformJsonPathConfig(path="$.name") + data = {"name": "John", "age": 30} + assert transform_json_path(data, config) == "John" + + def test_nested_field(self) -> None: + """Test extracting nested field.""" + config = ModelTransformJsonPathConfig(path="$.user.name") + data = {"user": {"name": "Jane", "id": 123}} + assert transform_json_path(data, config) == "Jane" + + def test_missing_field(self) -> None: + """Test error on missing field.""" + config = ModelTransformJsonPathConfig(path="$.missing") + data = {"name": "John"} + with pytest.raises(ModelOnexError): + transform_json_path(data, config) + + def test_root_access(self) -> None: + """Test accessing root with $.""" + config = ModelTransformJsonPathConfig(path="$") + data = {"name": "John"} + assert transform_json_path(data, config) == data + + def test_deeply_nested_field(self) -> None: + """Test extracting deeply nested field.""" + config = ModelTransformJsonPathConfig(path="$.a.b.c.d") + data = {"a": {"b": {"c": {"d": "deep"}}}} + assert transform_json_path(data, config) == "deep" + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestExecuteTransformation: + """Tests for execute_transformation dispatcher.""" + + def test_identity_no_config(self) -> None: + """Test identity works without config.""" + result = execute_transformation("hello", EnumTransformationType.IDENTITY, None) + assert result == "hello" + + def test_case_with_config(self) -> None: + """Test case conversion with config.""" + config = ModelTransformCaseConfig(mode=EnumCaseMode.UPPER) + result = execute_transformation( + "hello", EnumTransformationType.CASE_CONVERSION, config + ) + assert result == "HELLO" + + def test_requires_config_for_non_identity(self) -> None: + """Test non-identity transformations require config.""" + with pytest.raises(ModelOnexError): + execute_transformation("hello", EnumTransformationType.REGEX, None) + + def test_trim_with_config(self) -> None: + """Test trim transformation via dispatcher.""" + config = ModelTransformTrimConfig(mode=EnumTrimMode.BOTH) + result = execute_transformation( + " hello ", EnumTransformationType.TRIM, config + ) + assert result == "hello" + + def test_unicode_with_config(self) -> None: + """Test unicode normalization via dispatcher.""" + config = ModelTransformUnicodeConfig(form=EnumUnicodeForm.NFC) + decomposed = "e\u0301" + result = execute_transformation( + decomposed, EnumTransformationType.NORMALIZE_UNICODE, config + ) + assert result == "\xe9" + + +@pytest.mark.unit +@pytest.mark.timeout(5) +class TestResolveMappingPath: + """Tests for resolve_mapping_path.""" + + def test_full_input(self) -> None: + """Test resolving $.input.""" + input_data = {"text": "hello"} + result = resolve_mapping_path("$.input", input_data, {}) + assert result == input_data + + def test_input_field(self) -> None: + """Test resolving $.input..""" + input_data = {"text": "hello", "count": 5} + result = resolve_mapping_path("$.input.text", input_data, {}) + assert result == "hello" + + def test_nested_input_field(self) -> None: + """Test resolving $.input...""" + input_data = {"user": {"name": "John", "id": 123}} + result = resolve_mapping_path("$.input.user.name", input_data, {}) + assert result == "John" + + def test_step_output(self) -> None: + """Test resolving $.steps..output.""" + metadata = ModelComputeStepMetadata(duration_ms=5.0) + step_result = ModelComputeStepResult( + step_name="transform", + output="HELLO", + metadata=metadata, + ) + step_results = {"transform": step_result} + + result = resolve_mapping_path("$.steps.transform.output", {}, step_results) + assert result == "HELLO" + + def test_step_without_output_suffix(self) -> None: + """Test resolving $.steps. returns output.""" + metadata = ModelComputeStepMetadata(duration_ms=5.0) + step_result = ModelComputeStepResult( + step_name="transform", + output="HELLO", + metadata=metadata, + ) + step_results = {"transform": step_result} + + result = resolve_mapping_path("$.steps.transform", {}, step_results) + assert result == "HELLO" + + def test_invalid_path_prefix(self) -> None: + """Test error on invalid path prefix.""" + with pytest.raises(ModelOnexError): + resolve_mapping_path("invalid.path", {}, {}) + + def test_missing_input_field(self) -> None: + """Test error on missing input field.""" + input_data = {"name": "John"} + with pytest.raises(ModelOnexError): + resolve_mapping_path("$.input.missing", input_data, {}) + + def test_missing_step(self) -> None: + """Test error on missing step.""" + with pytest.raises(ModelOnexError): + resolve_mapping_path("$.steps.nonexistent.output", {}, {}) + + def test_invalid_step_subpath(self) -> None: + """Test error on invalid step subpath.""" + metadata = ModelComputeStepMetadata(duration_ms=5.0) + step_result = ModelComputeStepResult( + step_name="transform", + output="HELLO", + metadata=metadata, + ) + step_results = {"transform": step_result} + + with pytest.raises(ModelOnexError): + resolve_mapping_path("$.steps.transform.invalid", {}, step_results) + + +@pytest.mark.unit +@pytest.mark.timeout(30) +class TestExecuteComputePipeline: + """Tests for execute_compute_pipeline. + + Note: 30-second timeout protects against pipeline execution hangs. + """ + + def test_simple_pipeline(self) -> None: + """Test executing a simple transformation pipeline.""" + contract = ModelComputeSubcontract( + operation_name="text_upper", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="to_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, "hello world", context) + + assert result.success is True + assert result.output == "HELLO WORLD" + assert result.steps_executed == ["to_upper"] + assert "to_upper" in result.step_results + + def test_multi_step_pipeline(self) -> None: + """Test executing a multi-step pipeline.""" + contract = ModelComputeSubcontract( + operation_name="text_normalize", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="trim", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ModelComputePipelineStep( + step_name="to_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, " hello world ", context) + + assert result.success is True + assert result.output == "HELLO WORLD" + assert result.steps_executed == ["trim", "to_upper"] + + def test_abort_on_failure(self) -> None: + """Test pipeline aborts on first failure.""" + contract = ModelComputeSubcontract( + operation_name="failing_pipeline", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="to_upper", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + ), + ModelComputePipelineStep( + step_name="never_reached", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + # Pass integer to cause failure in case conversion + result = execute_compute_pipeline(contract, 123, context) + + assert result.success is False + assert result.error_step == "to_upper" + assert "never_reached" not in result.step_results + + def test_identity_transformation(self) -> None: + """Test identity transformation in pipeline.""" + contract = ModelComputeSubcontract( + operation_name="passthrough", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="identity", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + # No transformation_config for IDENTITY + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, {"data": "unchanged"}, context) + + assert result.success is True + assert result.output == {"data": "unchanged"} + + def test_disabled_step_skipped(self) -> None: + """Test disabled steps are skipped.""" + contract = ModelComputeSubcontract( + operation_name="with_disabled", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="disabled_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.CASE_CONVERSION, + transformation_config=ModelTransformCaseConfig( + mode=EnumCaseMode.UPPER + ), + enabled=False, + ), + ModelComputePipelineStep( + step_name="enabled_step", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.TRIM, + transformation_config=ModelTransformTrimConfig( + mode=EnumTrimMode.BOTH + ), + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, " hello ", context) + + assert result.success is True + # Disabled step not executed, so not uppercase + assert result.output == "hello" + assert "disabled_step" not in result.steps_executed + assert "enabled_step" in result.steps_executed + + def test_empty_pipeline(self) -> None: + """Test empty pipeline returns input unchanged.""" + contract = ModelComputeSubcontract( + operation_name="empty", + operation_version="1.0.0", + pipeline=[], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, "input", context) + + assert result.success is True + assert result.output == "input" + assert result.steps_executed == [] + + def test_pipeline_tracks_timing(self) -> None: + """Test pipeline tracks processing time.""" + contract = ModelComputeSubcontract( + operation_name="timed", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="step1", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.IDENTITY, + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, "data", context) + + assert result.success is True + assert result.processing_time_ms >= 0 + assert result.step_results["step1"].metadata.duration_ms >= 0 + + def test_regex_transformation_in_pipeline(self) -> None: + """Test regex transformation in pipeline.""" + contract = ModelComputeSubcontract( + operation_name="regex_clean", + operation_version="1.0.0", + pipeline=[ + ModelComputePipelineStep( + step_name="normalize_spaces", + step_type=EnumComputeStepType.TRANSFORMATION, + transformation_type=EnumTransformationType.REGEX, + transformation_config=ModelTransformRegexConfig( + pattern=r"\s+", + replacement=" ", + ), + ), + ], + ) + context = ModelComputeExecutionContext(operation_id=uuid4()) + + result = execute_compute_pipeline(contract, "hello world", context) + + assert result.success is True + assert result.output == "hello world" diff --git a/tests/unit/utils/test_field_converter.py b/tests/unit/utils/test_field_converter.py index 7037c4de6e..08231b358b 100644 --- a/tests/unit/utils/test_field_converter.py +++ b/tests/unit/utils/test_field_converter.py @@ -6,6 +6,7 @@ from omnibase_core.utils.util_field_converter import FieldConverter +@pytest.mark.unit class TestFieldConverter: """Test FieldConverter functionality.""" diff --git a/tests/unit/utils/test_model_field_converter.py b/tests/unit/utils/test_model_field_converter.py index 2a915a121f..418b7e4641 100644 --- a/tests/unit/utils/test_model_field_converter.py +++ b/tests/unit/utils/test_model_field_converter.py @@ -453,7 +453,8 @@ def test_field_converter_frozen_dataclass(self) -> None: ) # Should not be able to modify frozen dataclass - with pytest.raises(Exception): # FrozenInstanceError in Python 3.11+ + # FrozenInstanceError (Python 3.10+) is a subclass of AttributeError + with pytest.raises(AttributeError): converter.field_name = "modified" # type: ignore[misc] def test_complex_conversion_workflow(self) -> None: diff --git a/tests/unit/validation/test_contracts.py b/tests/unit/validation/test_contracts.py index 093598743e..bfc9526cca 100644 --- a/tests/unit/validation/test_contracts.py +++ b/tests/unit/validation/test_contracts.py @@ -4,6 +4,7 @@ import pytest import yaml +from pydantic import ValidationError from omnibase_core.validation.contracts import ( load_and_validate_yaml_model, @@ -35,7 +36,7 @@ def test_load_empty_yaml(self): """Test loading empty YAML.""" yaml_content = "" - with pytest.raises(Exception): + with pytest.raises((ValidationError, TypeError)): load_and_validate_yaml_model(yaml_content) def test_load_invalid_yaml(self): @@ -43,7 +44,7 @@ def test_load_invalid_yaml(self): yaml_content = """ invalid: [unclosed """ - with pytest.raises(Exception): + with pytest.raises(yaml.YAMLError): load_and_validate_yaml_model(yaml_content) diff --git a/tests/unit/validation/test_contracts_extended.py b/tests/unit/validation/test_contracts_extended.py index ab7e2d0115..a4cb488d9d 100644 --- a/tests/unit/validation/test_contracts_extended.py +++ b/tests/unit/validation/test_contracts_extended.py @@ -19,6 +19,7 @@ import pytest import yaml +from pydantic import ValidationError from omnibase_core.enums.enum_core_error_code import EnumCoreErrorCode from omnibase_core.models.errors.model_onex_error import ModelOnexError @@ -96,7 +97,7 @@ def test_load_invalid_yaml_missing_required_field(self) -> None: operations: [] """ - with pytest.raises(Exception): # Pydantic validation error + with pytest.raises(ValidationError): load_and_validate_yaml_model(yaml_content) def test_load_yaml_with_extra_fields(self) -> None: @@ -125,14 +126,15 @@ def test_load_yaml_empty_string(self) -> None: yaml_content = "" # Empty YAML should fail validation - with pytest.raises(Exception): + with pytest.raises((ValidationError, TypeError)): load_and_validate_yaml_model(yaml_content) def test_load_yaml_whitespace_only(self) -> None: """Test loading whitespace-only YAML.""" yaml_content = " \n\t \n " - with pytest.raises(Exception): + # Tab characters in YAML can raise scanner errors + with pytest.raises((ValidationError, TypeError, yaml.YAMLError)): load_and_validate_yaml_model(yaml_content) def test_load_yaml_with_null_values(self) -> None: @@ -143,7 +145,7 @@ def test_load_yaml_with_null_values(self) -> None: operations: [] """ - with pytest.raises(Exception): + with pytest.raises(ValidationError): load_and_validate_yaml_model(yaml_content)