Repository navigation
feat(OMN-2318): integrate SPI 0.9.0 LLM cost tracking contracts #345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
bac4976
feat(OMN-2318): integrate SPI 0.9.0 LLM cost tracking contracts into …
jonahgabriel c3f9ca5
fix(review): [major] correct has_usage provenance bug + [minor] edge …
jonahgabriel 466c0b6
fix(review): [major] consistent usage_source provenance + migration c…
jonahgabriel 52614a3
fix(review): [minor] simplify converter total-token fallback
jonahgabriel 622df41
fix(review): [minor] type-safe tokens_total + Ollama isinstance cleanup
jonahgabriel 444f42e
fix(ci): regenerate schema fingerprint after migration 031 changes
jonahgabriel cdfa893
fix(review): address all PR #345 CodeRabbit review issues
jonahgabriel 6527a9f
fix(review): [minor] widen input_hash column, provision LLM topic, re…
jonahgabriel 1ffa012
fix(review): [minor] guard against inconsistent provider total_tokens
jonahgabriel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| # Schema migration fingerprint for omnibase_infra (auto-generated) | ||
| # Regenerate: python scripts/check_schema_fingerprint.py stamp | ||
| # Verify: python scripts/check_schema_fingerprint.py verify | ||
| sha256:56ef7e39b9d1af17bea9caa92336ce3b4acca18e30ac9c92cc539a42f1547135 | ||
| generated_at: 2026-02-15T23:12:51Z | ||
| sha256:0db5f5c925c17240690fc1f82eeabeeb599bd12c2d5eedc40863a3e2a702ef13 | ||
| generated_at: 2026-02-16T19:23:56Z | ||
| migration_file_count: 17 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
146 changes: 146 additions & 0 deletions
146
src/omnibase_infra/nodes/effects/models/adapter_llm_usage_to_contract.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| # SPDX-License-Identifier: MIT | ||
| # Copyright (c) 2025 OmniNode Team | ||
| """Adapter bridging ModelLlmUsage to SPI LLM cost tracking contracts. | ||
|
|
||
| This module provides pure functions that translate between the infra-layer | ||
| ``ModelLlmUsage`` value object and the SPI measurement contracts: | ||
|
|
||
| - ``ContractLlmCallMetrics`` | ||
| - ``ContractLlmUsageRaw`` | ||
| - ``ContractLlmUsageNormalized`` | ||
| - ``ContractEnumUsageSource`` | ||
|
|
||
| All functions are stateless and produce frozen Pydantic models suitable for | ||
| downstream measurement pipeline ingestion. | ||
|
|
||
| Related: | ||
| - OMN-2318: Integrate SPI 0.9.0 LLM cost tracking contracts | ||
| - ModelLlmUsage: Source infra-layer usage model | ||
| - ContractLlmCallMetrics: Target SPI per-call metrics contract | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from omnibase_infra.nodes.effects.models.model_llm_usage import ModelLlmUsage | ||
| from omnibase_spi.contracts.measurement import ( | ||
| ContractEnumUsageSource, | ||
| ContractLlmCallMetrics, | ||
| ContractLlmUsageNormalized, | ||
| ContractLlmUsageRaw, | ||
| ) | ||
|
|
||
|
|
||
| def to_usage_raw( | ||
| usage: ModelLlmUsage, | ||
| provider: str = "", | ||
| ) -> ContractLlmUsageRaw: | ||
| """Convert raw provider usage data from ModelLlmUsage to ContractLlmUsageRaw. | ||
|
|
||
| When ``usage.raw_provider_usage`` is ``None``, returns an empty raw | ||
| container with the given provider identifier. | ||
|
|
||
| Args: | ||
| usage: The infra-layer usage model. | ||
| provider: Provider identifier (e.g. ``"openai"``, ``"ollama"``, | ||
| ``"vllm"``). | ||
|
|
||
| Returns: | ||
| A frozen ``ContractLlmUsageRaw`` preserving the verbatim provider data. | ||
| """ | ||
| return ContractLlmUsageRaw( | ||
| provider=provider, | ||
| raw_data=usage.raw_provider_usage or {}, | ||
| ) | ||
|
|
||
|
|
||
| def to_usage_normalized(usage: ModelLlmUsage) -> ContractLlmUsageNormalized: | ||
| """Convert ModelLlmUsage token counts to ContractLlmUsageNormalized. | ||
|
|
||
| Maps infra-layer field names (``tokens_input``, ``tokens_output``, | ||
| ``tokens_total``) to the SPI canonical names (``prompt_tokens``, | ||
| ``completion_tokens``, ``total_tokens``). | ||
|
|
||
| The ``source`` and ``usage_is_estimated`` fields are derived from | ||
| ``usage.usage_source``: | ||
|
|
||
| - ``API`` -> ``source=API, usage_is_estimated=False`` | ||
| - ``ESTIMATED`` -> ``source=ESTIMATED, usage_is_estimated=True`` | ||
| - ``MISSING`` -> ``source=MISSING, usage_is_estimated=False`` | ||
|
|
||
| Args: | ||
| usage: The infra-layer usage model. | ||
|
|
||
| Returns: | ||
| A frozen ``ContractLlmUsageNormalized`` with canonical token counts | ||
| and provenance. | ||
| """ | ||
| source = usage.usage_source | ||
| usage_is_estimated = source == ContractEnumUsageSource.ESTIMATED | ||
| total = usage.tokens_total or 0 | ||
|
|
||
| return ContractLlmUsageNormalized( | ||
| prompt_tokens=usage.tokens_input, | ||
| completion_tokens=usage.tokens_output, | ||
| total_tokens=total, | ||
| source=source, | ||
| usage_is_estimated=usage_is_estimated, | ||
| ) | ||
|
|
||
|
|
||
| def to_call_metrics( | ||
| usage: ModelLlmUsage, | ||
| model_id: str, | ||
| *, | ||
| provider: str = "", | ||
| latency_ms: float | None = None, | ||
| timestamp_iso: str = "", | ||
| reporting_source: str = "", | ||
| ) -> ContractLlmCallMetrics: | ||
| """Convert ModelLlmUsage to a full ContractLlmCallMetrics record. | ||
|
|
||
| Builds both the raw and normalized usage sub-contracts and assembles | ||
| them into the top-level per-call metrics contract. | ||
|
|
||
| Args: | ||
| usage: The infra-layer usage model. | ||
| model_id: Identifier of the LLM model (e.g. ``"gpt-4o"``, | ||
| ``"qwen2.5-coder-14b"``). Required and must be non-empty. | ||
| provider: Provider identifier for the raw usage envelope. | ||
| latency_ms: End-to-end call latency in milliseconds. | ||
| timestamp_iso: ISO-8601 timestamp string of the call. | ||
| reporting_source: Provenance label for this metrics record | ||
| (e.g. ``"llm-inference-effect"``). | ||
|
|
||
| Returns: | ||
| A frozen ``ContractLlmCallMetrics`` with token counts, cost, | ||
| raw/normalized usage, and metadata. | ||
|
|
||
| Raises: | ||
| ValueError: If *model_id* is empty. | ||
| """ | ||
| if not model_id: | ||
| raise ValueError("model_id must be a non-empty string") | ||
|
|
||
| total = usage.tokens_total or 0 | ||
| is_estimated = usage.usage_source == ContractEnumUsageSource.ESTIMATED | ||
|
|
||
| return ContractLlmCallMetrics( | ||
| model_id=model_id, | ||
| prompt_tokens=usage.tokens_input, | ||
| completion_tokens=usage.tokens_output, | ||
| total_tokens=total, | ||
| estimated_cost_usd=usage.cost_usd, | ||
| latency_ms=latency_ms, | ||
| usage_raw=to_usage_raw(usage, provider=provider), | ||
| usage_normalized=to_usage_normalized(usage), | ||
| usage_is_estimated=is_estimated, | ||
| timestamp_iso=timestamp_iso, | ||
| reporting_source=reporting_source, | ||
| ) | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
|
|
||
| __all__ = [ | ||
| "to_call_metrics", | ||
| "to_usage_normalized", | ||
| "to_usage_raw", | ||
| ] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename module to match required Python file naming patterns.
The current filename
converter_llm_usage_to_contract.pydoes not match the allowed prefixes. Please rename it (and update imports) to an approved pattern (e.g.,adapter_llm_usage_to_contract.pyormodel_llm_usage_to_contract.py, depending on intent).As per coding guidelines, File naming must follow pattern: model_.py, adapter_.py, dispatcher_.py, enum_.py, mixin_.py, protocol_.py, service_.py, store_.py, validator_.py, registry_infra_.py.
Also applies to: 136-140
🤖 Prompt for AI Agents