Repository navigation
Conversation
Add Alibaba-specific extensions to the GenAI metrics semantic conventions, including client token usage, operation duration, time-to-first-token, time-per- output-token, workflow duration, and skill duration metrics enriched with Alibaba infrastructure attributes (env, idc, RL experiment/group/job IDs). 🤖 Generated with [Qoder][https://qoder.com]
Restore full metric attribute groups (gen_ai.provider.name, gen_ai.operation.name, gen_ai.request/response.model, server.address/port, error.type) per reference spec. Include all client/server/workflow/skill metrics. Remove alibaba.* attributes - they belong only in the custom branch. 🤖 Generated with [Qoder][https://qoder.com]
Add client.time_to_first_token, client.time_per_output_token, workflow.duration, skill.duration, and inference engine metrics (prompt_tokens, generation_tokens, cached_tokens, num_requests_running/waiting, e2e_request_latency) to metrics.yaml. Add gen_ai.workflow.name and gen_ai.skill.* attributes to registry.yaml. Remove model/gen-ai/alibaba/metrics.yaml. 🤖 Generated with [Qoder][https://qoder.com]
- Remove duplicate time_to_first_chunk and time_per_output_chunk metric definitions from metrics.yaml (fixes table-check) - Add blank line before table separator in gen-ai-metrics.md (fixes MD058) - Add reference/ to .markdownlintignore - Add changelog entry 🤖 Generated with [Qoder][https://qoder.com]
| metric_value_type: int | ||
| brief: 'GenAI workflow invocation duration.' | ||
| instrument: histogram | ||
| unit: "ms" |
There was a problem hiding this comment.
I believe the same metrics is defined in "s" unit in upstream: https://github.com/open-telemetry/semantic-conventions/pull/3565/changes#diff-6d0e31edd5dc255528affad80f7475e07af0ccaaba6870fa6af97040e36d7975.
If we need a metrics with different unit, we should define it with a different name.
| stability: development | ||
| extends: metric_attributes.gen_ai | ||
|
|
||
| - id: metric.gen_ai.workflow.duration |
There was a problem hiding this comment.
Could we name this as metric.alibaba_group.gen_ai.xxx or metric.gen_ai.xxx.alibaba_group? Such as https://github.com/open-telemetry/semantic-conventions/blob/99e91d9f0867f8b027f976410ea6751d79002d93/model/gen-ai/spans.yaml#L145.
Metrics defined in group and could are so different that we should try to define them as precisely as possible. Perhaps it would be more reasonable to maintain separate implementations for the time being. cc @steverao
| - [Metric: `gen_ai.server.request.duration`](#metric-gen_aiserverrequestduration) | ||
| - [Metric: `gen_ai.server.time_per_output_token`](#metric-gen_aiservertime_per_output_token) | ||
| - [Metric: `gen_ai.server.time_to_first_token`](#metric-gen_aiservertime_to_first_token) | ||
| - [Workflow and Skill Metrics](#workflow-and-skill-metrics) |
There was a problem hiding this comment.
@clongbupt @steverao
If we plan to achieve these indicators separately, they may not be suitable to be included in gen-ai-metrics.md. It might be better to create a separate document for them.
Or at least we can distinguish the implementation of cloud and group using chapters.
Remove gen_ai.skill.duration metric definition from metrics.yaml and its documentation from gen-ai-metrics.md. The metric was removed from the YAML but the markdown still referenced it, causing table-generation to fail. 🤖 Generated with [Qoder][https://qoder.com]
Create docs/gen-ai/alibaba/gen-ai-metrics.md with Alibaba-specific GenAI metrics for workflow, agent, and tool operations: - metric.gen_ai.workflow.duration - metric.gen_ai.agent.duration - metric.gen_ai.tool.duration Move these metric definitions from model/gen-ai/metrics.yaml to model/alibaba/metrics.yaml to separate Alibaba-specific conventions. 🤖 Generated with [Qoder][https://qoder.com]
|
This PR contains changes to area(s) that do not have an active SIG/project and will be auto-closed:
Such changes may be rejected or put on hold until a new SIG/project is established. Please refer to the Semantic Convention Areas |
2 participants
Changes
Add more metric semantic conventions, follow community definitions below:
gen_ai.client.operation.time_to_first_chunkand `gen_ai.c… open-telemetry/semantic-conventions#3377Important
Pull requests acceptance are subject to the triage process as described in Issue and PR Triage Management.
PRs that do not follow the guidance above, may be automatically rejected and closed.
Merge requirement checklist
[chore]