-
Notifications
You must be signed in to change notification settings - Fork 369
Define dynamic request attributes for Google GenAI LLM inference spans #2125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
michaelsafyan
wants to merge
14
commits into
open-telemetry:main
from
michaelsafyan:gcp_genai_request_star_attributes
Closed
Changes from 2 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
005a551
Define dynamic request attribute for Google generative AI systems and…
michaelsafyan 0e1f9fc
Add a changelog entry.
michaelsafyan 2f02d20
Merge branch 'open-telemetry:main' into gcp_genai_request_star_attrib…
michaelsafyan 8833c97
Fix lint issue.
michaelsafyan e7b9410
Merge branch 'main' into gcp_genai_request_star_attributes
michaelsafyan 194362b
Update name, type, and add comments in response to PR feedback.
michaelsafyan 1119ed3
Fix typo 'to to'.
michaelsafyan 5265266
Merge branch 'main' into gcp_genai_request_star_attributes
michaelsafyan c9ba89b
Update attribute 'brief' description per PR feedback.
michaelsafyan d62d193
Merge branch 'main' into gcp_genai_request_star_attributes
michaelsafyan 6748c88
Merge branch 'open-telemetry:main' into gcp_genai_request_star_attrib…
michaelsafyan a750ee0
Update definition per feedback in LLM SIG.
michaelsafyan 9b2be36
Merge and resolve conflicts.
michaelsafyan dd4a6f3
Updated per PR feedback.
michaelsafyan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| change_type: enhancement | ||
| component: [gcp, gen-ai] | ||
| note: Define dynamic request attributes for Google generative AI systems. | ||
| issues: [2053] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| <!--- Hugo front matter used to generate the website version of this page: | ||
| linkTitle: Google Generative AI Semantic Conventions | ||
| ---> | ||
|
|
||
| # Semantic conventions for Google Generative AI operations | ||
|
|
||
| **Status**: [Development][DocumentStatus] | ||
|
|
||
| The Semantic Conventions for [Gemini](https://ai.google.dev/gemini-api/docs) and [Vertex AI](https://cloud.google.com/vertex-ai) extend and override the [GenAI Semantic Conventions](README.md). | ||
|
|
||
| ## Spans | ||
|
|
||
| <!-- semconv span.gen_ai.gcp.client --> | ||
| <!-- NOTE: THIS TEXT IS AUTOGENERATED. DO NOT EDIT BY HAND. --> | ||
| <!-- see templates/registry/markdown/snippet.md.j2 --> | ||
| <!-- prettier-ignore-start --> | ||
| <!-- markdownlint-capture --> | ||
| <!-- markdownlint-disable --> | ||
|
|
||
| **Status:**  | ||
|
|
||
| Semantic Conventions for model inference spans when calling Google systems. | ||
|
|
||
| `gen_ai.system` MUST be set to `"gcp.gen_ai"`, `"gcp.vertex_ai"`, or `"gcp.gemini"`. | ||
|
|
||
| **Span kind** SHOULD be `CLIENT`. | ||
|
|
||
| **Span status** SHOULD follow the [Recording Errors](/docs/general/recording-errors.md) document. | ||
|
|
||
| | Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability | | ||
| |---|---|---|---|---|---| | ||
| | [`gen_ai.operation.name`](/docs/attributes-registry/gen-ai.md) | string | The name of the operation being performed. [1] | `chat`; `generate_content`; `text_completion` | `Required` |  | | ||
| | [`error.type`](/docs/attributes-registry/error.md) | string | Describes a class of error the operation ended with. [2] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | `Conditionally Required` if the operation ended in an error |  | | ||
| | [`gcp.client.service`](/docs/attributes-registry/gcp.md) | string | Identifies the Google Cloud service for which the official client library is intended. [3] | `appengine`; `run`; `firestore`; `alloydb`; `spanner` | `Conditionally Required` If available |  | | ||
| | [`gen_ai.output.type`](/docs/attributes-registry/gen-ai.md) | string | Represents the content type requested by the client. [4] | `text`; `json`; `image` | `Conditionally Required` [5] |  | | ||
| | [`gen_ai.request.choice.count`](/docs/attributes-registry/gen-ai.md) | int | The target number of candidate completions to return. | `3` | `Conditionally Required` if available, in the request, and !=1 |  | | ||
| | [`gen_ai.request.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the GenAI model a request is being made to. [6] | `gpt-4` | `Conditionally Required` If available. |  | | ||
| | [`gen_ai.request.seed`](/docs/attributes-registry/gen-ai.md) | int | Requests with same seed value more likely to return same result. | `100` | `Conditionally Required` if applicable and if the request includes a seed |  | | ||
| | [`server.port`](/docs/attributes-registry/server.md) | int | GenAI server port. [7] | `80`; `8080`; `443` | `Conditionally Required` If `server.address` is set. |  | | ||
| | [`gen_ai.request.encoding_formats`](/docs/attributes-registry/gen-ai.md) | string[] | The encoding formats requested in an embeddings operation, if specified. [8] | `["base64"]`; `["float", "binary"]` | `Recommended` |  | | ||
| | [`gen_ai.request.frequency_penalty`](/docs/attributes-registry/gen-ai.md) | double | The frequency penalty setting for the GenAI request. | `0.1` | `Recommended` |  | | ||
| | [`gen_ai.request.max_tokens`](/docs/attributes-registry/gen-ai.md) | int | The maximum number of tokens the model generates for a request. | `100` | `Recommended` |  | | ||
| | [`gen_ai.request.presence_penalty`](/docs/attributes-registry/gen-ai.md) | double | The presence penalty setting for the GenAI request. | `0.1` | `Recommended` |  | | ||
| | [`gen_ai.request.stop_sequences`](/docs/attributes-registry/gen-ai.md) | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | `Recommended` |  | | ||
| | [`gen_ai.request.temperature`](/docs/attributes-registry/gen-ai.md) | double | The temperature setting for the GenAI request. | `0.0` | `Recommended` |  | | ||
| | [`gen_ai.request.top_p`](/docs/attributes-registry/gen-ai.md) | double | The top_p sampling setting for the GenAI request. | `1.0` | `Recommended` |  | | ||
| | [`gen_ai.response.finish_reasons`](/docs/attributes-registry/gen-ai.md) | string[] | Array of reasons the model stopped generating tokens, corresponding to each generation received. | `["stop"]`; `["stop", "length"]` | `Recommended` |  | | ||
| | [`gen_ai.response.id`](/docs/attributes-registry/gen-ai.md) | string | The unique identifier for the completion. | `chatcmpl-123` | `Recommended` |  | | ||
| | [`gen_ai.response.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the model that generated the response. [9] | `gpt-4-0613` | `Recommended` |  | | ||
| | [`gen_ai.usage.input_tokens`](/docs/attributes-registry/gen-ai.md) | int | The number of tokens used in the GenAI input (prompt). | `100` | `Recommended` |  | | ||
| | [`gen_ai.usage.output_tokens`](/docs/attributes-registry/gen-ai.md) | int | The number of tokens used in the GenAI response (completion). | `180` | `Recommended` |  | | ||
| | [`server.address`](/docs/attributes-registry/server.md) | string | GenAI server address. [10] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | `Recommended` |  | | ||
| | [`gcp.gen_ai.request.<key>`](/docs/attributes-registry/gcp.md) | string[] | Request configuration options named in a Google-specific manner. [11] | | `Opt-In` |  | | ||
|
|
||
| **[1] `gen_ai.operation.name`:** If one of the predefined values applies, but specific system uses a different name it's RECOMMENDED to document it in the semantic conventions for specific GenAI system and use system-specific name in the instrumentation. If a different name is not documented, instrumentation libraries SHOULD use applicable predefined value. | ||
|
|
||
| **[2] `error.type`:** The `error.type` SHOULD match the error code returned by the Generative AI provider or the client library, | ||
| the canonical name of exception that occurred, or another low-cardinality error identifier. | ||
| Instrumentations SHOULD document the list of errors they report. | ||
|
|
||
| **[3] `gcp.client.service`:** Intended to be a stable identifier for Google Cloud client libraries that is uniform across implementation languages. The value should be derived from the canonical service domain for the service; for example, 'foo.googleapis.com' should result in a value of 'foo'. | ||
|
|
||
| **[4] `gen_ai.output.type`:** This attribute SHOULD be used when the client requests output of a specific type. The model may return zero or more outputs of this type. | ||
| This attribute specifies the output modality and not the actual output format. For example, if an image is requested, the actual output could be a URL pointing to an image file. | ||
| Additional output format details may be recorded in the future in the `gen_ai.output.{type}.*` attributes. | ||
|
|
||
| **[5] `gen_ai.output.type`:** when applicable and if the request includes an output format. | ||
|
|
||
| **[6] `gen_ai.request.model`:** The name of the GenAI model a request is being made to. If the model is supplied by a vendor, then the value must be the exact name of the model requested. If the model is a fine-tuned custom model, the value should have a more specific name than the base model that's been fine-tuned. | ||
|
|
||
| **[7] `server.port`:** When observed from the client side, and when communicating through an intermediary, `server.port` SHOULD represent the server port behind any intermediaries, for example proxies, if it's available. | ||
|
|
||
| **[8] `gen_ai.request.encoding_formats`:** In some GenAI systems the encoding formats are called embedding types. Also, some GenAI systems only accept a single format per request. | ||
|
|
||
| **[9] `gen_ai.response.model`:** If available. The name of the GenAI model that provided the response. If the model is supplied by a vendor, then the value must be the exact name of the model actually used. If the model is a fine-tuned custom model, the value should have a more specific name than the base model that's been fine-tuned. | ||
|
|
||
| **[10] `server.address`:** When observed from the client side, and when communicating through an intermediary, `server.address` SHOULD represent the server address behind any intermediaries, for example proxies, if it's available. | ||
|
|
||
| **[11] `gcp.gen_ai.request`:** Key names are derived from the data structures in the Google Gen AI SDK ("google.genai.types.*" in https://github.com/googleapis/python-genai). | ||
| Key names are dependent on the particular operation in question. Each operation has its corresponding "*Config" data structure; for example, the "generate_content" operation has "GenerateContentConfig", while "generate_images" has a corresponding "GenerateImagesConfig". | ||
| Key names correspond to a flattened version of the config data structure. For example {"foo": {"bar": {"baz": 5}}} results in a key called "gcp.gen_ai.request.foo.bar.baz" with value 5. | ||
|
|
||
| --- | ||
|
|
||
| `error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | ||
|
|
||
| | Value | Description | Stability | | ||
| |---|---|---| | ||
| | `_OTHER` | A fallback error value to be used when the instrumentation doesn't define a custom value. |  | | ||
|
|
||
| --- | ||
|
|
||
| `gen_ai.operation.name` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | ||
|
|
||
| | Value | Description | Stability | | ||
| |---|---|---| | ||
| | `chat` | Chat completion operation such as [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) |  | | ||
| | `create_agent` | Create GenAI agent |  | | ||
| | `embeddings` | Embeddings operation such as [OpenAI Create embeddings API](https://platform.openai.com/docs/api-reference/embeddings/create) |  | | ||
| | `execute_tool` | Execute a tool |  | | ||
| | `generate_content` | Multimodal content generation operation such as [Gemini Generate Content](https://ai.google.dev/api/generate-content) |  | | ||
| | `text_completion` | Text completions operation such as [OpenAI Completions API (Legacy)](https://platform.openai.com/docs/api-reference/completions) |  | | ||
|
|
||
| --- | ||
|
|
||
| `gen_ai.output.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | ||
|
|
||
| | Value | Description | Stability | | ||
| |---|---|---| | ||
| | `image` | Image |  | | ||
| | `json` | JSON object with known or unknown schema |  | | ||
| | `speech` | Speech |  | | ||
| | `text` | Plain text |  | | ||
|
|
||
| <!-- markdownlint-restore --> | ||
| <!-- prettier-ignore-end --> | ||
| <!-- END AUTOGENERATED TEXT --> | ||
| <!-- endsemconv --> | ||
|
|
||
| ## Events | ||
|
|
||
| Google generative AI events follow generic [Generative AI events](gen-ai-events.md). | ||
|
|
||
| ## Metrics | ||
|
|
||
| Google generative AI metrics follow generic [Generative AI metrics](gen-ai-metrics.md). | ||
|
|
||
| [DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| groups: | ||
| - id: attributes.gcp.gen_ai | ||
| type: attribute_group | ||
| name: gcp.gen_ai | ||
| stability: development | ||
| brief: > | ||
| Attributes that are used for Google generative AI systems | ||
| and that are common to multiple Google endpoints (i.e. common | ||
| to both the Vertex AI and to the Gemini API endpoints). | ||
| attributes: | ||
| - ref: gcp.gen_ai.request | ||
|
|
||
| - id: attributes.gcp.vertex_ai | ||
| type: attribute_group | ||
| extends: attributes.gcp.gen_ai | ||
| name: gcp.vertex_ai | ||
| stability: development | ||
| brief: > | ||
| Attributes that are used for Vertex AI. | ||
| attributes: | ||
| - ref: gcp.client.service | ||
| note: Should be "aiplatform" for the Vertex AI endpoint. | ||
|
|
||
| - id: attributes.gcp.gemini | ||
| type: attribute_group | ||
| extends: attributes.gcp.gen_ai | ||
| name: gcp.gemini | ||
| stability: development | ||
| brief: > | ||
| Attributes that are used for Gemini. | ||
| attributes: | ||
| - ref: gcp.client.service | ||
| note: Should be "generativelanguage" for the Gemini endpoint. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.