[GenAI Connectors] Token telemetry#186936
Merged
stephmilovic merged 5 commits intoelastic:mainfrom Jun 28, 2024
Merged
Conversation
stephmilovic
added
release_note:skip
stephmilovic
commented
Jun 26, 2024
| }, | ||
| name: undefined, | ||
| id: 'action1', | ||
| describe('Event log', () => { |
Contributor
Author
There was a problem hiding this comment.
All the spacing changes below are due to adding this describe wrapper. The only new test is reports telemetry for token count events
Contributor
Author
|
@elasticmachine merge upstream |
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]
To update your PR or re-run it, just comment with: |
3 tasks
P1llus
added a commit
that referenced
this pull request
Feb 6, 2025
## Summary This extends initial connector telemetry from PR ref #186936. The PR adds the following optional fields when instantiating a new actionClient as part of its `subActionParams`: ```ts { telemetryMetadata : { pluginId: "your plugin name or unique identifier", aggregateBy: "ID to aggregate on" } } ``` The support is added to all AI connector models for both stream/non-stream/raw. The PR also adds token count usage for bedrock `InvokeAIRaw`, as that was currently not added correctly. Pierre also helped with adding a new metadata optional field for the `NL to ESQL functions`, so that users can pass in similar metadata for LLM conversations using the InfereceClient. PluginId is a field used to filter telemetry in the way the team wants to implement it. It could be a team name, a plugin name etc, all depending on how the team wants to group and filter on the telemetry event. AggregateBy is intended to be used to group multiple LLM calls for aggregations and stats, for example a conversationId that has multiple LLM calls. Both fields are optional, so when you do not want to aggregate the option can simply be ignored. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 6, 2025
## Summary This extends initial connector telemetry from PR ref elastic#186936. The PR adds the following optional fields when instantiating a new actionClient as part of its `subActionParams`: ```ts { telemetryMetadata : { pluginId: "your plugin name or unique identifier", aggregateBy: "ID to aggregate on" } } ``` The support is added to all AI connector models for both stream/non-stream/raw. The PR also adds token count usage for bedrock `InvokeAIRaw`, as that was currently not added correctly. Pierre also helped with adding a new metadata optional field for the `NL to ESQL functions`, so that users can pass in similar metadata for LLM conversations using the InfereceClient. PluginId is a field used to filter telemetry in the way the team wants to implement it. It could be a team name, a plugin name etc, all depending on how the team wants to group and filter on the telemetry event. AggregateBy is intended to be used to group multiple LLM calls for aggregations and stats, for example a conversationId that has multiple LLM calls. Both fields are optional, so when you do not want to aggregate the option can simply be ignored. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> (cherry picked from commit 3394b69)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 6, 2025
## Summary This extends initial connector telemetry from PR ref elastic#186936. The PR adds the following optional fields when instantiating a new actionClient as part of its `subActionParams`: ```ts { telemetryMetadata : { pluginId: "your plugin name or unique identifier", aggregateBy: "ID to aggregate on" } } ``` The support is added to all AI connector models for both stream/non-stream/raw. The PR also adds token count usage for bedrock `InvokeAIRaw`, as that was currently not added correctly. Pierre also helped with adding a new metadata optional field for the `NL to ESQL functions`, so that users can pass in similar metadata for LLM conversations using the InfereceClient. PluginId is a field used to filter telemetry in the way the team wants to implement it. It could be a team name, a plugin name etc, all depending on how the team wants to group and filter on the telemetry event. AggregateBy is intended to be used to group multiple LLM calls for aggregations and stats, for example a conversationId that has multiple LLM calls. Both fields are optional, so when you do not want to aggregate the option can simply be ignored. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> (cherry picked from commit 3394b69)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 6, 2025
## Summary This extends initial connector telemetry from PR ref elastic#186936. The PR adds the following optional fields when instantiating a new actionClient as part of its `subActionParams`: ```ts { telemetryMetadata : { pluginId: "your plugin name or unique identifier", aggregateBy: "ID to aggregate on" } } ``` The support is added to all AI connector models for both stream/non-stream/raw. The PR also adds token count usage for bedrock `InvokeAIRaw`, as that was currently not added correctly. Pierre also helped with adding a new metadata optional field for the `NL to ESQL functions`, so that users can pass in similar metadata for LLM conversations using the InfereceClient. PluginId is a field used to filter telemetry in the way the team wants to implement it. It could be a team name, a plugin name etc, all depending on how the team wants to group and filter on the telemetry event. AggregateBy is intended to be used to group multiple LLM calls for aggregations and stats, for example a conversationId that has multiple LLM calls. Both fields are optional, so when you do not want to aggregate the option can simply be ignored. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> (cherry picked from commit 3394b69)
kibanamachine
added a commit
that referenced
this pull request
Feb 6, 2025
#210016) # Backport This will backport the following commits from `main` to `8.18`: - [[GenAI Connectors] Add support for telemetry metadata (#208180)](#208180) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Marius Iversen","email":"marius.iversen@elastic.co"},"sourceCommit":{"committedDate":"2025-02-06T14:24:35Z","message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[GenAI Connectors] Add support for telemetry metadata","number":208180,"url":"https://github.com/elastic/kibana/pull/208180","mergeCommit":{"message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208180","number":208180,"mergeCommit":{"message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Marius Iversen <marius.iversen@elastic.co>
drewdaemon
pushed a commit
to drewdaemon/kibana
that referenced
this pull request
Feb 6, 2025
## Summary This extends initial connector telemetry from PR ref elastic#186936. The PR adds the following optional fields when instantiating a new actionClient as part of its `subActionParams`: ```ts { telemetryMetadata : { pluginId: "your plugin name or unique identifier", aggregateBy: "ID to aggregate on" } } ``` The support is added to all AI connector models for both stream/non-stream/raw. The PR also adds token count usage for bedrock `InvokeAIRaw`, as that was currently not added correctly. Pierre also helped with adding a new metadata optional field for the `NL to ESQL functions`, so that users can pass in similar metadata for LLM conversations using the InfereceClient. PluginId is a field used to filter telemetry in the way the team wants to implement it. It could be a team name, a plugin name etc, all depending on how the team wants to group and filter on the telemetry event. AggregateBy is intended to be used to group multiple LLM calls for aggregations and stats, for example a conversationId that has multiple LLM calls. Both fields are optional, so when you do not want to aggregate the option can simply be ignored. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Feb 7, 2025
…#210018) # Backport This will backport the following commits from `main` to `9.0`: - [[GenAI Connectors] Add support for telemetry metadata (#208180)](#208180) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Marius Iversen","email":"marius.iversen@elastic.co"},"sourceCommit":{"committedDate":"2025-02-06T14:24:35Z","message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[GenAI Connectors] Add support for telemetry metadata","number":208180,"url":"https://github.com/elastic/kibana/pull/208180","mergeCommit":{"message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208180","number":208180,"mergeCommit":{"message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Marius Iversen <marius.iversen@elastic.co>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 7, 2025
## Summary This extends initial connector telemetry from PR ref elastic#186936. The PR adds the following optional fields when instantiating a new actionClient as part of its `subActionParams`: ```ts { telemetryMetadata : { pluginId: "your plugin name or unique identifier", aggregateBy: "ID to aggregate on" } } ``` The support is added to all AI connector models for both stream/non-stream/raw. The PR also adds token count usage for bedrock `InvokeAIRaw`, as that was currently not added correctly. Pierre also helped with adding a new metadata optional field for the `NL to ESQL functions`, so that users can pass in similar metadata for LLM conversations using the InfereceClient. PluginId is a field used to filter telemetry in the way the team wants to implement it. It could be a team name, a plugin name etc, all depending on how the team wants to group and filter on the telemetry event. AggregateBy is intended to be used to group multiple LLM calls for aggregations and stats, for example a conversationId that has multiple LLM calls. Both fields are optional, so when you do not want to aggregate the option can simply be ignored. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> (cherry picked from commit 3394b69)
kibanamachine
added a commit
that referenced
this pull request
Feb 7, 2025
…#210157) # Backport This will backport the following commits from `main` to `8.x`: - [[GenAI Connectors] Add support for telemetry metadata (#208180)](#208180) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Marius Iversen","email":"marius.iversen@elastic.co"},"sourceCommit":{"committedDate":"2025-02-06T14:24:35Z","message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[GenAI Connectors] Add support for telemetry metadata","number":208180,"url":"https://github.com/elastic/kibana/pull/208180","mergeCommit":{"message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210018","number":210018,"state":"MERGED","mergeCommit":{"sha":"a4075782f4df67ec25dddec1046de50161dfc89b","message":"[9.0] [GenAI Connectors] Add support for telemetry metadata (#208180) (#210018)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.0`:\n- [[GenAI Connectors] Add support for telemetry metadata\n(#208180)](https://github.com/elastic/kibana/pull/208180)\n\n<!--- Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Marius\nIversen\",\"email\":\"marius.iversen@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-06T14:24:35Z\",\"message\":\"[GenAI\nConnectors] Add support for telemetry metadata (#208180)\\n\\n##\nSummary\\n\\nThis extends initial connector telemetry from PR\nref\\nhttps://github.com//pull/186936.\\n\\nThe PR adds the\nfollowing optional fields when instantiating a new\\nactionClient as part\nof its `subActionParams`:\\n\\n```ts\\n{\\n telemetryMetadata : {\\n\npluginId: \\\"your plugin name or unique identifier\\\",\\n aggregateBy: \\\"ID\nto aggregate on\\\" \\n }\\n}\\n```\\n\\nThe support is added to all AI\nconnector models for both\\nstream/non-stream/raw.\\n\\nThe PR also adds\ntoken count usage for bedrock `InvokeAIRaw`, as that\\nwas currently not\nadded correctly.\\n\\nPierre also helped with adding a new metadata\noptional field for the `NL\\nto ESQL functions`, so that users can pass\nin similar metadata for LLM\\nconversations using the\nInfereceClient.\\n\\nPluginId is a field used to filter telemetry in the\nway the team wants\\nto implement it. It could be a team name, a plugin\nname etc, all\\ndepending on how the team wants to group and filter on\nthe telemetry\\nevent.\\n\\nAggregateBy is intended to be used to group\nmultiple LLM calls for\\naggregations and stats, for example a\nconversationId that has multiple\\nLLM calls.\\n\\nBoth fields are\noptional, so when you do not want to aggregate the\\noption can simply be\nignored.\\n\\n### Checklist\\n\\nCheck the PR satisfies following\nconditions. \\n\\nReviewers should verify this PR satisfies this list as\nwell.\\n\\n- [x] Any text added follows [EUI's\nwriting\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\nsentence case text and includes\n[i18n\\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\\n-\n[x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common scenarios\\n- [x] The PR\ndescription includes the appropriate Release Notes section,\\nand the\ncorrect `release_note:*` label is applied per\nthe\\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\\n\\n---------\\n\\nCo-authored-by:\npgayvallet\n<pierre.gayvallet@elastic.co>\",\"sha\":\"3394b691b1582d504195182013f833ba727c5e7e\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:skip\",\"v9.0.0\",\"Team:\nSecuritySolution\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\"],\"title\":\"[GenAI\nConnectors] Add support for telemetry\nmetadata\",\"number\":208180,\"url\":\"https://github.com/elastic/kibana/pull/208180\",\"mergeCommit\":{\"message\":\"[GenAI\nConnectors] Add support for telemetry metadata (#208180)\\n\\n##\nSummary\\n\\nThis extends initial connector telemetry from PR\nref\\nhttps://github.com//pull/186936.\\n\\nThe PR adds the\nfollowing optional fields when instantiating a new\\nactionClient as part\nof its `subActionParams`:\\n\\n```ts\\n{\\n telemetryMetadata : {\\n\npluginId: \\\"your plugin name or unique identifier\\\",\\n aggregateBy: \\\"ID\nto aggregate on\\\" \\n }\\n}\\n```\\n\\nThe support is added to all AI\nconnector models for both\\nstream/non-stream/raw.\\n\\nThe PR also adds\ntoken count usage for bedrock `InvokeAIRaw`, as that\\nwas currently not\nadded correctly.\\n\\nPierre also helped with adding a new metadata\noptional field for the `NL\\nto ESQL functions`, so that users can pass\nin similar metadata for LLM\\nconversations using the\nInfereceClient.\\n\\nPluginId is a field used to filter telemetry in the\nway the team wants\\nto implement it. It could be a team name, a plugin\nname etc, all\\ndepending on how the team wants to group and filter on\nthe telemetry\\nevent.\\n\\nAggregateBy is intended to be used to group\nmultiple LLM calls for\\naggregations and stats, for example a\nconversationId that has multiple\\nLLM calls.\\n\\nBoth fields are\noptional, so when you do not want to aggregate the\\noption can simply be\nignored.\\n\\n### Checklist\\n\\nCheck the PR satisfies following\nconditions. \\n\\nReviewers should verify this PR satisfies this list as\nwell.\\n\\n- [x] Any text added follows [EUI's\nwriting\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\nsentence case text and includes\n[i18n\\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\\n-\n[x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common scenarios\\n- [x] The PR\ndescription includes the appropriate Release Notes section,\\nand the\ncorrect `release_note:*` label is applied per\nthe\\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\\n\\n---------\\n\\nCo-authored-by:\npgayvallet\n<pierre.gayvallet@elastic.co>\",\"sha\":\"3394b691b1582d504195182013f833ba727c5e7e\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/208180\",\"number\":208180,\"mergeCommit\":{\"message\":\"[GenAI\nConnectors] Add support for telemetry metadata (#208180)\\n\\n##\nSummary\\n\\nThis extends initial connector telemetry from PR\nref\\nhttps://github.com//pull/186936.\\n\\nThe PR adds the\nfollowing optional fields when instantiating a new\\nactionClient as part\nof its `subActionParams`:\\n\\n```ts\\n{\\n telemetryMetadata : {\\n\npluginId: \\\"your plugin name or unique identifier\\\",\\n aggregateBy: \\\"ID\nto aggregate on\\\" \\n }\\n}\\n```\\n\\nThe support is added to all AI\nconnector models for both\\nstream/non-stream/raw.\\n\\nThe PR also adds\ntoken count usage for bedrock `InvokeAIRaw`, as that\\nwas currently not\nadded correctly.\\n\\nPierre also helped with adding a new metadata\noptional field for the `NL\\nto ESQL functions`, so that users can pass\nin similar metadata for LLM\\nconversations using the\nInfereceClient.\\n\\nPluginId is a field used to filter telemetry in the\nway the team wants\\nto implement it. It could be a team name, a plugin\nname etc, all\\ndepending on how the team wants to group and filter on\nthe telemetry\\nevent.\\n\\nAggregateBy is intended to be used to group\nmultiple LLM calls for\\naggregations and stats, for example a\nconversationId that has multiple\\nLLM calls.\\n\\nBoth fields are\noptional, so when you do not want to aggregate the\\noption can simply be\nignored.\\n\\n### Checklist\\n\\nCheck the PR satisfies following\nconditions. \\n\\nReviewers should verify this PR satisfies this list as\nwell.\\n\\n- [x] Any text added follows [EUI's\nwriting\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\nsentence case text and includes\n[i18n\\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\\n-\n[x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common scenarios\\n- [x] The PR\ndescription includes the appropriate Release Notes section,\\nand the\ncorrect `release_note:*` label is applied per\nthe\\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\\n\\n---------\\n\\nCo-authored-by:\npgayvallet\n<pierre.gayvallet@elastic.co>\",\"sha\":\"3394b691b1582d504195182013f833ba727c5e7e\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by: Marius Iversen <marius.iversen@elastic.co>"}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210016","number":210016,"state":"MERGED","mergeCommit":{"sha":"bf9401474bfe63139e25bb4c3546509f926954c7","message":"[8.18] [GenAI Connectors] Add support for telemetry metadata (#208180) (#210016)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.18`:\n- [[GenAI Connectors] Add support for telemetry metadata\n(#208180)](https://github.com/elastic/kibana/pull/208180)\n\n<!--- Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Marius\nIversen\",\"email\":\"marius.iversen@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-06T14:24:35Z\",\"message\":\"[GenAI\nConnectors] Add support for telemetry metadata (#208180)\\n\\n##\nSummary\\n\\nThis extends initial connector telemetry from PR\nref\\nhttps://github.com//pull/186936.\\n\\nThe PR adds the\nfollowing optional fields when instantiating a new\\nactionClient as part\nof its `subActionParams`:\\n\\n```ts\\n{\\n telemetryMetadata : {\\n\npluginId: \\\"your plugin name or unique identifier\\\",\\n aggregateBy: \\\"ID\nto aggregate on\\\" \\n }\\n}\\n```\\n\\nThe support is added to all AI\nconnector models for both\\nstream/non-stream/raw.\\n\\nThe PR also adds\ntoken count usage for bedrock `InvokeAIRaw`, as that\\nwas currently not\nadded correctly.\\n\\nPierre also helped with adding a new metadata\noptional field for the `NL\\nto ESQL functions`, so that users can pass\nin similar metadata for LLM\\nconversations using the\nInfereceClient.\\n\\nPluginId is a field used to filter telemetry in the\nway the team wants\\nto implement it. It could be a team name, a plugin\nname etc, all\\ndepending on how the team wants to group and filter on\nthe telemetry\\nevent.\\n\\nAggregateBy is intended to be used to group\nmultiple LLM calls for\\naggregations and stats, for example a\nconversationId that has multiple\\nLLM calls.\\n\\nBoth fields are\noptional, so when you do not want to aggregate the\\noption can simply be\nignored.\\n\\n### Checklist\\n\\nCheck the PR satisfies following\nconditions. \\n\\nReviewers should verify this PR satisfies this list as\nwell.\\n\\n- [x] Any text added follows [EUI's\nwriting\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\nsentence case text and includes\n[i18n\\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\\n-\n[x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common scenarios\\n- [x] The PR\ndescription includes the appropriate Release Notes section,\\nand the\ncorrect `release_note:*` label is applied per\nthe\\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\\n\\n---------\\n\\nCo-authored-by:\npgayvallet\n<pierre.gayvallet@elastic.co>\",\"sha\":\"3394b691b1582d504195182013f833ba727c5e7e\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:skip\",\"v9.0.0\",\"Team:\nSecuritySolution\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\"],\"title\":\"[GenAI\nConnectors] Add support for telemetry\nmetadata\",\"number\":208180,\"url\":\"https://github.com/elastic/kibana/pull/208180\",\"mergeCommit\":{\"message\":\"[GenAI\nConnectors] Add support for telemetry metadata (#208180)\\n\\n##\nSummary\\n\\nThis extends initial connector telemetry from PR\nref\\nhttps://github.com//pull/186936.\\n\\nThe PR adds the\nfollowing optional fields when instantiating a new\\nactionClient as part\nof its `subActionParams`:\\n\\n```ts\\n{\\n telemetryMetadata : {\\n\npluginId: \\\"your plugin name or unique identifier\\\",\\n aggregateBy: \\\"ID\nto aggregate on\\\" \\n }\\n}\\n```\\n\\nThe support is added to all AI\nconnector models for both\\nstream/non-stream/raw.\\n\\nThe PR also adds\ntoken count usage for bedrock `InvokeAIRaw`, as that\\nwas currently not\nadded correctly.\\n\\nPierre also helped with adding a new metadata\noptional field for the `NL\\nto ESQL functions`, so that users can pass\nin similar metadata for LLM\\nconversations using the\nInfereceClient.\\n\\nPluginId is a field used to filter telemetry in the\nway the team wants\\nto implement it. It could be a team name, a plugin\nname etc, all\\ndepending on how the team wants to group and filter on\nthe telemetry\\nevent.\\n\\nAggregateBy is intended to be used to group\nmultiple LLM calls for\\naggregations and stats, for example a\nconversationId that has multiple\\nLLM calls.\\n\\nBoth fields are\noptional, so when you do not want to aggregate the\\noption can simply be\nignored.\\n\\n### Checklist\\n\\nCheck the PR satisfies following\nconditions. \\n\\nReviewers should verify this PR satisfies this list as\nwell.\\n\\n- [x] Any text added follows [EUI's\nwriting\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\nsentence case text and includes\n[i18n\\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\\n-\n[x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common scenarios\\n- [x] The PR\ndescription includes the appropriate Release Notes section,\\nand the\ncorrect `release_note:*` label is applied per\nthe\\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\\n\\n---------\\n\\nCo-authored-by:\npgayvallet\n<pierre.gayvallet@elastic.co>\",\"sha\":\"3394b691b1582d504195182013f833ba727c5e7e\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/208180\",\"number\":208180,\"mergeCommit\":{\"message\":\"[GenAI\nConnectors] Add support for telemetry metadata (#208180)\\n\\n##\nSummary\\n\\nThis extends initial connector telemetry from PR\nref\\nhttps://github.com//pull/186936.\\n\\nThe PR adds the\nfollowing optional fields when instantiating a new\\nactionClient as part\nof its `subActionParams`:\\n\\n```ts\\n{\\n telemetryMetadata : {\\n\npluginId: \\\"your plugin name or unique identifier\\\",\\n aggregateBy: \\\"ID\nto aggregate on\\\" \\n }\\n}\\n```\\n\\nThe support is added to all AI\nconnector models for both\\nstream/non-stream/raw.\\n\\nThe PR also adds\ntoken count usage for bedrock `InvokeAIRaw`, as that\\nwas currently not\nadded correctly.\\n\\nPierre also helped with adding a new metadata\noptional field for the `NL\\nto ESQL functions`, so that users can pass\nin similar metadata for LLM\\nconversations using the\nInfereceClient.\\n\\nPluginId is a field used to filter telemetry in the\nway the team wants\\nto implement it. It could be a team name, a plugin\nname etc, all\\ndepending on how the team wants to group and filter on\nthe telemetry\\nevent.\\n\\nAggregateBy is intended to be used to group\nmultiple LLM calls for\\naggregations and stats, for example a\nconversationId that has multiple\\nLLM calls.\\n\\nBoth fields are\noptional, so when you do not want to aggregate the\\noption can simply be\nignored.\\n\\n### Checklist\\n\\nCheck the PR satisfies following\nconditions. \\n\\nReviewers should verify this PR satisfies this list as\nwell.\\n\\n- [x] Any text added follows [EUI's\nwriting\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\nsentence case text and includes\n[i18n\\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\\n-\n[x] [Unit or\nfunctional\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\nwere\nupdated or added to match the most common scenarios\\n- [x] The PR\ndescription includes the appropriate Release Notes section,\\nand the\ncorrect `release_note:*` label is applied per\nthe\\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\\n\\n---------\\n\\nCo-authored-by:\npgayvallet\n<pierre.gayvallet@elastic.co>\",\"sha\":\"3394b691b1582d504195182013f833ba727c5e7e\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by: Marius Iversen <marius.iversen@elastic.co>"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208180","number":208180,"mergeCommit":{"message":"[GenAI Connectors] Add support for telemetry metadata (#208180)\n\n## Summary\n\nThis extends initial connector telemetry from PR ref\nhttps://github.com//pull/186936.\n\nThe PR adds the following optional fields when instantiating a new\nactionClient as part of its `subActionParams`:\n\n```ts\n{\n telemetryMetadata : {\n pluginId: \"your plugin name or unique identifier\",\n aggregateBy: \"ID to aggregate on\" \n }\n}\n```\n\nThe support is added to all AI connector models for both\nstream/non-stream/raw.\n\nThe PR also adds token count usage for bedrock `InvokeAIRaw`, as that\nwas currently not added correctly.\n\nPierre also helped with adding a new metadata optional field for the `NL\nto ESQL functions`, so that users can pass in similar metadata for LLM\nconversations using the InfereceClient.\n\nPluginId is a field used to filter telemetry in the way the team wants\nto implement it. It could be a team name, a plugin name etc, all\ndepending on how the team wants to group and filter on the telemetry\nevent.\n\nAggregateBy is intended to be used to group multiple LLM calls for\naggregations and stats, for example a conversationId that has multiple\nLLM calls.\n\nBoth fields are optional, so when you do not want to aggregate the\noption can simply be ignored.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: pgayvallet <pierre.gayvallet@elastic.co>","sha":"3394b691b1582d504195182013f833ba727c5e7e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Marius Iversen <marius.iversen@elastic.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds telemetry to token count for GenAI connectors
This requires event based telemetry, so
core.analyticswas added to theactionExecutorHere are a few of the events from
telemetry-v2-staging: