Skip to content

[Obs AI Assistant] Rename score properties to esScore and llmScore for more clarity#215699

Merged
viduni94 merged 12 commits intoelastic:mainfrom
viduni94:rename-properties-of-kb-entry-scores
Mar 27, 2025
Merged

[Obs AI Assistant] Rename score properties to esScore and llmScore for more clarity#215699
viduni94 merged 12 commits intoelastic:mainfrom
viduni94:rename-properties-of-kb-entry-scores

Conversation

@viduni94
Copy link
Contributor

@viduni94 viduni94 commented Mar 24, 2025 •
edited by kibanamachine
Loading

Closes #214801

Summary

In the recall and score function and score suggestions function we refer to both Elasticsearch score and LLM score as score. This is confusing and difficult to differentiate.

This PR renames the score property based on the context of the score to either esScore or llmScore.

Checklist

  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

@viduni94 viduni94 added release_note:skip Skip the PR/issue when compiling release notes Team:Obs AI Assistant Observability AI Assistant backport:version Backport to applied version labels v9.1.0 v8.19.0 labels Mar 24, 2025
@viduni94 viduni94 self-assigned this Mar 24, 2025
@viduni94 viduni94 requested a review from a team as a code owner March 24, 2025 13:09
@elasticmachine
Copy link
Contributor

elasticmachine commented Mar 24, 2025

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

sorenlouv
sorenlouv reviewed Mar 24, 2025
View reviewed changes
x-pack/platform/plugins/shared/observability_ai_assistant/server/analytics/recall_ranking.ts Outdated
Comment on lines 24 to 27
esScore: {
type: 'float',
_meta: {
description: 'The score produced by ELSER text_expansion',
Copy link
Member

@sorenlouv sorenlouv Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we should update the description as well:

Suggested change
esScore: {
type: 'float',
_meta: {
description: 'The score produced by ELSER text_expansion',
esScore: {
type: 'float',
_meta: {
description: 'The score produced by Elasticsearch',

Btw. I am not sure if you need to handle mapping changes elsewhere too. Try asking @afharo

Copy link
Contributor Author

@viduni94 viduni94 Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Hi @afharo
I am updating a property of a Telemetry server event that we are sending, from elserScore to esScore.
Do I need to handle these mapping changes elsewhere?

Copy link
Member

@afharo afharo Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the late reply. I can see that this is an EBT server-side event.

In that case, if you are relying on the generic index ebt-kibana-server (docs), no mapping update is required (maybe a runtime field update in your dataview).

If your team already has an indexer or a BigQuery pipeline leveraging this event type (observability_ai_assistant_recall_ranking), then you should update it accordingly.

cc @elastic/platform-analytics

Copy link
Contributor Author

@viduni94 viduni94 Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @afharo

@elasticmachine
Copy link
Contributor

elasticmachine commented Mar 25, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #48 / EQL execution logic API @ess @serverless @serverlessQA EQL type rules parses shard failures for EQL event query

Metrics [docs]

✅ unchanged

History

cc @viduni94

@elastic-vault-github-plugin-prod elastic-vault-github-plugin-prod bot requested a review from a team as a code owner March 26, 2025 15:06
@elasticmachine
Copy link
Contributor

elasticmachine commented Mar 26, 2025

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

cc @viduni94

@viduni94 viduni94 merged commit 34cf355 into elastic:main Mar 27, 2025
9 checks passed
@kibanamachine
Copy link
Contributor

kibanamachine commented Mar 27, 2025

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/14097561039

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Mar 27, 2025
@viduni94 @kibanamachine
[Obs AI Assistant] Rename score properties to esScore and llmScore fo…
db90d62 
…r more clarity (elastic#215699)

Closes elastic#214801

## Summary

In the recall and score function and score suggestions function we refer
to both Elasticsearch score and LLM score as `score`. This is confusing
and difficult to differentiate.

This PR renames the `score` property based on the context of the score
to either `esScore` or `llmScore`.

### Checklist

- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 34cf355)
@kibanamachine kibanamachine mentioned this pull request Mar 27, 2025
Merged
@kibanamachine
Copy link
Contributor

kibanamachine commented Mar 27, 2025

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine mentioned this pull request Mar 27, 2025
Merged
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Mar 31, 2025
@kibanamachine
Copy link
Contributor

kibanamachine commented Mar 31, 2025

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

cqliu1 pushed a commit to cqliu1/kibana that referenced this pull request Mar 31, 2025
@viduni94 @kibanamachine @cqliu1
[Obs AI Assistant] Rename score properties to esScore and llmScore fo…
7fd8cac 
…r more clarity (elastic#215699)

Closes elastic#214801

## Summary

In the recall and score function and score suggestions function we refer
to both Elasticsearch score and LLM score as `score`. This is confusing
and difficult to differentiate.

This PR renames the `score` property based on the context of the score
to either `esScore` or `llmScore`.


### Checklist

- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine added a commit that referenced this pull request Mar 31, 2025
@kibanamachine @viduni94
[8.x] [Obs AI Assistant] Rename score properties to esScore and llmSc…
5011aaf 
…ore for more clarity (#215699) (#216119)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Obs AI Assistant] Rename score properties to esScore and llmScore
for more clarity
(#215699)](#215699)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Viduni
Wickramarachchi","email":"viduni.wickramarachchi@elastic.co"},"sourceCommit":{"committedDate":"2025-03-27T02:31:01Z","message":"[Obs
AI Assistant] Rename score properties to esScore and llmScore for more
clarity (#215699)\n\nCloses
https://github.com/elastic/kibana/issues/214801\n\n## Summary\n\nIn the
recall and score function and score suggestions function we refer\nto
both Elasticsearch score and LLM score as `score`. This is
confusing\nand difficult to differentiate.\n\nThis PR renames the
`score` property based on the context of the score\nto either `esScore`
or `llmScore`.\n\n\n### Checklist\n\n- [x] The PR description includes
the appropriate Release Notes section,\nand the correct `release_note:*`
label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"34cf355080b3e4ea346da4c86801f4fb4f6824cd","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Obs
AI Assistant","backport:version","v9.1.0","v8.19.0"],"title":"[Obs AI
Assistant] Rename score properties to esScore and llmScore for more
clarity","number":215699,"url":"https://github.com/elastic/kibana/pull/215699","mergeCommit":{"message":"[Obs
AI Assistant] Rename score properties to esScore and llmScore for more
clarity (#215699)\n\nCloses
https://github.com/elastic/kibana/issues/214801\n\n## Summary\n\nIn the
recall and score function and score suggestions function we refer\nto
both Elasticsearch score and LLM score as `score`. This is
confusing\nand difficult to differentiate.\n\nThis PR renames the
`score` property based on the context of the score\nto either `esScore`
or `llmScore`.\n\n\n### Checklist\n\n- [x] The PR description includes
the appropriate Release Notes section,\nand the correct `release_note:*`
label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"34cf355080b3e4ea346da4c86801f4fb4f6824cd"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/215699","number":215699,"mergeCommit":{"message":"[Obs
AI Assistant] Rename score properties to esScore and llmScore for more
clarity (#215699)\n\nCloses
https://github.com/elastic/kibana/issues/214801\n\n## Summary\n\nIn the
recall and score function and score suggestions function we refer\nto
both Elasticsearch score and LLM score as `score`. This is
confusing\nand difficult to differentiate.\n\nThis PR renames the
`score` property based on the context of the score\nto either `esScore`
or `llmScore`.\n\n\n### Checklist\n\n- [x] The PR description includes
the appropriate Release Notes section,\nand the correct `release_note:*`
label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"34cf355080b3e4ea346da4c86801f4fb4f6824cd"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Viduni Wickramarachchi <viduni.wickramarachchi@elastic.co>
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Mar 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sorenlouv sorenlouv sorenlouv approved these changes

@afharo afharo afharo left review comments

Assignees

@viduni94 viduni94

Labels

backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team:Obs AI Assistant Observability AI Assistant v8.19.0 v9.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Obs AI Assistant] Rename the score params to esScore and llmScore

5 participants

@viduni94 @elasticmachine @kibanamachine @sorenlouv @afharo