Skip to content

[APM][OTel] Change the alerts query to include environment not defined value#219228

Merged
jennypavlova merged 8 commits intoelastic:mainfrom
jennypavlova:217914-make-alerts-compatible-with-undefined-serviceenvironment
Apr 29, 2025
Merged

[APM][OTel] Change the alerts query to include environment not defined value#219228
jennypavlova merged 8 commits intoelastic:mainfrom
jennypavlova:217914-make-alerts-compatible-with-undefined-serviceenvironment

Conversation

@jennypavlova
Copy link
Member

@jennypavlova jennypavlova commented Apr 25, 2025

Closes #217914

Summary

This PR fixes the issue with the alerts filtering when the service environment is not defined.

output

Testing

  • Using synthtrace (any scenario) inject some data: for example:
 node scripts/synthtrace simple_trace --scenarioOpts pipeline=apmToOtel --live --uniqueIds
  • Change the scenario - the same one, so the same services have one synthrace env and one undefined (set the environment to undefined)
  • Run the scenario again in a different terminal (also using live without closing the previous one)
  • Configure some alert rules (that will generate alerts for both), for example, latency > 1ms or error count > 0
  • Check the environment drop-down and the alerts tab (should behave the same as in the description example here - the env filter should be applied to the table results and the alert count badge - similar on the service inventory page - there we should retest with the 790 deployment environment discrepancy #217899 changes when merged )

output1

@jennypavlova jennypavlova self-assigned this Apr 25, 2025
@jennypavlova jennypavlova requested a review from a team April 25, 2025 11:45
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@jennypavlova jennypavlova force-pushed the 217914-make-alerts-compatible-with-undefined-serviceenvironment branch from 6866bf6 to ebacefc Compare April 25, 2025 13:48
@elasticmachine
Copy link
Contributor

elasticmachine commented Apr 28, 2025

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #74 / APM API tests services/annotations.spec.ts trial no archive Annotations with a trial license when mixing stored and derived annotations returns stored annotations for the given environment
  • [job] [logs] FTR Configs #74 / APM API tests services/annotations.spec.ts trial no archive Annotations with a trial license when mixing stored and derived annotations returns stored annotations for the given environment
  • [job] [logs] FTR Configs #1 / serverless search UI - feature flags Serverless Synonyms Overview Synonyms get started Page should be able to create a new test synonyms set

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
apm 2.6MB 2.6MB -13.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
apm 35.6KB 35.6KB -7.0B

History

cc @jennypavlova

@jennypavlova jennypavlova merged commit bb025a8 into elastic:main Apr 29, 2025
10 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19, 9.0

https://github.com/elastic/kibana/actions/runs/14728845025

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.19 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.19:
- 790 deployment environment discrepancy (#217899)
9.0 Backport failed because of merge conflicts

You might need to backport the following PRs to 9.0:
- 790 deployment environment discrepancy (#217899)

Manual backport

To create the backport manually run:

node scripts/backport --pr 219228

Questions ?

Please refer to the Backport tool documentation

jennypavlova added a commit that referenced this pull request Apr 29, 2025
## Summary

Follow up: #219228

To ensure the filtering is correct, I am adding this change as the
previous alerts change might break other functionality where the not
defined is missing
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Apr 30, 2025
…d value (elastic#219228)

Closes elastic#217914 

## Summary

This PR fixes the issue with the alerts filtering when the service
environment is not defined.


![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)

## Testing
- Using synthtrace (any scenario) inject some data: for example: 
```
 node scripts/synthtrace simple_trace --scenarioOpts pipeline=apmToOtel --live --uniqueIds
```
- Change the scenario - the same one, so the same services have one
synthrace env and one `undefined` (set the environment to `undefined`)
- Run the scenario again in a different terminal (also using `live`
without closing the previous one)
- Configure some alert rules (that will generate alerts for both), for
example, latency > 1ms or error count > 0
- Check the environment drop-down and the alerts tab (should behave the
same as in the description example here - the env filter should be
applied to the table results and the alert count badge - similar on the
service inventory page - there we should retest with the
elastic#217899 changes when merged )
 

![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Apr 30, 2025
## Summary

Follow up: elastic#219228

To ensure the filtering is correct, I am adding this change as the
previous alerts change might break other functionality where the not
defined is missing
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 1, 2025
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 219228 locally
cc: @jennypavlova

@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 219228 locally
cc: @jennypavlova

crespocarlos pushed a commit to crespocarlos/kibana that referenced this pull request May 2, 2025
…d value (elastic#219228)

Closes elastic#217914

## Summary

This PR fixes the issue with the alerts filtering when the service
environment is not defined.

![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)

## Testing
- Using synthtrace (any scenario) inject some data: for example:
```
 node scripts/synthtrace simple_trace --scenarioOpts pipeline=apmToOtel --live --uniqueIds
```
- Change the scenario - the same one, so the same services have one
synthrace env and one `undefined` (set the environment to `undefined`)
- Run the scenario again in a different terminal (also using `live`
without closing the previous one)
- Configure some alert rules (that will generate alerts for both), for
example, latency > 1ms or error count > 0
- Check the environment drop-down and the alerts tab (should behave the
same as in the description example here - the env filter should be
applied to the table results and the alert count badge - similar on the
service inventory page - there we should retest with the
elastic#217899 changes when merged )

![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)

(cherry picked from commit bb025a8)

# Conflicts:
#	x-pack/solutions/observability/plugins/apm/common/environment_filter_values.ts
crespocarlos pushed a commit to crespocarlos/kibana that referenced this pull request May 2, 2025
…d value (elastic#219228)

Closes elastic#217914

## Summary

This PR fixes the issue with the alerts filtering when the service
environment is not defined.

![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)

## Testing
- Using synthtrace (any scenario) inject some data: for example:
```
 node scripts/synthtrace simple_trace --scenarioOpts pipeline=apmToOtel --live --uniqueIds
```
- Change the scenario - the same one, so the same services have one
synthrace env and one `undefined` (set the environment to `undefined`)
- Run the scenario again in a different terminal (also using `live`
without closing the previous one)
- Configure some alert rules (that will generate alerts for both), for
example, latency > 1ms or error count > 0
- Check the environment drop-down and the alerts tab (should behave the
same as in the description example here - the env filter should be
applied to the table results and the alert count badge - similar on the
service inventory page - there we should retest with the
elastic#217899 changes when merged )

![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)

(cherry picked from commit bb025a8)

# Conflicts:
#	x-pack/solutions/observability/plugins/apm/common/environment_filter_values.ts
@crespocarlos
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
9.0
8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

crespocarlos pushed a commit to crespocarlos/kibana that referenced this pull request May 2, 2025
## Summary

Follow up: elastic#219228

To ensure the filtering is correct, I am adding this change as the
previous alerts change might break other functionality where the not
defined is missing

(cherry picked from commit 0a60ccc)

# Conflicts:
#	x-pack/solutions/observability/plugins/apm/common/utils/environment_query.test.ts
#	x-pack/solutions/observability/plugins/apm/common/utils/environment_query.ts
crespocarlos pushed a commit to crespocarlos/kibana that referenced this pull request May 2, 2025
## Summary

Follow up: elastic#219228

To ensure the filtering is correct, I am adding this change as the
previous alerts change might break other functionality where the not
defined is missing

(cherry picked from commit 0a60ccc)

# Conflicts:
#	x-pack/solutions/observability/plugins/apm/common/utils/environment_query.test.ts
#	x-pack/solutions/observability/plugins/apm/common/utils/environment_query.ts
crespocarlos added a commit that referenced this pull request May 5, 2025
… defined value (#219228) (#219955)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[APM][OTel] Change the alerts query to include environment not
defined value (#219228)](#219228)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"jennypavlova","email":"dzheni.pavlova@elastic.co"},"sourceCommit":{"committedDate":"2025-04-29T10:16:58Z","message":"[APM][OTel]
Change the alerts query to include environment not defined value
(#219228)\n\nCloses #217914 \n\n## Summary\n\nThis PR fixes the issue
with the alerts filtering when the service\nenvironment is not
defined.\n\n\n![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)\n\n##
Testing\n- Using synthtrace (any scenario) inject some data: for
example: \n```\n node scripts/synthtrace simple_trace --scenarioOpts
pipeline=apmToOtel --live --uniqueIds\n```\n- Change the scenario - the
same one, so the same services have one\nsynthrace env and one
`undefined` (set the environment to `undefined`)\n- Run the scenario
again in a different terminal (also using `live`\nwithout closing the
previous one)\n- Configure some alert rules (that will generate alerts
for both), for\nexample, latency > 1ms or error count > 0\n- Check the
environment drop-down and the alerts tab (should behave the\nsame as in
the description example here - the env filter should be\napplied to the
table results and the alert count badge - similar on the\nservice
inventory page - there we should retest with
the\nhttps://github.com//pull/217899 changes when merged
)\n
\n\n![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)","sha":"bb025a82c94ef46105e914bf33a79cc7e13c3acc","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport
missing","v9.0.0","Team:obs-ux-infra_services","backport:version","v9.1.0","v8.19.0","v9.0.1"],"title":"[APM][OTel]
Change the alerts query to include environment not defined
value","number":219228,"url":"https://github.com/elastic/kibana/pull/219228","mergeCommit":{"message":"[APM][OTel]
Change the alerts query to include environment not defined value
(#219228)\n\nCloses #217914 \n\n## Summary\n\nThis PR fixes the issue
with the alerts filtering when the service\nenvironment is not
defined.\n\n\n![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)\n\n##
Testing\n- Using synthtrace (any scenario) inject some data: for
example: \n```\n node scripts/synthtrace simple_trace --scenarioOpts
pipeline=apmToOtel --live --uniqueIds\n```\n- Change the scenario - the
same one, so the same services have one\nsynthrace env and one
`undefined` (set the environment to `undefined`)\n- Run the scenario
again in a different terminal (also using `live`\nwithout closing the
previous one)\n- Configure some alert rules (that will generate alerts
for both), for\nexample, latency > 1ms or error count > 0\n- Check the
environment drop-down and the alerts tab (should behave the\nsame as in
the description example here - the env filter should be\napplied to the
table results and the alert count badge - similar on the\nservice
inventory page - there we should retest with
the\nhttps://github.com//pull/217899 changes when merged
)\n
\n\n![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)","sha":"bb025a82c94ef46105e914bf33a79cc7e13c3acc"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/219228","number":219228,"mergeCommit":{"message":"[APM][OTel]
Change the alerts query to include environment not defined value
(#219228)\n\nCloses #217914 \n\n## Summary\n\nThis PR fixes the issue
with the alerts filtering when the service\nenvironment is not
defined.\n\n\n![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)\n\n##
Testing\n- Using synthtrace (any scenario) inject some data: for
example: \n```\n node scripts/synthtrace simple_trace --scenarioOpts
pipeline=apmToOtel --live --uniqueIds\n```\n- Change the scenario - the
same one, so the same services have one\nsynthrace env and one
`undefined` (set the environment to `undefined`)\n- Run the scenario
again in a different terminal (also using `live`\nwithout closing the
previous one)\n- Configure some alert rules (that will generate alerts
for both), for\nexample, latency > 1ms or error count > 0\n- Check the
environment drop-down and the alerts tab (should behave the\nsame as in
the description example here - the env filter should be\napplied to the
table results and the alert count badge - similar on the\nservice
inventory page - there we should retest with
the\nhttps://github.com//pull/217899 changes when merged
)\n
\n\n![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)","sha":"bb025a82c94ef46105e914bf33a79cc7e13c3acc"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: jennypavlova <dzheni.pavlova@elastic.co>
crespocarlos added a commit that referenced this pull request May 5, 2025
…defined value (#219228) (#219953)

# Backport

This will backport the following commits from `main` to `9.0`:
- [[APM][OTel] Change the alerts query to include environment not
defined value (#219228)](#219228)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"jennypavlova","email":"dzheni.pavlova@elastic.co"},"sourceCommit":{"committedDate":"2025-04-29T10:16:58Z","message":"[APM][OTel]
Change the alerts query to include environment not defined value
(#219228)\n\nCloses #217914 \n\n## Summary\n\nThis PR fixes the issue
with the alerts filtering when the service\nenvironment is not
defined.\n\n\n![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)\n\n##
Testing\n- Using synthtrace (any scenario) inject some data: for
example: \n```\n node scripts/synthtrace simple_trace --scenarioOpts
pipeline=apmToOtel --live --uniqueIds\n```\n- Change the scenario - the
same one, so the same services have one\nsynthrace env and one
`undefined` (set the environment to `undefined`)\n- Run the scenario
again in a different terminal (also using `live`\nwithout closing the
previous one)\n- Configure some alert rules (that will generate alerts
for both), for\nexample, latency > 1ms or error count > 0\n- Check the
environment drop-down and the alerts tab (should behave the\nsame as in
the description example here - the env filter should be\napplied to the
table results and the alert count badge - similar on the\nservice
inventory page - there we should retest with
the\nhttps://github.com//pull/217899 changes when merged
)\n
\n\n![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)","sha":"bb025a82c94ef46105e914bf33a79cc7e13c3acc","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport
missing","v9.0.0","Team:obs-ux-infra_services","backport:version","v9.1.0","v8.19.0","v9.0.1"],"title":"[APM][OTel]
Change the alerts query to include environment not defined
value","number":219228,"url":"https://github.com/elastic/kibana/pull/219228","mergeCommit":{"message":"[APM][OTel]
Change the alerts query to include environment not defined value
(#219228)\n\nCloses #217914 \n\n## Summary\n\nThis PR fixes the issue
with the alerts filtering when the service\nenvironment is not
defined.\n\n\n![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)\n\n##
Testing\n- Using synthtrace (any scenario) inject some data: for
example: \n```\n node scripts/synthtrace simple_trace --scenarioOpts
pipeline=apmToOtel --live --uniqueIds\n```\n- Change the scenario - the
same one, so the same services have one\nsynthrace env and one
`undefined` (set the environment to `undefined`)\n- Run the scenario
again in a different terminal (also using `live`\nwithout closing the
previous one)\n- Configure some alert rules (that will generate alerts
for both), for\nexample, latency > 1ms or error count > 0\n- Check the
environment drop-down and the alerts tab (should behave the\nsame as in
the description example here - the env filter should be\napplied to the
table results and the alert count badge - similar on the\nservice
inventory page - there we should retest with
the\nhttps://github.com//pull/217899 changes when merged
)\n
\n\n![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)","sha":"bb025a82c94ef46105e914bf33a79cc7e13c3acc"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/219228","number":219228,"mergeCommit":{"message":"[APM][OTel]
Change the alerts query to include environment not defined value
(#219228)\n\nCloses #217914 \n\n## Summary\n\nThis PR fixes the issue
with the alerts filtering when the service\nenvironment is not
defined.\n\n\n![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)\n\n##
Testing\n- Using synthtrace (any scenario) inject some data: for
example: \n```\n node scripts/synthtrace simple_trace --scenarioOpts
pipeline=apmToOtel --live --uniqueIds\n```\n- Change the scenario - the
same one, so the same services have one\nsynthrace env and one
`undefined` (set the environment to `undefined`)\n- Run the scenario
again in a different terminal (also using `live`\nwithout closing the
previous one)\n- Configure some alert rules (that will generate alerts
for both), for\nexample, latency > 1ms or error count > 0\n- Check the
environment drop-down and the alerts tab (should behave the\nsame as in
the description example here - the env filter should be\napplied to the
table results and the alert count badge - similar on the\nservice
inventory page - there we should retest with
the\nhttps://github.com//pull/217899 changes when merged
)\n
\n\n![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)","sha":"bb025a82c94ef46105e914bf33a79cc7e13c3acc"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: jennypavlova <dzheni.pavlova@elastic.co>
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 5, 2025
@mistic
Copy link
Contributor

mistic commented May 6, 2025

This PR didn't make it into the latest 9.0.1 BC. Updating the labels.

akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request May 29, 2025
…d value (elastic#219228)

Closes elastic#217914 

## Summary

This PR fixes the issue with the alerts filtering when the service
environment is not defined.


![output](https://github.com/user-attachments/assets/05ef791a-1de4-4d06-bb2d-3b99b5f8afc4)

## Testing
- Using synthtrace (any scenario) inject some data: for example: 
```
 node scripts/synthtrace simple_trace --scenarioOpts pipeline=apmToOtel --live --uniqueIds
```
- Change the scenario - the same one, so the same services have one
synthrace env and one `undefined` (set the environment to `undefined`)
- Run the scenario again in a different terminal (also using `live`
without closing the previous one)
- Configure some alert rules (that will generate alerts for both), for
example, latency > 1ms or error count > 0
- Check the environment drop-down and the alerts tab (should behave the
same as in the description example here - the env filter should be
applied to the table results and the alert count badge - similar on the
service inventory page - there we should retest with the
elastic#217899 changes when merged )
 

![output1](https://github.com/user-attachments/assets/f77e6da1-8e60-4c8e-a1ee-c0e0e4a632e7)
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request May 29, 2025
## Summary

Follow up: elastic#219228

To ensure the filtering is correct, I am adding this change as the
previous alerts change might break other functionality where the not
defined is missing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels release_note:fix Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. v8.19.0 v9.0.2 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make alerts compatible with undefined service.environment

6 participants