[APM] Use excluded data tiers setting#192373
Merged
smith merged 9 commits intoelastic:mainfrom Sep 23, 2024
Merged
Conversation
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
Contributor
Author
|
/ci |
crespocarlos
force-pushed
the
190559-apm-use-excluded_data_tiers-setting
branch
5 times, most recently
from
5144b69 to
495b445
Compare
Contributor
Author
|
/ci |
crespocarlos
force-pushed
the
190559-apm-use-excluded_data_tiers-setting
branch
5 times, most recently
from
c4f2334 to
e612deb
Compare
Contributor
Author
|
/ci |
crespocarlos
force-pushed
the
190559-apm-use-excluded_data_tiers-setting
branch
from
91190f2 to
d46ee97
Compare
Contributor
Author
|
/ci |
Update config key
crespocarlos
force-pushed
the
190559-apm-use-excluded_data_tiers-setting
branch
from
d46ee97 to
0860b34
Compare
Contributor
Author
|
/ci |
Contributor
Author
|
/ci |
crespocarlos
added
release_note:skip
miloszmarcinkowski
approved these changes
Sep 20, 2024
dgieselaar
reviewed
Sep 20, 2024
x-pack/plugins/observability_solution/apm_data_access/server/lib/helpers/tier_filter.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/observability_solution/apm_data_access/server/lib/helpers/tier_filter.ts
Show resolved
Hide resolved
x-pack/plugins/observability_solution/observability_shared/common/ilm_types.ts
Show resolved
Hide resolved
...lugins/observability_solution/apm/server/routes/historical_data/has_historical_agent_data.ts
Outdated
Show resolved
Hide resolved
botelastic
bot
added
the
Team:actionable-obs
Contributor
|
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management) |
Contributor
Author
|
@elasticmachine merge upstream |
dgieselaar
approved these changes
Sep 23, 2024
Contributor
Author
|
@elasticmachine merge upstream |
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Module Count
Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
History
To update your PR or re-run it, just comment with: |
bmorelli25
approved these changes
Sep 23, 2024
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
cauemarcondes
pushed a commit
to cauemarcondes/kibana
that referenced
this pull request
Sep 24, 2024
closes [elastic#190559](elastic#190559) ## Summary This PR updates the ES clients in APM to respect the excluded tier configuration. When this config is set, the ES clients will automatically add a filter to exclude the specified tiers from queries. <img width="600" alt="image" src="https://github.com/user-attachments/assets/9b0de76d-242c-4343-bc30-d5c787316f59"> All queries in APM should have the `_tier` filter (via `get_apm_events_client`) <img width="600" alt="image" src="https://github.com/user-attachments/assets/c525602f-f239-4be8-99c4-65d617962656"> This change also affects alerting (via `alerting_es_client`) <img width="600" alt="image" src="https://github.com/user-attachments/assets/750df4d7-5b49-4de5-9294-7afedf11d7e5"> And it impacts the alerts column (via `get_apm_alert_client`) <img width="600" alt="image" src="https://github.com/user-attachments/assets/44bd9129-1e72-4a3a-af32-d42a9cd9164d"> ### What won't automatically add a filter for `_tier` - Embeddables - ML queries ### How to test - Set the config in Advanced Settings to exclude `data_frozen` and `data_cold` (optional) - Navigate to APM and check the query `Inspect` to see if the filter is present. - Click through APM to confirm things still work. - Create one of each type of APM alerts - Without the config set, queries should not include the `_tier` filter` --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit ee5ef81)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
cauemarcondes
pushed a commit
to cauemarcondes/kibana
that referenced
this pull request
Sep 24, 2024
closes [elastic#190559](elastic#190559) ## Summary This PR updates the ES clients in APM to respect the excluded tier configuration. When this config is set, the ES clients will automatically add a filter to exclude the specified tiers from queries. <img width="600" alt="image" src="https://github.com/user-attachments/assets/9b0de76d-242c-4343-bc30-d5c787316f59"> All queries in APM should have the `_tier` filter (via `get_apm_events_client`) <img width="600" alt="image" src="https://github.com/user-attachments/assets/c525602f-f239-4be8-99c4-65d617962656"> This change also affects alerting (via `alerting_es_client`) <img width="600" alt="image" src="https://github.com/user-attachments/assets/750df4d7-5b49-4de5-9294-7afedf11d7e5"> And it impacts the alerts column (via `get_apm_alert_client`) <img width="600" alt="image" src="https://github.com/user-attachments/assets/44bd9129-1e72-4a3a-af32-d42a9cd9164d"> ### What won't automatically add a filter for `_tier` - Embeddables - ML queries ### How to test - Set the config in Advanced Settings to exclude `data_frozen` and `data_cold` (optional) - Navigate to APM and check the query `Inspect` to see if the filter is present. - Click through APM to confirm things still work. - Create one of each type of APM alerts - Without the config set, queries should not include the `_tier` filter` --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit ee5ef81) # Conflicts: # x-pack/packages/observability/observability_utils/es/queries/exclude_frozen_query.ts # x-pack/plugins/cloud_integrations/cloud_experiments/public/launch_darkly_client/index.ts # x-pack/plugins/observability_solution/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.test.ts # x-pack/plugins/observability_solution/apm/server/lib/helpers/get_apm_event_client.ts # x-pack/plugins/observability_solution/apm/server/lib/helpers/tier_filter.ts # x-pack/plugins/observability_solution/apm_data_access/server/lib/helpers/create_es_client/create_apm_event_client/index.ts # x-pack/plugins/observability_solution/apm_data_access/server/lib/helpers/index.ts # x-pack/plugins/observability_solution/apm_data_access/tsconfig.json # x-pack/plugins/observability_solution/observability/server/ui_settings.ts
3 tasks
cauemarcondes
added a commit
that referenced
this pull request
Sep 24, 2024
# Backport This will backport the following commits from `main` to `8.x`: - [[APM] Use excluded data tiers setting (#192373)](#192373) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Carlos Crespo","email":"crespocarlos@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-09-23T15:59:08Z","message":"[APM] Use excluded data tiers setting (#192373)\n\ncloses [#190559](https://github.com/elastic/kibana/issues/190559)\r\n\r\n## Summary\r\n\r\nThis PR updates the ES clients in APM to respect the excluded tier\r\nconfiguration. When this config is set, the ES clients will\r\nautomatically add a filter to exclude the specified tiers from queries.\r\n\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9b0de76d-242c-4343-bc30-d5c787316f59\">\r\n\r\nAll queries in APM should have the `_tier` filter (via\r\n`get_apm_events_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/c525602f-f239-4be8-99c4-65d617962656\">\r\n\r\nThis change also affects alerting (via `alerting_es_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/750df4d7-5b49-4de5-9294-7afedf11d7e5\">\r\n\r\nAnd it impacts the alerts column (via `get_apm_alert_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/44bd9129-1e72-4a3a-af32-d42a9cd9164d\">\r\n\r\n### What won't automatically add a filter for `_tier`\r\n\r\n- Embeddables\r\n- ML queries\r\n\r\n### How to test\r\n- Set the config in Advanced Settings to exclude `data_frozen` and\r\n`data_cold` (optional)\r\n- Navigate to APM and check the query `Inspect` to see if the filter is\r\npresent.\r\n- Click through APM to confirm things still work.\r\n- Create one of each type of APM alerts\r\n- Without the config set, queries should not include the `_tier` filter`\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"ee5ef8166b74041f359862f3f22e0eb491f1443c","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","apm","backport:prev-major","ci:project-deploy-observability","Team:obs-ux-infra_services","Team:obs-ux-management","v8.16.0"],"number":192373,"url":"https://github.com/elastic/kibana/pull/192373","mergeCommit":{"message":"[APM] Use excluded data tiers setting (#192373)\n\ncloses [#190559](https://github.com/elastic/kibana/issues/190559)\r\n\r\n## Summary\r\n\r\nThis PR updates the ES clients in APM to respect the excluded tier\r\nconfiguration. When this config is set, the ES clients will\r\nautomatically add a filter to exclude the specified tiers from queries.\r\n\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9b0de76d-242c-4343-bc30-d5c787316f59\">\r\n\r\nAll queries in APM should have the `_tier` filter (via\r\n`get_apm_events_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/c525602f-f239-4be8-99c4-65d617962656\">\r\n\r\nThis change also affects alerting (via `alerting_es_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/750df4d7-5b49-4de5-9294-7afedf11d7e5\">\r\n\r\nAnd it impacts the alerts column (via `get_apm_alert_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/44bd9129-1e72-4a3a-af32-d42a9cd9164d\">\r\n\r\n### What won't automatically add a filter for `_tier`\r\n\r\n- Embeddables\r\n- ML queries\r\n\r\n### How to test\r\n- Set the config in Advanced Settings to exclude `data_frozen` and\r\n`data_cold` (optional)\r\n- Navigate to APM and check the query `Inspect` to see if the filter is\r\npresent.\r\n- Click through APM to confirm things still work.\r\n- Create one of each type of APM alerts\r\n- Without the config set, queries should not include the `_tier` filter`\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"ee5ef8166b74041f359862f3f22e0eb491f1443c"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192373","number":192373,"mergeCommit":{"message":"[APM] Use excluded data tiers setting (#192373)\n\ncloses [#190559](https://github.com/elastic/kibana/issues/190559)\r\n\r\n## Summary\r\n\r\nThis PR updates the ES clients in APM to respect the excluded tier\r\nconfiguration. When this config is set, the ES clients will\r\nautomatically add a filter to exclude the specified tiers from queries.\r\n\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9b0de76d-242c-4343-bc30-d5c787316f59\">\r\n\r\nAll queries in APM should have the `_tier` filter (via\r\n`get_apm_events_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/c525602f-f239-4be8-99c4-65d617962656\">\r\n\r\nThis change also affects alerting (via `alerting_es_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/750df4d7-5b49-4de5-9294-7afedf11d7e5\">\r\n\r\nAnd it impacts the alerts column (via `get_apm_alert_client`)\r\n<img width=\"600\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/44bd9129-1e72-4a3a-af32-d42a9cd9164d\">\r\n\r\n### What won't automatically add a filter for `_tier`\r\n\r\n- Embeddables\r\n- ML queries\r\n\r\n### How to test\r\n- Set the config in Advanced Settings to exclude `data_frozen` and\r\n`data_cold` (optional)\r\n- Navigate to APM and check the query `Inspect` to see if the filter is\r\npresent.\r\n- Click through APM to confirm things still work.\r\n- Create one of each type of APM alerts\r\n- Without the config set, queries should not include the `_tier` filter`\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"ee5ef8166b74041f359862f3f22e0eb491f1443c"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Carlos Crespo <crespocarlos@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #190559
Summary
This PR updates the ES clients in APM to respect the excluded tier configuration. When this config is set, the ES clients will automatically add a filter to exclude the specified tiers from queries.
All queries in APM should have the
_tierfilter (viaget_apm_events_client)This change also affects alerting (via
alerting_es_client)And it impacts the alerts column (via
get_apm_alert_client)What won't automatically add a filter for
_tierHow to test
data_frozenanddata_cold(optional)Inspectto see if the filter is present._tierfilter`