Skip to content

Comments

[Dataset Quality] Skip tests which require read_failure_store on ES 9.0#225283

Merged
awahab07 merged 2 commits intomainfrom
225201_Fix-DS-API-test-failure-read_failure_store-provilege
Jun 26, 2025
Merged

[Dataset Quality] Skip tests which require read_failure_store on ES 9.0#225283
awahab07 merged 2 commits intomainfrom
225201_Fix-DS-API-test-failure-read_failure_store-provilege

Conversation

@awahab07
Copy link
Contributor

@awahab07 awahab07 commented Jun 25, 2025

Fixes #225201

Summary

The PR skips the Dataset Quality API integration test suite which depends on reading Failure Store index privileges. Because when this test suite runs in Kibana 8.19 against ES 9.0 in a forward compatibility test runs, it raises the error:

[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.223+00:00][WARN ][http.server.kbn-internal-api-restricted] Access to uri [/internal/dataset_quality/data_streams/logs-synth.fs-default/details] with method [get] is deprecated {"service":{"node":{"roles":["background_tasks","ui"]}}}
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.252+00:00][ERROR][plugins.datasetQuality] illegal_argument_exception
[00:04:43]           │ proc [kibana] 	Root causes:
[00:04:43]           │ proc [kibana] 		illegal_argument_exception: unknown index privilege [read_failure_store]. a privilege must be either one of the predefined fixed indices privileges [all,auto_configure,create,create_doc,create_index,cross_cluster_replication,cross_cluster_replication_internal,delete,delete_index,index,maintenance,manage,manage_data_stream_lifecycle,manage_follow_index,manage_ilm,manage_leader_index,monitor,none,read,read_cross_cluster,view_index_metadata,write] or a pattern over one of the available index actions {"service":{"node":{"roles":["background_tasks","ui"]}}}
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.291+00:00][WARN ][http.server.kbn-internal-api-restricted] Access to uri [/internal/dataset_quality/data_streams/logs-synth.no-default/details] with method [get] is deprecated {"service":{"node":{"roles":["background_tasks","ui"]}}}
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.305+00:00][ERROR][plugins.datasetQuality] illegal_argument_exception

This happens because the test accesses endpoint data_stream/{index}/details which inquire user's index privileges including read_failure_store, which is not available in ES 9.0 as ES 9.0 does not support Failure Store.

When tests run in Kibana 8.19 against ES 9.0 in a forward compatibility test runs, they raise the error:

```bash
illegal_argument_exception: unknown index privilege [read_failure_store]. a privilege must be either one of the predefined fixed indices privileges ...
```

This happens because the tests access endpoint which inquire user's index privileges including `read_failure_store`, which is not available in **ES 9.0** as `ES 9.0` does not support Failure Store.
@awahab07 awahab07 requested a review from a team as a code owner June 25, 2025 12:55
@awahab07 awahab07 added bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes Team:obs-onboarding Observability Onboarding Team backport:version Backport to applied version labels v9.1.0 v8.19.0 v9.0.3 labels Jun 25, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #96 / Journey[infra_hosts_view] "before all" hook for "Navigate to Hosts view and load 500 hosts"

Metrics [docs]

✅ unchanged

@awahab07 awahab07 enabled auto-merge (squash) June 26, 2025 08:31
@awahab07 awahab07 merged commit 5cc1fb3 into main Jun 26, 2025
10 checks passed
@awahab07 awahab07 deleted the 225201_Fix-DS-API-test-failure-read_failure_store-provilege branch June 26, 2025 08:33
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19, 9.0

https://github.com/elastic/kibana/actions/runs/15896974854

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jun 26, 2025
… 9.0 (elastic#225283)

Skips the Dataset Quality API integration test suite which
depends on reading Failure Store index privileges. Because when this
test suite runs in Kibana 8.19 against ES 9.0 in a forward compatibility
test runs, it raises the error:

```bash
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.223+00:00][WARN ][http.server.kbn-internal-api-restricted] Access to uri [/internal/dataset_quality/data_streams/logs-synth.fs-default/details] with method [get] is deprecated {"service":{"node":{"roles":["background_tasks","ui"]}}}
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.252+00:00][ERROR][plugins.datasetQuality] illegal_argument_exception
[00:04:43]           │ proc [kibana] 	Root causes:
[00:04:43]           │ proc [kibana] 		illegal_argument_exception: unknown index privilege [read_failure_store]. a privilege must be either one of the predefined fixed indices privileges [all,auto_configure,create,create_doc,create_index,cross_cluster_replication,cross_cluster_replication_internal,delete,delete_index,index,maintenance,manage,manage_data_stream_lifecycle,manage_follow_index,manage_ilm,manage_leader_index,monitor,none,read,read_cross_cluster,view_index_metadata,write] or a pattern over one of the available index actions {"service":{"node":{"roles":["background_tasks","ui"]}}}
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.291+00:00][WARN ][http.server.kbn-internal-api-restricted] Access to uri [/internal/dataset_quality/data_streams/logs-synth.no-default/details] with method [get] is deprecated {"service":{"node":{"roles":["background_tasks","ui"]}}}
[00:04:43]           │ proc [kibana] [2025-06-25T05:40:33.305+00:00][ERROR][plugins.datasetQuality] illegal_argument_exception
```

This happens because the test accesses endpoint
`data_stream/{index}/details` which inquire user's index privileges
including `read_failure_store`, which is not available in **ES 9.0** as
`ES 9.0` does not support Failure Store.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 5cc1fb3)
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
8.19
9.0 Backport failed because of merge conflicts

You might need to backport the following PRs to 9.0:
- Remove playwright-chromium package from dependencies (#225221)
- Adds missing model Claude 3.7 to accepted models (#224943)

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 225283

Questions ?

Please refer to the Backport tool documentation

@awahab07
Copy link
Contributor Author

awahab07 commented Jun 26, 2025

Does not need to be backported to 9.0 as x-pack/test/api_integration/deployment_agnostic/apis/observability/dataset_quality/data_streams_failure_store.ts has been removed from 9.0.

@awahab07 awahab07 removed the v9.0.3 label Jun 26, 2025
kibanamachine added a commit that referenced this pull request Jun 26, 2025
…` on ES 9.0 (#225283) (#225415)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Dataset Quality] Skip tests which require `read_failure_store` on ES
9.0 (#225283)](#225283)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Abdul Wahab
Zahid","email":"awahab07@yahoo.com"},"sourceCommit":{"committedDate":"2025-06-26T08:33:01Z","message":"[Dataset
Quality] Skip tests which require `read_failure_store` on ES 9.0
(#225283)\n\nSkips the Dataset Quality API integration test suite
which\ndepends on reading Failure Store index privileges. Because when
this\ntest suite runs in Kibana 8.19 against ES 9.0 in a forward
compatibility\ntest runs, it raises the error:\n\n```bash\n[00:04:43]
│ proc [kibana] [2025-06-25T05:40:33.223+00:00][WARN
][http.server.kbn-internal-api-restricted] Access to uri
[/internal/dataset_quality/data_streams/logs-synth.fs-default/details]
with method [get] is deprecated
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana]
[2025-06-25T05:40:33.252+00:00][ERROR][plugins.datasetQuality]
illegal_argument_exception\n[00:04:43] │ proc [kibana] \tRoot
causes:\n[00:04:43] │ proc [kibana] \t\tillegal_argument_exception:
unknown index privilege [read_failure_store]. a privilege must be either
one of the predefined fixed indices privileges
[all,auto_configure,create,create_doc,create_index,cross_cluster_replication,cross_cluster_replication_internal,delete,delete_index,index,maintenance,manage,manage_data_stream_lifecycle,manage_follow_index,manage_ilm,manage_leader_index,monitor,none,read,read_cross_cluster,view_index_metadata,write]
or a pattern over one of the available index actions
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana] [2025-06-25T05:40:33.291+00:00][WARN
][http.server.kbn-internal-api-restricted] Access to uri
[/internal/dataset_quality/data_streams/logs-synth.no-default/details]
with method [get] is deprecated
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana]
[2025-06-25T05:40:33.305+00:00][ERROR][plugins.datasetQuality]
illegal_argument_exception\n```\n\nThis happens because the test
accesses endpoint\n`data_stream/{index}/details` which inquire user's
index privileges\nincluding `read_failure_store`, which is not available
in **ES 9.0** as\n`ES 9.0` does not support Failure
Store.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"5cc1fb33aa669cbb78fc52f2072541344ccad2eb","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","Team:obs-ux-logs","backport:version","v9.1.0","v8.19.0","v9.0.3"],"title":"[Dataset
Quality] Skip tests which require `read_failure_store` on ES
9.0","number":225283,"url":"https://github.com/elastic/kibana/pull/225283","mergeCommit":{"message":"[Dataset
Quality] Skip tests which require `read_failure_store` on ES 9.0
(#225283)\n\nSkips the Dataset Quality API integration test suite
which\ndepends on reading Failure Store index privileges. Because when
this\ntest suite runs in Kibana 8.19 against ES 9.0 in a forward
compatibility\ntest runs, it raises the error:\n\n```bash\n[00:04:43]
│ proc [kibana] [2025-06-25T05:40:33.223+00:00][WARN
][http.server.kbn-internal-api-restricted] Access to uri
[/internal/dataset_quality/data_streams/logs-synth.fs-default/details]
with method [get] is deprecated
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana]
[2025-06-25T05:40:33.252+00:00][ERROR][plugins.datasetQuality]
illegal_argument_exception\n[00:04:43] │ proc [kibana] \tRoot
causes:\n[00:04:43] │ proc [kibana] \t\tillegal_argument_exception:
unknown index privilege [read_failure_store]. a privilege must be either
one of the predefined fixed indices privileges
[all,auto_configure,create,create_doc,create_index,cross_cluster_replication,cross_cluster_replication_internal,delete,delete_index,index,maintenance,manage,manage_data_stream_lifecycle,manage_follow_index,manage_ilm,manage_leader_index,monitor,none,read,read_cross_cluster,view_index_metadata,write]
or a pattern over one of the available index actions
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana] [2025-06-25T05:40:33.291+00:00][WARN
][http.server.kbn-internal-api-restricted] Access to uri
[/internal/dataset_quality/data_streams/logs-synth.no-default/details]
with method [get] is deprecated
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana]
[2025-06-25T05:40:33.305+00:00][ERROR][plugins.datasetQuality]
illegal_argument_exception\n```\n\nThis happens because the test
accesses endpoint\n`data_stream/{index}/details` which inquire user's
index privileges\nincluding `read_failure_store`, which is not available
in **ES 9.0** as\n`ES 9.0` does not support Failure
Store.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"5cc1fb33aa669cbb78fc52f2072541344ccad2eb"}},"sourceBranch":"main","suggestedTargetBranches":["8.19","9.0"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/225283","number":225283,"mergeCommit":{"message":"[Dataset
Quality] Skip tests which require `read_failure_store` on ES 9.0
(#225283)\n\nSkips the Dataset Quality API integration test suite
which\ndepends on reading Failure Store index privileges. Because when
this\ntest suite runs in Kibana 8.19 against ES 9.0 in a forward
compatibility\ntest runs, it raises the error:\n\n```bash\n[00:04:43]
│ proc [kibana] [2025-06-25T05:40:33.223+00:00][WARN
][http.server.kbn-internal-api-restricted] Access to uri
[/internal/dataset_quality/data_streams/logs-synth.fs-default/details]
with method [get] is deprecated
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana]
[2025-06-25T05:40:33.252+00:00][ERROR][plugins.datasetQuality]
illegal_argument_exception\n[00:04:43] │ proc [kibana] \tRoot
causes:\n[00:04:43] │ proc [kibana] \t\tillegal_argument_exception:
unknown index privilege [read_failure_store]. a privilege must be either
one of the predefined fixed indices privileges
[all,auto_configure,create,create_doc,create_index,cross_cluster_replication,cross_cluster_replication_internal,delete,delete_index,index,maintenance,manage,manage_data_stream_lifecycle,manage_follow_index,manage_ilm,manage_leader_index,monitor,none,read,read_cross_cluster,view_index_metadata,write]
or a pattern over one of the available index actions
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana] [2025-06-25T05:40:33.291+00:00][WARN
][http.server.kbn-internal-api-restricted] Access to uri
[/internal/dataset_quality/data_streams/logs-synth.no-default/details]
with method [get] is deprecated
{\"service\":{\"node\":{\"roles\":[\"background_tasks\",\"ui\"]}}}\n[00:04:43]
│ proc [kibana]
[2025-06-25T05:40:33.305+00:00][ERROR][plugins.datasetQuality]
illegal_argument_exception\n```\n\nThis happens because the test
accesses endpoint\n`data_stream/{index}/details` which inquire user's
index privileges\nincluding `read_failure_store`, which is not available
in **ES 9.0** as\n`ES 9.0` does not support Failure
Store.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"5cc1fb33aa669cbb78fc52f2072541344ccad2eb"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Abdul Wahab Zahid <awahab07@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes Team:obs-onboarding Observability Onboarding Team v8.19.0 v9.1.0

Projects

None yet

4 participants