Skip to content

[Dataset quality] Integrations deployment-agnostic api tests#193144

Merged
yngrdyn merged 6 commits intoelastic:mainfrom
yngrdyn:191980-dataset-quality-use-deployment-agnostic-testing-in-api-tests
Sep 17, 2024
Merged

[Dataset quality] Integrations deployment-agnostic api tests#193144
yngrdyn merged 6 commits intoelastic:mainfrom
yngrdyn:191980-dataset-quality-use-deployment-agnostic-testing-in-api-tests

Conversation

@yngrdyn
Copy link
Contributor

@yngrdyn yngrdyn commented Sep 17, 2024 •
edited by kibanamachine
Loading

Relates to #191980.

This PR aims to introduce deployment-agnostic configuration for dataset quality tests, specifically targeting GET /internal/dataset_quality/integrations.

@yngrdyn yngrdyn requested review from a team as code owners September 17, 2024 10:54
@botelastic botelastic bot added the ci:project-deploy-observability Create an Observability project label Sep 17, 2024
@yngrdyn yngrdyn added the release_note:skip Skip the PR/issue when compiling release notes label Sep 17, 2024
@obltmachine
Copy link

obltmachine commented Sep 17, 2024

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

dmlemeshko
dmlemeshko reviewed Sep 17, 2024
View reviewed changes
dmlemeshko
dmlemeshko reviewed Sep 17, 2024
View reviewed changes
dmlemeshko
dmlemeshko approved these changes Sep 17, 2024
View reviewed changes
Copy link
Contributor

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM, thanks a lot for taking your time and adding deployment-agnostic tests 🚀

@yngrdyn yngrdyn force-pushed the 191980-dataset-quality-use-deployment-agnostic-testing-in-api-tests branch from 04cc7ee to f39dccd Compare September 17, 2024 13:09
@yngrdyn yngrdyn force-pushed the 191980-dataset-quality-use-deployment-agnostic-testing-in-api-tests branch from f39dccd to 4d7cae0 Compare September 17, 2024 14:33
@kibana-ci
Copy link

kibana-ci commented Sep 17, 2024 •
edited
Loading

💛 Build succeeded, but was flaky

  • Buildkite Build
  • Commit: 4d7cae0
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-193144-4d7cae066c0b

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

achyutjhunjhunwala
achyutjhunjhunwala reviewed Sep 17, 2024
View reviewed changes
x-pack/test/api_integration/deployment_agnostic/services/package_api.ts
type: 'logs' | 'metrics' | 'synthetics' | 'traces';
}

export function PackageApiProvider({ getService }: DeploymentAgnosticFtrProviderContext) {
Copy link
Contributor

@achyutjhunjhunwala achyutjhunjhunwala Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for creating this.

achyutjhunjhunwala
achyutjhunjhunwala approved these changes Sep 17, 2024
View reviewed changes
Copy link
Contributor

@achyutjhunjhunwala achyutjhunjhunwala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kudos on doing this. great to see the 1st one coming.

@yngrdyn yngrdyn merged commit 45c71c5 into elastic:main Sep 17, 2024
@yngrdyn yngrdyn changed the title [Dataset quality] Integrations agnostic-deployment api tests [Dataset quality] Integrations deployment-agnostic api tests Sep 17, 2024
@yngrdyn yngrdyn mentioned this pull request Sep 17, 2024
Closed
@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 17, 2024

💔 All backports failed

Status Branch Result
❌ 8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [ftr] enable mock-idp-plugin for stateful (deployment-agnostic) tests (#192279)

Manual backport

To create the backport manually run:

node scripts/backport --pr 193144

Questions ?

Please refer to the Backport tool documentation

@yngrdyn yngrdyn mentioned this pull request Sep 25, 2024
Merged
@yngrdyn
Copy link
Contributor Author

yngrdyn commented Sep 25, 2024

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

yngrdyn added a commit to yngrdyn/kibana that referenced this pull request Sep 25, 2024
@yngrdyn
[Dataset quality] Integrations agnostic-deployment api tests (elastic…
8c58173 
…#193144)

Relates to elastic#191980.

This PR aims to introduce deployment-agnostic configuration for dataset
quality tests, specifically targeting `GET
/internal/dataset_quality/integrations`.

(cherry picked from commit 45c71c5)
yngrdyn added a commit that referenced this pull request Sep 25, 2024
@yngrdyn
[8.x] [Dataset quality] Integrations agnostic-deployment api tests (#…
f1a6bd5 
…193144) (#193939)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Dataset quality] Integrations agnostic-deployment api tests
(#193144)](#193144)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Yngrid
Coello","email":"yngrid.coello@elastic.co"},"sourceCommit":{"committedDate":"2024-09-17T17:52:17Z","message":"[Dataset
quality] Integrations agnostic-deployment api tests (#193144)\n\nRelates
to https://github.com/elastic/kibana/issues/191980.\r\n\r\nThis PR aims
to introduce deployment-agnostic configuration for dataset\r\nquality
tests, specifically targeting
`GET\r\n/internal/dataset_quality/integrations`.","sha":"45c71c5a6b8f53eeb2c82c4319dfac0205668a8c","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","ci:project-deploy-observability","v8.16.0"],"number":193144,"url":"https://github.com/elastic/kibana/pull/193144","mergeCommit":{"message":"[Dataset
quality] Integrations agnostic-deployment api tests (#193144)\n\nRelates
to https://github.com/elastic/kibana/issues/191980.\r\n\r\nThis PR aims
to introduce deployment-agnostic configuration for dataset\r\nquality
tests, specifically targeting
`GET\r\n/internal/dataset_quality/integrations`.","sha":"45c71c5a6b8f53eeb2c82c4319dfac0205668a8c"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193144","number":193144,"mergeCommit":{"message":"[Dataset
quality] Integrations agnostic-deployment api tests (#193144)\n\nRelates
to https://github.com/elastic/kibana/issues/191980.\r\n\r\nThis PR aims
to introduce deployment-agnostic configuration for dataset\r\nquality
tests, specifically targeting
`GET\r\n/internal/dataset_quality/integrations`.","sha":"45c71c5a6b8f53eeb2c82c4319dfac0205668a8c"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
@gbamparop gbamparop added the Team:obs-onboarding Observability Onboarding Team label Oct 1, 2024
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 1, 2024

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

sorenlouv
sorenlouv reviewed Apr 2, 2025
View reviewed changes
...egration/deployment_agnostic/apis/observability/dataset_quality/integrations/integrations.ts
Comment on lines +115 to +117
expect(body.integrations.map((integration: Integration) => integration.name)).to.eql([
'my.custom.integration',
]);
Copy link
Member

@sorenlouv sorenlouv Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yngrdyn I noticed this was failing occasionally in the flaky test runner:

         └-> returns custom integrations and its datasets map
           └-> "before each" hook: global before each for "returns custom integrations and its datasets map"
           └- ✖ fail: Stateful Observability - Deployment-agnostic API integration tests Dataset quality Integrations gets the custom installed integrations returns custom integrations and its datasets map
           │       Error: expected [] to sort of equal [ 'my.custom.integration' ]
           │       + expected - actual
           │
           │
           │       at Assertion.assert (expect.js:100:11)
           │       at Assertion.eql (expect.js:244:8)
           │       at Context.<anonymous> (integrations.ts:136:14)
           │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
           │       at Object.apply (wrap_function.js:74:16)
           │

@yngrdyn yngrdyn mentioned this pull request May 27, 2025
Merged
yngrdyn added a commit that referenced this pull request May 27, 2025
@yngrdyn
[Deployment Agnostic] Adding registryUrl configuration for DA tests (#…
1a5ba16 
…221628)

When introducing package registry configuration in
#193144 I missed the set up of
`xpack.fleet.registryUrl` which has resulted in flaky tests whenever the
test uses fleet underneath.
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request May 29, 2025
@yngrdyn @akowalska622
[Deployment Agnostic] Adding registryUrl configuration for DA tests (e…
c2e28cb 
…lastic#221628)

When introducing package registry configuration in
elastic#193144 I missed the set up of
`xpack.fleet.registryUrl` which has resulted in flaky tests whenever the
test uses fleet underneath.
yngrdyn added a commit that referenced this pull request Jun 5, 2025
@yngrdyn @dmlemeshko
[8.19] [Deployment Agnostic] Adding registryUrl configuration for DA …
739d2ab 
…tests (#221628) (#222736)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Deployment Agnostic] Adding registryUrl configuration for DA tests
(#221628)](#221628)

<!--- Backport version: 10.0.0 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Yngrid
Coello","email":"yngrid.coello@elastic.co"},"sourceCommit":{"committedDate":"2025-05-27T17:14:24Z","message":"[Deployment
Agnostic] Adding registryUrl configuration for DA tests
(#221628)\n\nWhen introducing package registry configuration
in\nhttps://github.com//pull/193144 I missed the set up
of\n`xpack.fleet.registryUrl` which has resulted in flaky tests whenever
the\ntest uses fleet
underneath.","sha":"1a5ba164838a212ded3760afb0a932f7d208cf11","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport","release_note:skip","backport
missing","v9.0.0","v9.1.0","v8.19.0"],"title":"[Deployment Agnostic]
Adding registryUrl configuration for DA
tests","number":221628,"url":"https://github.com/elastic/kibana/pull/221628","mergeCommit":{"message":"[Deployment
Agnostic] Adding registryUrl configuration for DA tests
(#221628)\n\nWhen introducing package registry configuration
in\nhttps://github.com//pull/193144 I missed the set up
of\n`xpack.fleet.registryUrl` which has resulted in flaky tests whenever
the\ntest uses fleet
underneath.","sha":"1a5ba164838a212ded3760afb0a932f7d208cf11"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/221628","number":221628,"mergeCommit":{"message":"[Deployment
Agnostic] Adding registryUrl configuration for DA tests
(#221628)\n\nWhen introducing package registry configuration
in\nhttps://github.com//pull/193144 I missed the set up
of\n`xpack.fleet.registryUrl` which has resulted in flaky tests whenever
the\ntest uses fleet
underneath.","sha":"1a5ba164838a212ded3760afb0a932f7d208cf11"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
tonyghiani pushed a commit to tonyghiani/kibana that referenced this pull request Jul 15, 2025
@yngrdyn
[Deployment Agnostic] Adding registryUrl configuration for DA tests (e…
f1bcc8f 
…lastic#221628)

When introducing package registry configuration in
elastic#193144 I missed the set up of
`xpack.fleet.registryUrl` which has resulted in flaky tests whenever the
test uses fleet underneath.

(cherry picked from commit 1a5ba16)

# Conflicts:
#	x-pack/test/api_integration/deployment_agnostic/default_configs/serverless.config.base.ts
#	x-pack/test/api_integration/deployment_agnostic/default_configs/stateful.config.base.ts
tonyghiani added a commit that referenced this pull request Jul 15, 2025
@tonyghiani @yngrdyn
[9.0] [Deployment Agnostic] Adding registryUrl configuration for DA t…
3fc4823 
…ests (#221628) (#227942)

# Backport

Closes #205017

This will backport the following commits from `main` to `9.0`:
- [[Deployment Agnostic] Adding registryUrl configuration for DA tests
(#221628)](#221628)

<!--- Backport version: 10.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Yngrid
Coello","email":"yngrid.coello@elastic.co"},"sourceCommit":{"committedDate":"2025-05-27T17:14:24Z","message":"[Deployment
Agnostic] Adding registryUrl configuration for DA tests
(#221628)\n\nWhen introducing package registry configuration
in\nhttps://github.com//pull/193144 I missed the set up
of\n`xpack.fleet.registryUrl` which has resulted in flaky tests whenever
the\ntest uses fleet
underneath.","sha":"1a5ba164838a212ded3760afb0a932f7d208cf11","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport","release_note:skip","v9.1.0","v8.19.0"],"title":"[Deployment
Agnostic] Adding registryUrl configuration for DA
tests","number":221628,"url":"https://github.com/elastic/kibana/pull/221628","mergeCommit":{"message":"[Deployment
Agnostic] Adding registryUrl configuration for DA tests
(#221628)\n\nWhen introducing package registry configuration
in\nhttps://github.com//pull/193144 I missed the set up
of\n`xpack.fleet.registryUrl` which has resulted in flaky tests whenever
the\ntest uses fleet
underneath.","sha":"1a5ba164838a212ded3760afb0a932f7d208cf11"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/221628","number":221628,"mergeCommit":{"message":"[Deployment
Agnostic] Adding registryUrl configuration for DA tests
(#221628)\n\nWhen introducing package registry configuration
in\nhttps://github.com//pull/193144 I missed the set up
of\n`xpack.fleet.registryUrl` which has resulted in flaky tests whenever
the\ntest uses fleet
underneath.","sha":"1a5ba164838a212ded3760afb0a932f7d208cf11"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/222736","number":222736,"state":"MERGED","mergeCommit":{"sha":"739d2abeedf4604c2c37b14c04263a2b370455a5","message":"[8.19]
[Deployment Agnostic] Adding registryUrl configuration for DA tests
(#221628) (#222736)\n\n# Backport\n\nThis will backport the following
commits from `main` to `8.19`:\n- [[Deployment Agnostic] Adding
registryUrl configuration for DA
tests\n(#221628)](https://github.com/elastic/kibana/pull/221628)\n\n\n\n###
Questions ?\nPlease refer to the [Backport
tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by:
Dzmitry Lemechko <dzmitry.lemechko@elastic.co>"}}]}] BACKPORT-->

Co-authored-by: Yngrid Coello <yngrid.coello@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sorenlouv sorenlouv sorenlouv left review comments

@achyutjhunjhunwala achyutjhunjhunwala achyutjhunjhunwala approved these changes

@dmlemeshko dmlemeshko dmlemeshko approved these changes

Assignees

No one assigned

Labels

ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:obs-onboarding Observability Onboarding Team v8.16.0 v9.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@yngrdyn @obltmachine @kibana-ci @kibanamachine @elasticmachine @sorenlouv @achyutjhunjhunwala @dmlemeshko @gbamparop