Skip to content

[Dataset quality] configuring dockerized package registry in tests#234891

Closed
yngrdyn wants to merge 15 commits intoelastic:mainfrom
yngrdyn:using-dockerized-package-registry-in-tests
Closed

[Dataset quality] configuring dockerized package registry in tests#234891
yngrdyn wants to merge 15 commits intoelastic:mainfrom
yngrdyn:using-dockerized-package-registry-in-tests

Conversation

@yngrdyn
Copy link
Copy Markdown
Contributor

@yngrdyn yngrdyn commented Sep 12, 2025

Closes #233620, #234087, #235057, #235275, #235633, #235723

We have been dealing with a lot of flakiness in our tests due to package registry not being available, after checking the configs I discovered we are not using the dockerized package registry, we are only using it in our api tests (which seems to be more stable).

This PR aims to use dockerized package registry for all our tests.
As a stretch goal of this PR, I agreed with @dmlemeshko to centralize the package-registry config into @kbn/test since was duplicated all over the place and it also benefit other teams, special configuration can be still added inside defineDockerServersConfig and overriden for special cases

@yngrdyn yngrdyn requested review from a team as code owners September 12, 2025 12:18
@yngrdyn yngrdyn requested a review from a team September 12, 2025 12:18
@yngrdyn yngrdyn requested a review from a team as a code owner September 12, 2025 12:18
@botelastic botelastic Bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. labels Sep 12, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@github-actions
Copy link
Copy Markdown
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@yngrdyn yngrdyn added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Sep 12, 2025
Copy link
Copy Markdown
Contributor

@mohamedhamed-ahmed mohamedhamed-ahmed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Great job spotting this 🤞 hope this makes our tests less flaky

@yngrdyn yngrdyn requested a review from a team as a code owner September 15, 2025 11:06
@dmlemeshko dmlemeshko self-requested a review September 15, 2025 11:16
@dmlemeshko dmlemeshko requested a review from a team as a code owner September 15, 2025 14:25
Copy link
Copy Markdown
Member

@jennypavlova jennypavlova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 Thank you for fixing that!

const bitbucketDatasetName = 'atlassian_bitbucket.audit';
const bitbucketAuditDataStreamName = `logs-${bitbucketDatasetName}-${defaultNamespace}`;
const bitbucketPkg = {
name: 'atlassian_bitbucket',
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

atlassian_bitbucket is not part of distribution:lite docker

`--server.publicBaseUrl=${kbnUrl}`,
// Allow dynamic config overrides in tests
`--coreApp.allowDynamicConfigOverrides=true`,
...(dockerRegistryPort
Copy link
Copy Markdown
Contributor Author

@yngrdyn yngrdyn Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmlemeshko this configuration was not set before the changes so I'm not sure the docker was being used
Do you know if that was the case?
And I believe that's why all scout tests are failing.
Does scout needs this configuration to query the local package registry?

`--permissionsPolicy.report_to=${JSON.stringify(['violations-endpoint'])}`,
// Allow dynamic config overrides in tests
`--coreApp.allowDynamicConfigOverrides=true`,
...(dockerRegistryPort
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here @dmlemeshko

@yngrdyn yngrdyn requested a review from a team as a code owner November 20, 2025 07:55
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Nov 20, 2025

💔 Build Failed

  • Buildkite Build
  • Commit: 0782f83
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-234891-0782f83e4660

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #47 / cloud_security_posture Serverless - Agentless CIS Integration Page "after all" hook in "Serverless - Agentless CIS Integration Page"
  • [job] [logs] FTR Configs #47 / cloud_security_posture Serverless - Agentless CIS Integration Page "after all" hook in "Serverless - Agentless CIS Integration Page"
  • [job] [logs] FTR Configs #47 / cloud_security_posture Serverless - Agentless CIS Integration Page Serverless - Agentless CIS_AWS Single Account Launch Cloud formation should show CIS_AWS Launch Cloud formation button when credentials selector is direct access keys
  • [job] [logs] FTR Configs #47 / cloud_security_posture Serverless - Agentless CIS Integration Page Serverless - Agentless CIS_AWS Single Account Launch Cloud formation should show CIS_AWS Launch Cloud formation button when credentials selector is direct access keys
  • [job] [logs] FTR Configs #36 / cloud_security_posture Test Cloud Security Posture Integrations on Serverless Integration installation form should not be available without required PLI
  • [job] [logs] FTR Configs #36 / cloud_security_posture Test Cloud Security Posture Integrations on Serverless Integration installation form should not be available without required PLI
  • [job] [logs] FTR Configs #33 / telemetry Snapshot telemetry should pass the schema validation (ensures BWC with Classic offering)
  • [job] [logs] FTR Configs #33 / telemetry Snapshot telemetry should pass the schema validation (ensures BWC with Classic offering)
  • [job] [logs] APM Cypress Tests #1 / when navigating to integration page should Display Debug section should Display Debug section
  • [job] [logs] APM Cypress Tests #1 / when navigating to integration page should Display Debug section should Display Debug section

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/test 310 318 +8
Unknown metric groups

API count

id before after diff
@kbn/test 367 376 +9

History

@mohamedhamed-ahmed
Copy link
Copy Markdown
Contributor

closing this and opening a new PR to carry on the work there as this is already too old to fix the conflicts.

[Dataset quality] configuring dockerized package registry in tests

mohamedhamed-ahmed added a commit that referenced this pull request Jan 23, 2026
…250040)

## Summary

We have been dealing with a lot of flakiness in our tests due to package
registry not being available, this PR changes the package registery
being used to the dockerized one in Dataset Quality. As well as
centralizing its usage in kbn-test package so that it can later on be
used from a central place by other teams instead of duplicating it in
every test suite.

This is work continuation for [[Dataset quality] configuring dockerized
package registry in
tests](#234891).
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 23, 2026
…lastic#250040)

## Summary

We have been dealing with a lot of flakiness in our tests due to package
registry not being available, this PR changes the package registery
being used to the dockerized one in Dataset Quality. As well as
centralizing its usage in kbn-test package so that it can later on be
used from a central place by other teams instead of duplicating it in
every test suite.

This is work continuation for [[Dataset quality] configuring dockerized
package registry in
tests](elastic#234891).

(cherry picked from commit 8902479)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 23, 2026
…lastic#250040)

## Summary

We have been dealing with a lot of flakiness in our tests due to package
registry not being available, this PR changes the package registery
being used to the dockerized one in Dataset Quality. As well as
centralizing its usage in kbn-test package so that it can later on be
used from a central place by other teams instead of duplicating it in
every test suite.

This is work continuation for [[Dataset quality] configuring dockerized
package registry in
tests](elastic#234891).

(cherry picked from commit 8902479)
mohamedhamed-ahmed added a commit to mohamedhamed-ahmed/kibana that referenced this pull request Jan 24, 2026
…lastic#250040)

## Summary

We have been dealing with a lot of flakiness in our tests due to package
registry not being available, this PR changes the package registery
being used to the dockerized one in Dataset Quality. As well as
centralizing its usage in kbn-test package so that it can later on be
used from a central place by other teams instead of duplicating it in
every test suite.

This is work continuation for [[Dataset quality] configuring dockerized
package registry in
tests](elastic#234891).

(cherry picked from commit 8902479)

# Conflicts:
#	x-pack/platform/test/api_integration_deployment_agnostic/default_configs/feature_flag.serverless.config.base.ts
#	x-pack/platform/test/api_integration_deployment_agnostic/default_configs/serverless.config.base.ts
#	x-pack/solutions/observability/test/functional/config.base.ts
mohamedhamed-ahmed added a commit to mohamedhamed-ahmed/kibana that referenced this pull request Jan 24, 2026
…lastic#250040)

## Summary

We have been dealing with a lot of flakiness in our tests due to package
registry not being available, this PR changes the package registery
being used to the dockerized one in Dataset Quality. As well as
centralizing its usage in kbn-test package so that it can later on be
used from a central place by other teams instead of duplicating it in
every test suite.

This is work continuation for [[Dataset quality] configuring dockerized
package registry in
tests](elastic#234891).

(cherry picked from commit 8902479)

# Conflicts:
#	x-pack/platform/test/api_integration_deployment_agnostic/default_configs/feature_flag.serverless.config.base.ts
#	x-pack/platform/test/api_integration_deployment_agnostic/default_configs/serverless.config.base.ts
#	x-pack/solutions/observability/test/functional/config.base.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failing test: Serverless Observability Functional Tests.x-pack/solutions/observability/test/serverless/functional/test_suites/dataset_quality/dataset_quality_details·ts - serverless observability UI Dataset Quality Dataset quality details navigate to dataset details should navigate to details page from a main page Failing test: X-Pack Observability Functional UI Tests.x-pack/solutions/observability/test/functional/apps/dataset_quality/dataset_quality_privileges·ts - Dataset Quality Dataset quality handles user privileges User can read logs-* User cannot monitor any data stream "before all" hook for "Estimated data are not available due to underprivileged user" Failing test: X-Pack Observability Functional UI Tests.x-pack/solutions/observability/test/functional/apps/dataset_quality/dataset_quality_privileges·ts - Dataset Quality Dataset quality handles user privileges User has access to dataset quality with limited privileges "after all" hook in "User has access to dataset quality with limited privileges" Failing test: X-Pack Observability Functional UI Tests.x-pack/solutions/observability/test/functional/apps/dataset_quality/dataset_quality_privileges·ts - Dataset Quality Dataset quality handles user privileges User has access to dataset quality with limited privileges User has access to a single data stream "before all" hook for "should still be able to navigate and use the dataset quality app" Failing test: Serverless Observability Functional Tests.x-pack/test_serverless/functional/test_suites/observability/dataset_quality/dataset_quality_details·ts - serverless observability UI Dataset Quality Dataset quality details navigate to dataset details should navigate to details page from a main page

9 participants