[scout] Migrate onboarding deployment-agnostic API tests#259436
[scout] Migrate onboarding deployment-agnostic API tests#259436dmlemeshko merged 21 commits intoelastic:mainfrom
Conversation
…ally, or add an exception to src/dev/yarn_deduplicate/index.ts and then commit the changes and push to your branch
d7ac3bd to
9d776b8
Compare
…ally, or add an exception to src/dev/yarn_deduplicate/index.ts and then commit the changes and push to your branch
rmyz
left a comment
There was a problem hiding this comment.
obs-presentation changes LGTM
...bservability_onboarding/test/scout/api/tests/onboarding_flow_update_progress_updates.spec.ts
Outdated
Show resolved
Hide resolved
...ility/plugins/observability_onboarding/test/scout/api/tests/onboarding_flow_progress.spec.ts
Outdated
Show resolved
Hide resolved
...observability/test/api_integration_deployment_agnostic/apis/onboarding/custom_roles/index.ts
Show resolved
Hide resolved
x-pack/solutions/observability/plugins/observability_onboarding/test/scout/README.md
Show resolved
Hide resolved
csr
left a comment
There was a problem hiding this comment.
Overall LGTM, I left a few comments. I also ran the tests on an observability MKI and they all pass.
| await apiTest.step('updates step status', async () => { | ||
| const createFlowResponse = await apiClient.post( | ||
| 'internal/observability_onboarding/flow', | ||
| { | ||
| headers: adminHeaders, | ||
| responseType: 'json', | ||
| } | ||
| ); | ||
| expect(createFlowResponse).toHaveStatusCode(200); | ||
| onboardingId = (createFlowResponse.body as { onboardingFlow: { id: string } }) | ||
| .onboardingFlow.id; |
There was a problem hiding this comment.
🟡 Medium tests/onboarding_flow_update_progress_updates.spec.ts:39
The test creates two onboarding flows (lines 40-49 and 78-87) but onboardingId is overwritten by the second assignment before afterEach runs, so the first saved object is never deleted. Store both IDs or nest the steps in separate tests so cleanup runs between them.
await apiTest.step('updates step status', async () => {
- const createFlowResponse = await apiClient.post(
+ const createFlowResponse1 = await apiClient.post(
'internal/observability_onboarding/flow',
{
headers: adminHeaders,
responseType: 'json',
}
);
- expect(createFlowResponse).toHaveStatusCode(200);
- onboardingId = (createFlowResponse.body as { onboardingFlow: { id: string } })
+ expect(createFlowResponse1).toHaveStatusCode(200);
+ onboardingId = (createFlowResponse1.body as { onboardingFlow: { id: string } })
.onboardingFlow.id;🤖 Copy this AI Prompt to have your agent fix this:
In file x-pack/solutions/observability/plugins/observability_onboarding/test/scout/api/tests/onboarding_flow_update_progress_updates.spec.ts around lines 39-49:
The test creates two onboarding flows (lines 40-49 and 78-87) but `onboardingId` is overwritten by the second assignment before `afterEach` runs, so the first saved object is never deleted. Store both IDs or nest the steps in separate tests so cleanup runs between them.
Evidence trail:
x-pack/solutions/observability/plugins/observability_onboarding/test/scout/api/tests/onboarding_flow_update_progress_updates.spec.ts (viewed at REVIEWED_COMMIT):
- Line 19: `let onboardingId = ''` (single variable)
- Lines 21-28: `afterEach` hook deletes saved object by `onboardingId`
- Lines 48-49: First assignment of `onboardingId` in first step
- Lines 86-87: Second assignment overwrites `onboardingId` in second step
- Both steps are inside a single test (lines 30-113), so `afterEach` only runs once after both steps complete
benakansara
left a comment
There was a problem hiding this comment.
actionable-obs changes LGTM
|
Starting backport for target branches: 9.3 https://github.com/elastic/kibana/actions/runs/23682974324 |
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]
History
|
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
) ## Summary Migrates Observability onboarding deployment-agnostic API integration coverage from the FTR suite (`api_integration_deployment_agnostic/apis/onboarding`) to Scout API tests under `observability_onboarding/test/scout/api`, with `@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless` tags, and auth aligned with the old FTR setup (cookies vs API keys where behavior differs). Removes the FTR onboarding API loadTestFile bundle and wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig + Scout kbn_references) so plugin-relative imports stay valid. | FTR file (removed) | Scout spec | | --- | --- | | `get_progress.ts` | `onboarding_flow_progress.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_updates.spec.ts` | | `custom_roles/index.ts` | *Inlined in `onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole` parity)* | --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
) ## Summary Migrates Observability onboarding deployment-agnostic API integration coverage from the FTR suite (`api_integration_deployment_agnostic/apis/onboarding`) to Scout API tests under `observability_onboarding/test/scout/api`, with `@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless` tags, and auth aligned with the old FTR setup (cookies vs API keys where behavior differs). Removes the FTR onboarding API loadTestFile bundle and wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig + Scout kbn_references) so plugin-relative imports stay valid. | FTR file (removed) | Scout spec | | --- | --- | | `get_progress.ts` | `onboarding_flow_progress.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_updates.spec.ts` | | `custom_roles/index.ts` | *Inlined in `onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole` parity)* | --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit da2f01b) # Conflicts: # .github/CODEOWNERS # x-pack/solutions/observability/plugins/observability_onboarding/moon.yml # x-pack/solutions/observability/plugins/observability_onboarding/tsconfig.json
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
…) (#260418) # Backport This will backport the following commits from `main` to `9.3`: - [[scout] Migrate onboarding deployment-agnostic API tests (#259436)](#259436) <!--- Backport version: 10.2.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2026-03-28T10:09:36Z","message":"[scout] Migrate onboarding deployment-agnostic API tests (#259436)\n\n## Summary\n\nMigrates Observability onboarding deployment-agnostic API integration\ncoverage from the FTR suite\n(`api_integration_deployment_agnostic/apis/onboarding`) to Scout API\ntests under `observability_onboarding/test/scout/api`, with\n`@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless`\ntags, and auth aligned with the old FTR setup (cookies vs API keys where\nbehavior differs). Removes the FTR onboarding API loadTestFile bundle\nand wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig +\nScout kbn_references) so plugin-relative imports stay valid.\n\n| FTR file (removed) | Scout spec |\n| --- | --- |\n| `get_progress.ts` | `onboarding_flow_progress.spec.ts` |\n| `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` |\n| `update_progress.ts` |\n`onboarding_flow_update_progress_updates.spec.ts` |\n| `custom_roles/index.ts` | *Inlined in\n`onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole`\nparity)* |\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"da2f01bb106a62a22fb0dcb57bc26fe760c939a9","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport missing","backport:version","v9.4.0","9.4.0","v9.3.3"],"title":"[scout] Migrate onboarding deployment-agnostic API tests","number":259436,"url":"https://github.com/elastic/kibana/pull/259436","mergeCommit":{"message":"[scout] Migrate onboarding deployment-agnostic API tests (#259436)\n\n## Summary\n\nMigrates Observability onboarding deployment-agnostic API integration\ncoverage from the FTR suite\n(`api_integration_deployment_agnostic/apis/onboarding`) to Scout API\ntests under `observability_onboarding/test/scout/api`, with\n`@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless`\ntags, and auth aligned with the old FTR setup (cookies vs API keys where\nbehavior differs). Removes the FTR onboarding API loadTestFile bundle\nand wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig +\nScout kbn_references) so plugin-relative imports stay valid.\n\n| FTR file (removed) | Scout spec |\n| --- | --- |\n| `get_progress.ts` | `onboarding_flow_progress.spec.ts` |\n| `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` |\n| `update_progress.ts` |\n`onboarding_flow_update_progress_updates.spec.ts` |\n| `custom_roles/index.ts` | *Inlined in\n`onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole`\nparity)* |\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"da2f01bb106a62a22fb0dcb57bc26fe760c939a9"}},"sourceBranch":"main","suggestedTargetBranches":["9.3"],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/259436","number":259436,"mergeCommit":{"message":"[scout] Migrate onboarding deployment-agnostic API tests (#259436)\n\n## Summary\n\nMigrates Observability onboarding deployment-agnostic API integration\ncoverage from the FTR suite\n(`api_integration_deployment_agnostic/apis/onboarding`) to Scout API\ntests under `observability_onboarding/test/scout/api`, with\n`@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless`\ntags, and auth aligned with the old FTR setup (cookies vs API keys where\nbehavior differs). Removes the FTR onboarding API loadTestFile bundle\nand wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig +\nScout kbn_references) so plugin-relative imports stay valid.\n\n| FTR file (removed) | Scout spec |\n| --- | --- |\n| `get_progress.ts` | `onboarding_flow_progress.spec.ts` |\n| `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` |\n| `update_progress.ts` |\n`onboarding_flow_update_progress_updates.spec.ts` |\n| `custom_roles/index.ts` | *Inlined in\n`onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole`\nparity)* |\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"da2f01bb106a62a22fb0dcb57bc26fe760c939a9"}},{"branch":"9.3","label":"v9.3.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
) ## Summary Migrates Observability onboarding deployment-agnostic API integration coverage from the FTR suite (`api_integration_deployment_agnostic/apis/onboarding`) to Scout API tests under `observability_onboarding/test/scout/api`, with `@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless` tags, and auth aligned with the old FTR setup (cookies vs API keys where behavior differs). Removes the FTR onboarding API loadTestFile bundle and wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig + Scout kbn_references) so plugin-relative imports stay valid. | FTR file (removed) | Scout spec | | --- | --- | | `get_progress.ts` | `onboarding_flow_progress.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_updates.spec.ts` | | `custom_roles/index.ts` | *Inlined in `onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole` parity)* | --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
) ## Summary Migrates Observability onboarding deployment-agnostic API integration coverage from the FTR suite (`api_integration_deployment_agnostic/apis/onboarding`) to Scout API tests under `observability_onboarding/test/scout/api`, with `@kbn/scout-oblt` + `synthtrace` fixtures, `stateful` + `serverless` tags, and auth aligned with the old FTR setup (cookies vs API keys where behavior differs). Removes the FTR onboarding API loadTestFile bundle and wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig + Scout kbn_references) so plugin-relative imports stay valid. | FTR file (removed) | Scout spec | | --- | --- | | `get_progress.ts` | `onboarding_flow_progress.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_404.spec.ts` | | `update_progress.ts` | `onboarding_flow_update_progress_updates.spec.ts` | | `custom_roles/index.ts` | *Inlined in `onboarding_flow_progress.spec.ts` (least-privilege / `noAccessUserRole` parity)* | --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…259520) ## Summary: During #259436 I faced few challenges + broken tests in Scout, hopefully SKILLs update make migration more robust File | Change (high level) -- | -- scout-api-testing/SKILL.md | Solution vs deploymentAgnostic tagging for API tests scout-create-scaffold/SKILL.md | Related skill link, Pattern A/B TS layout, post-generate steps scout-migrate-from-ftr/SKILL.md | DA tags, cookieHeader vs API key, synthtrace API, Pattern A/B, pitfalls
…lastic#259520) ## Summary: During elastic#259436 I faced few challenges + broken tests in Scout, hopefully SKILLs update make migration more robust File | Change (high level) -- | -- scout-api-testing/SKILL.md | Solution vs deploymentAgnostic tagging for API tests scout-create-scaffold/SKILL.md | Related skill link, Pattern A/B TS layout, post-generate steps scout-migrate-from-ftr/SKILL.md | DA tags, cookieHeader vs API key, synthtrace API, Pattern A/B, pitfalls
Summary
Migrates Observability onboarding deployment-agnostic API integration coverage from the FTR suite (
api_integration_deployment_agnostic/apis/onboarding) to Scout API tests underobservability_onboarding/test/scout/api, with@kbn/scout-oblt+synthtracefixtures,stateful+serverlesstags, and auth aligned with the old FTR setup (cookies vs API keys where behavior differs). Removes the FTR onboarding API loadTestFile bundle and wires TypeScript Pattern A (test/scout/**/* in the plugin tsconfig + Scout kbn_references) so plugin-relative imports stay valid.get_progress.tsonboarding_flow_progress.spec.tsupdate_progress.tsonboarding_flow_update_progress_404.spec.tsupdate_progress.tsonboarding_flow_update_progress_updates.spec.tscustom_roles/index.tsonboarding_flow_progress.spec.ts(least-privilege /noAccessUserRoleparity)