skills: improve Scout scaffold generation and migration guidance

.agent/skills/ftr-testing/SKILL.md

@@ -12,6 +12,7 @@ FTR (FunctionalTestRunner) runs Kibana UI functional tests written in mocha with
 
 1. Identify the FTR config and test file location.
    - FTR suites live under `test/**` or `x-pack/**/test/**` with config files.
+   - To confirm a config is actually executed in CI, check the relevant `.buildkite/ftr_*_configs.yml` and whether it’s listed under `enabled:` vs `disabled:`.
 2. Understand the test structure.
    - Tests export a provider function that defines a mocha suite.
    - Use `describe/it/before/beforeEach/after/afterEach`.

.agent/skills/scout-create-scaffold/SKILL.md

@@ -1,123 +1,76 @@
 ---
 name: Scout Create Scaffold
-description: Use when creating, relocating, or reviewing Scout test scaffolds for a Kibana module and you must choose the correct test/scout path, ui/api split, or file naming conventions.
+description: Generate or repair a Scout test scaffold for a Kibana plugin/package (test/scout*/{api,ui} Playwright configs, fixtures, example specs). Use when you need the initial Scout directory structure; prefer `node scripts/scout.js generate` with flags for non-interactive/LLM execution.
 ---
 
-# Create Scout Scaffold
+# Create Scout Scaffold (Generator-First)
 
-## Overview
+## Inputs to Collect
 
-Place Scout tests next to the module under test using the repo's Scout path conventions. Core principle: derive the test root from the target module path first, then choose the ui/api split and naming rules.
+- Module root (repo-relative path to the plugin/package directory, e.g. `x-pack/platform/plugins/shared/maps`)
+- Test type: `api`, `ui`, or `both`
+- Scout root directory name under `<moduleRoot>/test/`
+  - Default: `scout` (creates `<moduleRoot>/test/scout/...`)
+  - Custom config set: `scout_<configSet>` (for example `scout_uiam_local`, `scout_cspm_agentless`)
+- For UI scaffolds: whether tests can run in parallel (space-isolated). Default is parallel; use sequential when isolation is not possible.
 
-## Core workflow
+## Generate (Preferred)
 
-1. Determine the module root.
-   - Walk up from the target file path until you find `kibana.jsonc` (preferred).
-   - If none, use the directory that represents the module boundary (plugin/package root).
-2. Choose test type.
-   - UI test: uses Playwright page or page objects.
-   - API test: uses HTTP clients only (no browser).
-3. Choose the Scout root.
-   - Default: `<module-root>/test/scout`.
-   - If a custom Scout root exists (for example `test/scout_cspm_agentless`, `test/scout_uiam_local`), place new tests under the same root.
-4. Prefer scaffolding via the Scout CLI when starting from scratch.
-   - `node scripts/scout.js generate` (also picks the correct Scout package import for the module)
-5. Place files using Scout path conventions.
-6. Create the minimal scaffold (see checklist below).
+Run from the Kibana repo root:
 
-## Quick reference
+```bash
+node scripts/scout.js generate --path <moduleRoot> --type <api|ui|both>
+```
 
-| Purpose              | Path pattern                                                           |
-| -------------------- | ---------------------------------------------------------------------- |
-| UI tests (sequential) | `test/scout{_<configSet>}/ui/tests/**/*.spec.ts`                        |
-| UI tests (parallel)  | `test/scout{_<configSet>}/ui/parallel_tests/**/*.spec.ts`              |
-| API tests (sequential) | `test/scout{_<configSet>}/api/tests/**/*.spec.ts`                      |
-| API tests (parallel) | `test/scout{_<configSet>}/api/parallel_tests/**/*.spec.ts`             |
-| Global setup         | `test/scout{_<configSet>}/{ui,api}/{tests,parallel_tests}/global.setup.ts` |
+Common variants:
 
-## Path conventions
+```bash
+# UI scaffold, sequential (non-parallel)
+node scripts/scout.js generate --path <moduleRoot> --type ui --no-ui-parallel
 
-Scout test files must match:
+# Generate into a custom Scout root (test/scout_<configSet>/...)
+node scripts/scout.js generate --path <moduleRoot> --type both --scout-root scout_<configSet>
 
-UI:
-```
-<module-root>/test/scout{_<configSet>}/ui/{tests,parallel_tests}/**/*.spec.ts
-```
-
-API:
-```
-<module-root>/test/scout{_<configSet>}/api/{tests,parallel_tests}/**/*.spec.ts
+# If some Scout directories already exist, generate only missing sections without prompting
+node scripts/scout.js generate --path <moduleRoot> --type both --force
 ```
 
 Notes:
 
-- `{_<configSet>}` is optional. Examples: `test/scout` (default), `test/scout_cspm_agentless`, `test/scout_uiam_local`.
-- UI test files live in `ui/tests/` or `ui/parallel_tests/`.
-- API test files live in `api/tests/` (sequential) or `api/parallel_tests/` (parallel). Parallel API runs are supported but require careful isolation (indices, saved objects, etc.).
-- `global.setup.ts` lives under the configured `testDir` and uses worker-scoped fixtures only.
-- Non-test helpers can live under `test/scout*/ui/helpers/` or `test/scout*/common/` (shared UI+API).
-
-## Scaffold checklist
-
-### UI tests
-
-- `test/scout*/ui/playwright.config.ts`
-  - If missing, copy from a similar Scout-enabled plugin in the same solution.
-  - Filename must be exactly `playwright.config.ts` (tooling discovery).
-- Optional: `test/scout*/ui/parallel.playwright.config.ts` (plus `ui/parallel_tests/`)
-  - Filename must be exactly `parallel.playwright.config.ts` (tooling discovery).
-- `test/scout*/ui/fixtures/index.ts`
-  - Provide the `test` fixture export used by UI tests.
-- `test/scout*/ui/tests/`
-  - Place UI `.spec.ts` files here.
-- Optional: `test/scout*/ui/parallel_tests/`
-- Optional: `test/scout*/ui/tests/global.setup.ts` or `parallel_tests/global.setup.ts`
-- Optional: `test/scout*/ui/tsconfig.json` (follow existing pattern in the module).
-
-### API tests
-
-- `test/scout*/api/playwright.config.ts`
-  - If missing, copy from a similar Scout-enabled plugin in the same solution.
-  - Filename must be exactly `playwright.config.ts` (tooling discovery).
-- Optional: `test/scout*/api/parallel.playwright.config.ts` (plus `api/parallel_tests/`)
-  - Filename must be exactly `parallel.playwright.config.ts` (tooling discovery).
-- `test/scout*/api/fixtures/index.ts`
-  - Provide the `apiTest` fixture export used by API tests.
-- `test/scout*/api/tests/`
-  - Place API `.spec.ts` files here.
-- Optional: `test/scout*/api/parallel_tests/`
-- Optional: `test/scout*/api/tests/global.setup.ts` or `parallel_tests/global.setup.ts`
-- Optional: `test/scout*/api/tsconfig.json` (follow existing pattern in the module).
-
-## Example
-
-Request: "Create a new scout test for x-pack/solutions/observability/plugins/apm/public/components/service_overview.tsx"
-
-Outcome:
-
-- Module root: `x-pack/solutions/observability/plugins/apm`
-- UI tests path: `x-pack/solutions/observability/plugins/apm/test/scout/ui/tests/service_overview.spec.ts`
-- Ensure `x-pack/solutions/observability/plugins/apm/test/scout/ui/playwright.config.ts` exists
-- Ensure `x-pack/solutions/observability/plugins/apm/test/scout/ui/fixtures/index.ts` exists
-
-## Common mistakes
-
-- Put test files directly under `test/scout*/ui` instead of `test/scout*/ui/tests`.
-- Forget `{ tag: ... }` on UI suites/tests (Scout validates UI tags at runtime). API tests should also be tagged so CI/discovery can select the right deployment target.
-- Use `.test.ts` or `.ts` instead of `.spec.ts` for Scout tests.
-- Place UI tests under `api` (or API tests under `ui`).
-- Create a new `test/scout_*` root when a module already uses `test/scout` (or vice versa).
-
-## Red flags
-
-- Unsure where the module root is.
-- No `tests/` or `parallel_tests/` directory in the target path.
-- Using `.test.ts` or `.ts` for a Scout test file.
-- Creating a new `test/scout_*` root without a corresponding server config set or an existing precedent in the module.
-
-## Discovery / manifests
-
-- Scout config discovery and CI rely on `.meta` manifests under `<module-root>/test/scout*/.meta/...`.
-- After adding or moving Scout tests (or changing Playwright configs), run `node scripts/scout.js update-test-config-manifests` to refresh manifests.
-- If you add a new custom Scout root like `test/scout_<configSet>`, you typically also need a matching server config set under `src/platform/packages/shared/kbn-scout/src/servers/configs/custom/<configSet>`.
-- `run-tests` auto-detects the custom config dir from the Playwright config path; `start-server` requires `--config-dir <configSet>`.
+- The generator will not modify existing `test/<scout-root>/{api,ui}` sub-directories.
+- If any Scout directories already exist and you pass `--path`, you must also pass `--force` (otherwise the command fails rather than prompting).
+
+## What It Creates
+
+- API scaffold:
+  - `test/<scout-root>/api/playwright.config.ts`
+  - `test/<scout-root>/api/fixtures/constants.ts`
+  - `test/<scout-root>/api/fixtures/index.ts`
+  - `test/<scout-root>/api/tests/example.spec.ts`
+- UI scaffold (sequential):
+  - `test/<scout-root>/ui/playwright.config.ts`
+  - `test/<scout-root>/ui/fixtures/constants.ts`
+  - `test/<scout-root>/ui/fixtures/index.ts`
+  - `test/<scout-root>/ui/fixtures/page_objects/*`
+  - `test/<scout-root>/ui/tests/example.spec.ts`
+- UI scaffold (parallel):
+  - `test/<scout-root>/ui/parallel.playwright.config.ts`
+  - `test/<scout-root>/ui/parallel_tests/example_one.spec.ts`
+  - `test/<scout-root>/ui/parallel_tests/example_two.spec.ts`
+  - `test/<scout-root>/ui/parallel_tests/global.setup.ts`
+
+## After Generating
+
+- Update `.meta` manifests when adding/moving configs or tests:
+  - `node scripts/scout.js update-test-config-manifests`
+- CI plumbing:
+  - Add the plugin/package to `.buildkite/scout_ci_config.yml` (the generator warns about this).
+- Custom server config sets:
+  - If you create/use `test/scout_<configSet>`, you typically also need a matching server config under `src/platform/packages/shared/kbn-scout/src/servers/configs/custom/<configSet>`.
+  - `start-server` requires `--config-dir <configSet>` when using a custom server config set.
+
+## Path Conventions (Specs)
+
+- UI sequential specs: `test/scout*/ui/tests/**/*.spec.ts`
+- UI parallel specs: `test/scout*/ui/parallel_tests/**/*.spec.ts`
+- API specs: `test/scout*/api/tests/**/*.spec.ts`

.agent/skills/scout-migrate-from-ftr/references/migration-workflow.md

@@ -16,10 +16,34 @@ Use this as a checklist when migrating FTR tests to Scout.
 
 ## 3) Translate the test structure
 
-- `describe/it` -> `test.describe/test` or `apiTest.describe/apiTest`.
+- `describe/it` -> `test.describe/test` or `apiTest.describe/apiTest` (but don’t assume 1:1 `it` -> `test`).
 - `before/after` -> `test.beforeAll/test.afterAll`.
 - `beforeEach/afterEach` -> `test.beforeEach/test.afterEach`.
 
+### `it` blocks are sometimes steps (not full test cases)
+
+In FTR it’s common for multiple `it(...)` blocks in one `describe(...)` to behave like a single user journey (shared browser state across `it`s).
+In Scout (Playwright), each `test(...)` runs with a fresh browser context, so you usually can’t preserve that state across multiple `test`s.
+
+Guideline:
+
+- If the FTR suite uses multiple `it(...)` blocks as sequential steps of one flow, combine them into a single `test(...)` and convert the step boundaries into `test.step(...)`.
+- If an `it(...)` block is already an independent test case, keep it as its own `test(...)` and ensure it sets up its own preconditions.
+
+Minimal sketch:
+
+```ts
+// FTR: multiple `it`s continue in the same browser context
+it('create entity', async () => {});
+it('edit entity', async () => {}); // continues...
+
+// Scout: combine into one test and use `test.step` for debuggability
+test('create and edit entity', async () => {
+  await test.step('create entity', async () => {});
+  await test.step('edit entity', async () => {});
+});
+```
+
 ## 4) Replace FTR dependencies
 
 - Replace FTR services with Scout fixtures (`pageObjects`, `browserAuth`, `apiClient`, `apiServices`, `kbnClient`, `esArchiver`, `requestAuth`, `samlAuth`).