Skills playground by patrykkopycinski · Pull Request #248561 · elastic/kibana

patrykkopycinski · 2026-01-10T00:15:08Z

Summary

Summarize your PR. If it involves visual changes include a screenshot or gif.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

…angchain

…nto renovate/main-langchain

elasticmachine · 2026-01-10T00:15:25Z

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

Click to trigger kibana-pull-request for this PR!
Click to trigger kibana-deploy-project-from-pr for this PR!
Click to trigger kibana-deploy-cloud-from-pr for this PR!
Click to trigger kibana-entity-store-performance-from-pr for this PR!

…cutor

…otstrap' locally and then commit the changes and push to your branch

…cutor

# Conflicts: # package.json # x-pack/platform/packages/shared/agent-builder/agent-builder-common/index.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-common/tools/constants.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/agents/provider.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/runner/skill_provider.ts # x-pack/platform/packages/shared/kbn-evals/src/kibana_phoenix_client/client.ts # x-pack/platform/plugins/shared/agent_builder/server/services/agents/modes/run_agent.ts # x-pack/platform/plugins/shared/agent_builder/server/services/runner/runner.ts # x-pack/platform/plugins/shared/agent_builder/server/types.ts # x-pack/platform/plugins/shared/agent_builder_platform/kibana.jsonc # x-pack/platform/plugins/shared/cases/public/components/case_view/case_view_page.tsx # x-pack/platform/plugins/shared/cases/public/components/case_view/use_case_attachment_tabs.tsx # x-pack/platform/plugins/shared/dashboard_agent/server/plugin.ts # x-pack/platform/plugins/shared/fleet/kibana.jsonc # x-pack/platform/plugins/shared/fleet/server/plugin.ts # x-pack/solutions/observability/plugins/observability_agent_builder/kibana.jsonc # x-pack/solutions/observability/plugins/observability_agent_builder/server/tools/register_tools.ts # x-pack/solutions/observability/plugins/observability_agent_builder/server/types.ts # x-pack/solutions/security/plugins/security_solution/public/attack_discovery/pages/results/take_action/index.tsx # x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/register_tools.ts # x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/security_labs_search_tool.ts # x-pack/solutions/security/plugins/security_solution/server/plugin.ts # x-pack/solutions/security/test/security_solution_api_integration/test_suites/entity_analytics/utils/index.ts # yarn.lock

…eature/skills-all # Conflicts: # x-pack/platform/packages/shared/kbn-evals/src/evaluate.ts # x-pack/platform/packages/shared/kbn-evals/src/kibana_phoenix_client/client.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/http_handler_from_kbn_client.ts

# Conflicts: # package.json # x-pack/platform/packages/shared/agent-builder/agent-builder-server/agents/provider.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/allow_lists.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/attachments/index.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/attachments/type_definition.ts # x-pack/platform/packages/shared/agent-builder/kbn-evals-suite-agent-builder/evals/product_documentation/product_documentation.spec.ts # x-pack/platform/packages/shared/kbn-evals/index.ts # x-pack/platform/packages/shared/kbn-evals/src/evaluate.ts # x-pack/platform/packages/shared/kbn-evals/src/kibana_evals_executor/client.test.ts # x-pack/platform/packages/shared/kbn-evals/src/kibana_evals_executor/client.ts # x-pack/platform/packages/shared/kbn-evals/src/kibana_phoenix_client/client.ts # x-pack/platform/packages/shared/kbn-evals/src/types.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/analysis.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/evaluation_stats.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/report_model_score.test.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/report_model_score.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/reporting/evaluation_reporter.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/score_repository.ts # x-pack/platform/plugins/shared/agent_builder/server/plugin.ts # x-pack/platform/plugins/shared/agent_builder/server/services/create_services.ts # x-pack/platform/plugins/shared/agent_builder/server/services/runner/types.ts # x-pack/platform/plugins/shared/agent_builder/server/services/skills/index.ts # x-pack/platform/plugins/shared/agent_builder/server/services/skills/types.ts # x-pack/platform/plugins/shared/agent_builder/server/services/types.ts # x-pack/platform/plugins/shared/agent_builder/server/types.ts # x-pack/platform/plugins/shared/agent_builder_platform/server/attachment_types/index.ts # x-pack/platform/plugins/shared/cases/server/client/factory.ts # x-pack/platform/plugins/shared/cases/server/client/types.ts # x-pack/platform/plugins/shared/cases/server/plugin.ts # x-pack/platform/plugins/shared/cases/server/services/index.ts # x-pack/solutions/observability/packages/kbn-evals-suite-obs-ai-assistant/evals/connector/connector.spec.ts # x-pack/solutions/observability/packages/kbn-evals-suite-obs-ai-assistant/evals/documentation/documentation.spec.ts # x-pack/solutions/security/plugins/security_solution/public/plugin.tsx # x-pack/solutions/security/plugins/security_solution/server/agent_builder/skills/register_skills.ts # x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/alerts_tool.ts # x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/attack_discovery_search_tool.test.ts # x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/register_tools.ts # x-pack/solutions/security/plugins/security_solution/server/plugin.ts # yarn.lock

Migrate ~35 skills from legacy `Skill` type to the new `SkillDefinition` type using `defineSkillType` across platform, security, observability, dashboard, fleet, and ML plugins. Replace `createToolProxy` with `getAllowedTools` for built-in tool references, update registration calls from `agentBuilder.skills.register()` to `agentBuilder.skill.registerSkill()`, extend `SkillsDirectoryStructure` with new base paths, and remove legacy `create_tool_proxy` utilities. Update evals AGENTS.md to document new architecture.

- Agent builder: tracing, skill prompts, deep agent mode, tool selection - Osquery: live query service extraction with tests, skill improvements - Security evals: cases eval, HTML reporter, token usage evaluator - Elastic assistant: alert grouping, attack discovery batching/incremental, workflow steps - Inference tracing: baggage, elasticsearch exporter, span processors - Data sources: GitHub issue aggregator with tests - Evals framework: tool usage evaluator, Phoenix client, score repository - Endpoint scripts: agent skills demo, Caldera validation, GCP fleet VM tooling - Cases: attack discovery tab, user actions list updates - Observability: alerts skill updates

# Conflicts: # AGENTS.md # src/platform/packages/shared/kbn-tracing/src/init_tracing.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-common/attachments/versioned_attachment.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/allow_lists.ts # x-pack/platform/packages/shared/agent-builder/agent-builder-server/skills/type_definition.ts # x-pack/platform/packages/shared/agent-builder/kbn-evals-suite-agent-builder/playwright.config.ts # x-pack/platform/packages/shared/agent-builder/kbn-evals-suite-agent-builder/src/evaluate_dataset.ts # x-pack/platform/packages/shared/kbn-evals/README.md # x-pack/platform/packages/shared/kbn-evals/src/evaluate.ts # x-pack/platform/packages/shared/kbn-evals/src/utils/score_repository.ts # x-pack/platform/plugins/shared/agent_builder/public/services/attachments/default_renderers.tsx # x-pack/platform/plugins/shared/agent_builder/server/plugin.ts # x-pack/platform/plugins/shared/agent_builder/server/services/attachments/types.ts # x-pack/platform/plugins/shared/agent_builder/server/services/create_services.ts # x-pack/platform/plugins/shared/agent_builder/server/services/types.ts # x-pack/platform/plugins/shared/agent_builder/server/types.ts # x-pack/platform/plugins/shared/agent_builder_platform/server/attachment_types/index.ts # x-pack/platform/plugins/shared/dashboard_agent/server/plugin.ts # x-pack/platform/plugins/shared/fleet/kibana.jsonc # x-pack/solutions/observability/plugins/observability_agent_builder/server/tools/index.ts # x-pack/solutions/observability/plugins/observability_agent_builder/server/tools/register_tools.ts # x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts # x-pack/solutions/security/plugins/security_solution/server/agent_builder/skills/register_skills.ts # x-pack/solutions/security/plugins/security_solution/server/plugin.ts # yarn.lock

elastic-renovate-prod bot and others added 30 commits September 10, 2025 15:08

Update langchain

24cea34

Merge branch 'main' into renovate/main-langchain

cbe4c89

fix types

57a982e

fix types

7603bba

fix yarn.lock

e5f9701

resolve zod

f738bd9

fix

af56329

test

cf01d0f

fix

15b4145

fix

bfa6da3

test

4356165

Merge branch 'main' of github.com:elastic/kibana into renovate/main-l…

4a643ae

…angchain

Merge branch 'main' of github.com:elastic/kibana into renovate/main-l…

614fba9

…angchain

cleanup

1ecb5be

cleanup

a2a2d9e

revert openai

a113dd8

fix yarn.lock

82d4f48

update tests

a5b4416

cleanup

e37581a

Merge branch 'main' into renovate/main-langchain

b87cab0

Merge branch 'main' into renovate/main-langchain

8695be9

Merge branch 'main' into renovate/main-langchain

b24c361

Merge branch 'main' into renovate/main-langchain

1d2fa51

cleanup

518fdd2

Merge branch 'renovate/main-langchain' of github.com:elastic/kibana i…

bfc4fb9

…nto renovate/main-langchain

Merge branch 'main' into renovate/main-langchain

d9e4f40

Merge branch 'main' into renovate/main-langchain

8c151c2

test

6d0f594

Merge branch 'renovate/main-langchain' of github.com:elastic/kibana i…

c514530

…nto renovate/main-langchain

test

d863c3a

spong and others added 23 commits January 22, 2026 14:16

Add internal eval executor

df23c6c

Merge branch 'main' of github.com:elastic/kibana into kibana-eval-exe…

e37b9ce

…cutor

Changes from yarn openapi:bundle

74ee564

TO FIX: Run node 'scripts/check_pkg_json_semver_ranges && yarn kbn bo…

87edf50

…otstrap' locally and then commit the changes and push to your branch

Changes from security: 3rd-party dependencies

734d304

Changes from node scripts/eslint_all_files --no-cache --fix

2a3004b

Improve logging

a3dc3cc

CI fixes

bfee40a

Add tests, cleanup type usage, update readme

dbcbeb5

Readme update from PR feedback

204d904

Wrap phoenix upsert fix in env var check

9ea4456

Merge branch 'main' of github.com:elastic/kibana into kibana-eval-exe…

c90b003

…cutor

Changes from node scripts/eslint_all_files --no-cache --fix

48e3ef3

skills <-> attachments WIP

cbfc889

skills evals

0bee6e9

evals

3b78f2d

evals

0d881aa

Merge branch 'main' of github.com:elastic/kibana into feature/skills-all

2c07ac7

Merge branch 'main' of github.com:elastic/kibana into feature/skills-all

d1b5761

patrykkopycinski self-assigned this Feb 23, 2026

patrykkopycinski added ci:cloud-deploy Create or update a Cloud deployment ci:cloud-persist-deployment Persist cloud deployment indefinitely ci:cloud-deploy-elser If set, the ML node in the ES cluster will be deployed with considerations towards the ELSER model labels Feb 23, 2026

patrykkopycinski added 2 commits February 23, 2026 01:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skills playground#248561

Skills playground#248561
patrykkopycinski wants to merge 74 commits intoelastic:mainfrom
patrykkopycinski:feature/skills-all

patrykkopycinski commented Jan 10, 2026

Uh oh!

elasticmachine commented Jan 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

patrykkopycinski commented Jan 10, 2026

Summary

Checklist

Identify risks

Uh oh!

elasticmachine commented Jan 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants