[Agent Builder] Add detection engineering prebuilt skill with tools#258362
Closed
patrykkopycinski wants to merge 2 commits intoelastic:mainfrom
Closed
[Agent Builder] Add detection engineering prebuilt skill with tools#258362patrykkopycinski wants to merge 2 commits intoelastic:mainfrom
patrykkopycinski wants to merge 2 commits intoelastic:mainfrom
Conversation
Adds a comprehensive detection engineering skill for Agent Builder that covers the full detection lifecycle: threat analysis, rule creation, MITRE ATT&CK coverage assessment, rule testing, tuning, monitoring, prebuilt rule onboarding, and exception management. New registry tools: - security.find_rules — search and filter detection rules - security.manage_rules — bulk enable/disable/duplicate rules - security.preview_rule — test rule queries against live data - security.manage_exceptions — find, prepare, and overlap-check exceptions - security.coverage_overview — analyze MITRE ATT&CK coverage - security.rule_monitoring — check rule execution health The skill includes 9 end-to-end workflow definitions covering scenarios from new data source onboarding to SOC analyst bulk false positive fixes, plus reference content for rule type selection and MITRE ATT&CK mapping. All tools are gated behind the aiRuleCreationEnabled experimental flag. 145 unit tests covering schema validation, availability, and handlers.
Contributor
Author
|
/ci |
Contributor
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
…add eval suite - Refactor 6 detection engineering tools from globally-registered BuiltinToolDefinition to skill-scoped BuiltinSkillBoundedTool (inline tools), preventing global tool library pollution when the skill is not active - Update skill definition to accept core/logger dependencies and expose tools via getInlineTools instead of getRegistryTools - Create @kbn/evals-suite-security-detection-engineering evaluation suite with: - Skill selection dataset (6 examples across all tool types) - Tool invocation dataset (5 examples validating parameter accuracy) - Workflow quality dataset (4 multi-step e2e scenarios) - Criteria + ToolCalls + Correctness + Groundedness evaluators
This was referenced Apr 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a comprehensive detection engineering prebuilt skill for Agent Builder that covers the full detection lifecycle from threat analysis to production deployment and ongoing monitoring.
New registry tools (6)
security.find_rules— Search and filter detection rules by name, type, tags, enabled status, prebuilt/customsecurity.manage_rules— Bulk enable, disable, or duplicate rulessecurity.preview_rule— Test rule queries (ES|QL, EQL, KQL, threshold) against live data without creating a rulesecurity.manage_exceptions— Find existing exceptions, detect overlapping conditions, prepare new exception definitionssecurity.coverage_overview— Analyze MITRE ATT&CK coverage across all rules, identify gaps and unmapped rulessecurity.rule_monitoring— Check rule execution health: errors, performance metrics, alert generation ratesSkill content
security.alerts,security.create_detection_rule,security.security_labs_search,platform.core.cases,platform.core.execute_esql,platform.core.generate_esqlAll tools gated behind
aiRuleCreationEnabledexperimental feature flag.Files changed
skills/detection_engineering/index.ts,content.tsfind_rules_tool.ts,manage_rules_tool.ts,preview_rule_tool.ts,manage_exceptions_tool.ts,coverage_overview_tool.ts,rule_monitoring_tool.tsregister_skills.ts,register_tools.ts,tools/index.tsTest plan
yarn test:type_check --project)Made with Cursor
Production-Readiness Checklist — Agent Skills Ecosystem
Generated against [Epic] Creation of the Agent Skills Ecosystem for Elastic Security.
Narrative role: Detection Engineering skill. Target of the vision's feedback loop ("Triage → DE: FP patterns identified, rules tuned, exceptions created, new detections authored").
Must-do before this can ship
@kbn/evalssuites for each of the 9 e2e scenarios — 145 unit tests alone don't satisfy the vision's measurement requirementsecurity.apply_triage_feedbacktool (or equivalent) so the AI Triage skill's FP output can automatically land here as exceptions / rule tuning suggestions (with HITL approval)security.preview_ruleruns real ES queries — add a server-side time window + row cap guard to avoid runaway preview costcoverage_overviewmust return a structured MITRE matrix payload (not prose), so it can be consumed by the MITRE Auto-Mapper skill (#258978)required_fieldsandrelated_integrationssemantics with #260159 Data Source Catalog — today informational only, the DS catalog makes them validatableaiRuleCreationEnabledgating; add a kill switch path on the six new toolsFollow-ups (post-merge)