[Security Solution][Agent Builder] Clean up PR #256060 follow-ups (symlinks, broken eval JSON, PCI flag)#268146
Merged
patrykkopycinski merged 1 commit intoMay 7, 2026
Conversation
PR elastic#256060 (Add PCI compliance skill and tools for Agent Builder) accidentally committed two personal-machine symlinks and a malformed JSON file that breaks CI pipeline parsing. This PR: - Removes the `elastic-llm-benchmarker` symlink that pointed at `/Users/patrykkopycinski/Projects/automaker/elastic-llm-benchmarker` (an absolute path that only exists on a single contributor's machine; nothing in the tree references it). - Removes the `openspec/specs` symlink that recursively pointed at itself (`/Users/patrykkopycinski/Projects/kibana/openspec/specs`). No tracked file references it either. - Fixes invalid JSON in `.buildkite/pipelines/evals/evals.suites.json`: the new `pci-compliance` suite entry was missing the closing `},{` before the `security-automatic-migrations` entry, so the file could not be parsed by any JSON consumer (including the eval pipeline). Verified with `python3 -m json.tool`. - Flips `pciComplianceAgentBuilder` from `false` to `true` in `x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts` so the PCI compliance skill and its four backing tools register by default. The flag is still respected by `register_skills.ts` / `register_tools.ts` so individual environments can opt out.
patrykkopycinski
added
release_note:skip
4 tasks
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
|
KDKHD
approved these changes
May 7, 2026
romulets
pushed a commit
to romulets/kibana
that referenced
this pull request
May 8, 2026
…ups (symlinks, broken eval JSON, PCI flag) (elastic#268146) ## Summary Follow-up cleanup for elastic#256060 (Add PCI compliance skill and tools for Agent Builder). That PR accidentally committed two personal-machine symlinks and merged the new eval-suite entry in a way that produced invalid JSON. This PR fixes those regressions and enables the PCI compliance skill by default. ## What this PR does 1. **Removes `elastic-llm-benchmarker` symlink.** Pointed at `/Users/patrykkopycinski/Projects/automaker/elastic-llm-benchmarker` — an absolute path that only exists on a single contributor's machine, and a directory that is not part of this repo. Nothing tracked in the tree references it (`git grep elastic-llm-benchmarker` is empty). 2. **Removes `openspec/specs` symlink.** Pointed at `/Users/patrykkopycinski/Projects/kibana/openspec/specs` — a self-referential absolute path (the symlink itself). Same story: no tracked file references it. 3. **Fixes invalid JSON in `.buildkite/pipelines/evals/evals.suites.json`.** The new `pci-compliance` suite entry from elastic#256060 is missing the closing `},` before the next entry, so the file is not valid JSON as merged. Reproducible with `python3 -m json.tool < .buildkite/pipelines/evals/evals.suites.json`. After this PR the file parses and contains 15 well-formed suites. 4. **Flips `pciComplianceAgentBuilder` from `false` to `true`** in `x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts`. The flag is still respected by `register_skills.ts` (gates `pciComplianceSkill`) and `register_tools.ts` (gates the four PCI tools), so any environment can still opt out by setting it back to `false` via `xpack.securitySolution.enableExperimental`. ## Test plan - [ ] `node scripts/check_changes.ts` - [ ] CI loads `.buildkite/pipelines/evals/evals.suites.json` without parse error. - [ ] PCI compliance skill and tools register at startup with default config (no `enableExperimental` overrides). - [ ] Existing PCI-compliance Jest tests continue to pass: - `x-pack/solutions/security/plugins/security_solution/server/agent_builder/skills/pci_compliance/pci_compliance_skill.test.ts` - `x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/pci_*_tool.test.ts` ## Notes for reviewers - The two deleted symlinks were committed as gitlinks of mode `120000` and contained absolute paths from a single developer's laptop. They are dead weight in every other clone and should never have been tracked. - I confirmed neither `tsconfig.base.json`, `package.json`, `.github/CODEOWNERS`, nor any other tracked file references either symlink path, so removing them is risk-free.
js-jankisalvi
pushed a commit
to js-jankisalvi/kibana
that referenced
this pull request
May 12, 2026
…ups (symlinks, broken eval JSON, PCI flag) (elastic#268146) ## Summary Follow-up cleanup for elastic#256060 (Add PCI compliance skill and tools for Agent Builder). That PR accidentally committed two personal-machine symlinks and merged the new eval-suite entry in a way that produced invalid JSON. This PR fixes those regressions and enables the PCI compliance skill by default. ## What this PR does 1. **Removes `elastic-llm-benchmarker` symlink.** Pointed at `/Users/patrykkopycinski/Projects/automaker/elastic-llm-benchmarker` — an absolute path that only exists on a single contributor's machine, and a directory that is not part of this repo. Nothing tracked in the tree references it (`git grep elastic-llm-benchmarker` is empty). 2. **Removes `openspec/specs` symlink.** Pointed at `/Users/patrykkopycinski/Projects/kibana/openspec/specs` — a self-referential absolute path (the symlink itself). Same story: no tracked file references it. 3. **Fixes invalid JSON in `.buildkite/pipelines/evals/evals.suites.json`.** The new `pci-compliance` suite entry from elastic#256060 is missing the closing `},` before the next entry, so the file is not valid JSON as merged. Reproducible with `python3 -m json.tool < .buildkite/pipelines/evals/evals.suites.json`. After this PR the file parses and contains 15 well-formed suites. 4. **Flips `pciComplianceAgentBuilder` from `false` to `true`** in `x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts`. The flag is still respected by `register_skills.ts` (gates `pciComplianceSkill`) and `register_tools.ts` (gates the four PCI tools), so any environment can still opt out by setting it back to `false` via `xpack.securitySolution.enableExperimental`. ## Test plan - [ ] `node scripts/check_changes.ts` - [ ] CI loads `.buildkite/pipelines/evals/evals.suites.json` without parse error. - [ ] PCI compliance skill and tools register at startup with default config (no `enableExperimental` overrides). - [ ] Existing PCI-compliance Jest tests continue to pass: - `x-pack/solutions/security/plugins/security_solution/server/agent_builder/skills/pci_compliance/pci_compliance_skill.test.ts` - `x-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/pci_*_tool.test.ts` ## Notes for reviewers - The two deleted symlinks were committed as gitlinks of mode `120000` and contained absolute paths from a single developer's laptop. They are dead weight in every other clone and should never have been tracked. - I confirmed neither `tsconfig.base.json`, `package.json`, `.github/CODEOWNERS`, nor any other tracked file references either symlink path, so removing them is risk-free.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up cleanup for #256060 (Add PCI compliance skill and tools for Agent Builder). That PR accidentally committed two personal-machine symlinks and merged the new eval-suite entry in a way that produced invalid JSON. This PR fixes those regressions and enables the PCI compliance skill by default.
What this PR does
Removes
elastic-llm-benchmarkersymlink. Pointed at/Users/patrykkopycinski/Projects/automaker/elastic-llm-benchmarker— an absolute path that only exists on a single contributor's machine, and a directory that is not part of this repo. Nothing tracked in the tree references it (git grep elastic-llm-benchmarkeris empty).Removes
openspec/specssymlink. Pointed at/Users/patrykkopycinski/Projects/kibana/openspec/specs— a self-referential absolute path (the symlink itself). Same story: no tracked file references it.Fixes invalid JSON in
.buildkite/pipelines/evals/evals.suites.json. The newpci-compliancesuite entry from Add PCI compliance skill and tools for Agent Builder #256060 is missing the closing},before the next entry, so the file is not valid JSON as merged. Reproducible withpython3 -m json.tool < .buildkite/pipelines/evals/evals.suites.json. After this PR the file parses and contains 15 well-formed suites.Flips
pciComplianceAgentBuilderfromfalsetotrueinx-pack/solutions/security/plugins/security_solution/common/experimental_features.ts. The flag is still respected byregister_skills.ts(gatespciComplianceSkill) andregister_tools.ts(gates the four PCI tools), so any environment can still opt out by setting it back tofalseviaxpack.securitySolution.enableExperimental.Test plan
node scripts/check_changes.ts.buildkite/pipelines/evals/evals.suites.jsonwithout parse error.enableExperimentaloverrides).x-pack/solutions/security/plugins/security_solution/server/agent_builder/skills/pci_compliance/pci_compliance_skill.test.tsx-pack/solutions/security/plugins/security_solution/server/agent_builder/tools/pci_*_tool.test.tsNotes for reviewers
120000and contained absolute paths from a single developer's laptop. They are dead weight in every other clone and should never have been tracked.tsconfig.base.json,package.json,.github/CODEOWNERS, nor any other tracked file references either symlink path, so removing them is risk-free.