agent-centric connectors#267333
Merged
Merged
Conversation
Contributor
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
seanstory
force-pushed
the
seanstory/13759-agent-centric-connectors
branch
from
1b8be32 to
2a3dea2
Compare
seanstory
added
release_note:enhancement
Team:agent-builder
backport:skip
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#12097[✅] x-pack/platform/test/agent_builder_functional/config.ts: 25/25 tests passed. |
pgayvallet
reviewed
May 5, 2026
Comment on lines
+43
to
+45
| // Cache connector_ids per request so repeated sml_search calls within | ||
| // the same agent run don't re-fetch the agent definition from ES. | ||
| const connectorIdsCache = new WeakMap<object, Map<string, string[] | undefined>>(); |
Contributor
There was a problem hiding this comment.
WeakMaps are always code smells. I think it would be better to use an LRU cache instead? (with max entries + TTL). We have an lru or lru-cache something module or something we commonly use.
Member
Author
There was a problem hiding this comment.
TIL. I'd never heard of WeakMaps before claude spit this out, but my googling made them sound quite clever. Teach me to fish - what's wrong with them, when used for memoization/caching? This would have ensured that connectorIds didn't need to be looked up multiple times for a single request lifecycle, but that each request would have gotten a fresh lookup.
Comment on lines
+83
to
+88
| try { | ||
| const agentsService = getAgentsService(); | ||
| const registry = await agentsService.getRegistry({ request }); | ||
| const agent = await registry.get(agentEntry.agentId); | ||
| connectorIds = agent.configuration.connector_ids; | ||
| } catch (error) { |
Contributor
There was a problem hiding this comment.
This totally ignores the configuration_override layer (entry point (which we probably shouldn't).
I think I'd really prefer to expose the agent's config from the tool handler context somehow. This feels way more re-usable and generic if other tools / components needed to do the same thing, wdyt?
Member
Author
There was a problem hiding this comment.
Fixed (I think). I didn't expose overwriting connector_ids through the configuration_override, but I think that's ok for now?
lorenabalan
reviewed
May 5, 2026
elastic-vault-github-plugin-prod
Bot
requested a review
from a team
as a code owner
seanstory
force-pushed
the
seanstory/13759-agent-centric-connectors
branch
from
cf4c484 to
bcf9f1d
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent-builder): add connector_ids to agent storage mapping Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent-builder): plumb connector_ids through agent converters Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent-builder): add connector_ids to agent CRUD API schemas Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent_context_layer): add connectorIds filter to SmlService.search() Adds optional `connectorIds` param to `SmlService.search()` and `AgentContextLayerPluginStart.search()`. When provided, builds an ES bool filter that restricts connector-type SML results to those with a matching `origin_id`, while non-connector results pass through unaffected. Exports `buildConnectorFilter` for unit-test coverage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent-builder): sml_search tool filters connectors by agent connector_ids Resolves the executing agent's connector_ids from the run context at search time and forwards them to agentContextLayer.search(), enabling per-agent connector scoping in SML search results. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent-context-layer): accept connector_ids in SML HTTP search endpoint Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> feat(agent-builder): frontend SML search passes connector_ids for agent-scoped filtering Threads the active agent's connector_ids through SmlService.search(), useSmlSearch hook, the SML command menu component, and the prefetch hook so that SML results are filtered to the agent's configured connectors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> fix(agent-builder): make getAgentsService optional in SmlToolsOptions and apply lint fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Refactor connector_ids under a top-level 'filters' Ensure that we can configure agents to not allow any connectors Types fixing Pass AgentConfiguration through ToolHandlerContext Ensure agentConfiguration considers configurationOverrides for tool context Changes from make api-docs
seanstory
force-pushed
the
seanstory/13759-agent-centric-connectors
branch
from
dff6ac8 to
c00c45e
Compare
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
Unknown metric groupsAPI count
History
|
pgayvallet
approved these changes
May 8, 2026
Contributor
pgayvallet
left a comment
pgayvallet
left a comment
There was a problem hiding this comment.
LGTM (did not look at the browser-side code)
lorenabalan
approved these changes
May 8, 2026
Contributor
lorenabalan
left a comment
lorenabalan
left a comment
There was a problem hiding this comment.
Tested and works as expected!
4 tasks
nastasha-solomon
approved these changes
May 8, 2026
Member
nastasha-solomon
left a comment
nastasha-solomon
left a comment
There was a problem hiding this comment.
API doc updates look good, but it might be worth getting a quick check from @leemthompo since he's more familiar with agent builder than I am.
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 8, 2026
Builds on Peter's merged elastic#266573 by adding the schema fields the team converged on and refactoring title/description/content for BM25 + a single unified vector retrieval surface. Fields added: - origin (keyword, URI form `{type}://{id}`, auto-populated by indexer) - tags (keyword[], free-form) - payload (flattened, type-specific opaque data) - title_autocomplete (search_as_you_type, copy target of title) - unified_semantic (semantic_text, copy target of title/description/content) Behavior changes: - title becomes `text` with copy_to fanning into title_autocomplete and unified_semantic. Three retrieval modes (BM25/lexical, prefix/typeahead, semantic) from one producer-set field. - description and content become `text` with copy_to: 'unified_semantic'. One inference pass per record instead of three; recall doesn't fragment across overlapping content. - buildSmlSearchQuery: SAYT field paths move from title.* to title_autocomplete.*; the should-block uses match: { unified_semantic } in place of separate matches on content and description. - Indexer auto-populates origin = `{attachmentType}://{originId}` on every record alongside origin_id. Kept for back-compat: - origin_id retained alongside origin until the connector_ids filter (elastic#267333) and the search HTTP response migrate to origin. Type writers need no changes: they keep returning the same SmlChunk shape; the indexer constructs origin and ES copy_to handles the rest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 8, 2026
Builds on Peter's merged elastic#266573 by adding the schema fields the team converged on and refactoring title/description/content for BM25 + a single unified vector retrieval surface. Fields added: - origin (keyword, URI form `{type}://{id}`, auto-populated by indexer) - tags (keyword[], free-form) - payload (flattened, type-specific opaque data) - title_autocomplete (search_as_you_type, copy target of title) - unified_semantic (semantic_text, copy target of title/description/content) Behavior changes: - title becomes `text` with copy_to fanning into title_autocomplete and unified_semantic. Three retrieval modes (BM25/lexical, prefix/typeahead, semantic) from one producer-set field. - description and content become `text` with copy_to: 'unified_semantic'. One inference pass per record instead of three; recall doesn't fragment across overlapping content. - buildSmlSearchQuery: SAYT field paths move from title.* to title_autocomplete.*; the should-block uses match: { unified_semantic } in place of separate matches on content and description. - Indexer auto-populates origin = `{attachmentType}://{originId}` on every record alongside origin_id. Kept for back-compat: - origin_id retained alongside origin until the connector_ids filter (elastic#267333) and the search HTTP response migrate to origin. Type writers need no changes: they keep returning the same SmlChunk shape; the indexer constructs origin and ES copy_to handles the rest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 8, 2026
Builds on Peter's merged elastic#266573 by adding the schema fields the team converged on and refactoring title/description/content for BM25 + a single unified vector retrieval surface. Fields added: - origin (keyword, URI form `{type}://{id}`, auto-populated by indexer) - tags (keyword[], free-form) - payload (flattened, type-specific opaque data) - title_autocomplete (search_as_you_type, copy target of title) - unified_semantic (semantic_text, copy target of title/description/content) Behavior changes: - title becomes `text` with copy_to fanning into title_autocomplete and unified_semantic. Three retrieval modes (BM25/lexical, prefix/typeahead, semantic) from one producer-set field. - description and content become `text` with copy_to: 'unified_semantic'. One inference pass per record instead of three; recall doesn't fragment across overlapping content. - buildSmlSearchQuery: SAYT field paths move from title.* to title_autocomplete.*; the should-block uses match: { unified_semantic } in place of separate matches on content and description. - Indexer auto-populates origin = `{attachmentType}://{originId}` on every record alongside origin_id. Kept for back-compat: - origin_id retained alongside origin until the connector_ids filter (elastic#267333) and the search HTTP response migrate to origin. Type writers need no changes: they keep returning the same SmlChunk shape; the indexer constructs origin and ES copy_to handles the rest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 8, 2026
Builds on Peter's merged elastic#266573 by adding the schema fields the team converged on and refactoring title/description/content for BM25 + a single unified vector retrieval surface. Fields added: - origin (keyword, URI form `{type}://{id}`, auto-populated by indexer) - tags (keyword[], free-form) - payload (flattened, type-specific opaque data) - title_autocomplete (search_as_you_type, copy target of title) - unified_semantic (semantic_text, copy target of title/description/content) Behavior changes: - title becomes `text` with copy_to fanning into title_autocomplete and unified_semantic. Three retrieval modes (BM25/lexical, prefix/typeahead, semantic) from one producer-set field. - description and content become `text` with copy_to: 'unified_semantic'. One inference pass per record instead of three; recall doesn't fragment across overlapping content. - buildSmlSearchQuery: SAYT field paths move from title.* to title_autocomplete.*; the should-block uses match: { unified_semantic } in place of separate matches on content and description. - Indexer auto-populates origin = `{attachmentType}://{originId}` on every record alongside origin_id. Kept for back-compat: - origin_id retained alongside origin until the connector_ids filter (elastic#267333) and the search HTTP response migrate to origin. Type writers need no changes: they keep returning the same SmlChunk shape; the indexer constructs origin and ES copy_to handles the rest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 11, 2026
…nt connector filter support
Two changes that ship together because they touch the same FE surface:
1. Wire the @ menu typeahead to POST /sml/_autocomplete (instead of
the previously used POST /sml/_search). Adds:
- `smlService.autocomplete({ query, size, filters? })`
- `queryKeys.sml.autocomplete(query, filters?)`
- `useSmlAutocomplete(query, { filters? })` hook
- `render_sml_highlight.tsx` helper that parses ES highlight snippets
(`<em>...</em>`) into React fragments — currently inert on the SAYT
path because of ES bug elastic#53744, forward-compatible when it lands.
- `Sml` component switches to the new hook and renders per-row
`matched_discovery_labels` (`kind`-aware) instead of client-side
`EuiHighlight` on `${type}/${title}`.
- Drops `sml_command_menu_highlight.{ts,test.ts}` — the old
client-side highlight parser is no longer needed.
2. Restore agent-centric connector scoping (Sean's
`agent-centric connectors` feature, elastic#267333) on the autocomplete
route. The earlier autocomplete commit (03da4f4) added the route
without a `filters` parameter, which silently broke the @ menu's
respect for `agent.configuration.connector_ids`. This commit:
- BE: adds `filters?: SmlSearchFilters` to the autocomplete route
schema and threads it through `SmlService.autocomplete` and
`autocompleteSml`, which now calls the existing `buildTypeFilters`
helper (shared with the search path) so the filter clause is
identical between routes.
- FE: `useSmlAutocomplete` accepts `{ filters? }`,
`usePrefetchSml(filters?)` regains the filter param, the
command-menu orchestrator re-prefetches SML when the agent's
filters change after the async agent fetch resolves.
- Tests: new BE unit test asserting filters flow into the ES filter
clauses via `buildTypeFilters`; new FE hook test covering the
filters arg; existing `sml.test.tsx` + `use_prefetch_sml.test.tsx`
updated to the new mocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 tasks
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 12, 2026
…nt connector filter support
Two changes that ship together because they touch the same FE surface:
1. Wire the @ menu typeahead to POST /sml/_autocomplete (instead of
the previously used POST /sml/_search). Adds:
- `smlService.autocomplete({ query, size, filters? })`
- `queryKeys.sml.autocomplete(query, filters?)`
- `useSmlAutocomplete(query, { filters? })` hook
- `render_sml_highlight.tsx` helper that parses ES highlight snippets
(`<em>...</em>`) into React fragments — currently inert on the SAYT
path because of ES bug elastic#53744, forward-compatible when it lands.
- `Sml` component switches to the new hook and renders per-row
`matched_discovery_labels` (`kind`-aware) instead of client-side
`EuiHighlight` on `${type}/${title}`.
- Drops `sml_command_menu_highlight.{ts,test.ts}` — the old
client-side highlight parser is no longer needed.
2. Restore agent-centric connector scoping (Sean's
`agent-centric connectors` feature, elastic#267333) on the autocomplete
route. The earlier autocomplete commit (03da4f4) added the route
without a `filters` parameter, which silently broke the @ menu's
respect for `agent.configuration.connector_ids`. This commit:
- BE: adds `filters?: SmlSearchFilters` to the autocomplete route
schema and threads it through `SmlService.autocomplete` and
`autocompleteSml`, which now calls the existing `buildTypeFilters`
helper (shared with the search path) so the filter clause is
identical between routes.
- FE: `useSmlAutocomplete` accepts `{ filters? }`,
`usePrefetchSml(filters?)` regains the filter param, the
command-menu orchestrator re-prefetches SML when the agent's
filters change after the async agent fetch resolves.
- Tests: new BE unit test asserting filters flow into the ES filter
clauses via `buildTypeFilters`; new FE hook test covering the
filters arg; existing `sml.test.tsx` + `use_prefetch_sml.test.tsx`
updated to the new mocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 tasks
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 13, 2026
…nt connector filter support
Two changes that ship together because they touch the same FE surface:
1. Wire the @ menu typeahead to POST /sml/_autocomplete (instead of
the previously used POST /sml/_search). Adds:
- `smlService.autocomplete({ query, size, filters? })`
- `queryKeys.sml.autocomplete(query, filters?)`
- `useSmlAutocomplete(query, { filters? })` hook
- `render_sml_highlight.tsx` helper that parses ES highlight snippets
(`<em>...</em>`) into React fragments — currently inert on the SAYT
path because of ES bug elastic#53744, forward-compatible when it lands.
- `Sml` component switches to the new hook and renders per-row
`matched_discovery_labels` (`kind`-aware) instead of client-side
`EuiHighlight` on `${type}/${title}`.
- Drops `sml_command_menu_highlight.{ts,test.ts}` — the old
client-side highlight parser is no longer needed.
2. Restore agent-centric connector scoping (Sean's
`agent-centric connectors` feature, elastic#267333) on the autocomplete
route. The earlier autocomplete commit (03da4f4) added the route
without a `filters` parameter, which silently broke the @ menu's
respect for `agent.configuration.connector_ids`. This commit:
- BE: adds `filters?: SmlSearchFilters` to the autocomplete route
schema and threads it through `SmlService.autocomplete` and
`autocompleteSml`, which now calls the existing `buildTypeFilters`
helper (shared with the search path) so the filter clause is
identical between routes.
- FE: `useSmlAutocomplete` accepts `{ filters? }`,
`usePrefetchSml(filters?)` regains the filter param, the
command-menu orchestrator re-prefetches SML when the agent's
filters change after the async agent fetch resolves.
- Tests: new BE unit test asserting filters flow into the ES filter
clauses via `buildTypeFilters`; new FE hook test covering the
filters arg; existing `sml.test.tsx` + `use_prefetch_sml.test.tsx`
updated to the new mocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apmats
added a commit
to Apmats/kibana
that referenced
this pull request
May 13, 2026
…lastic#14363) Land the natural-language search API over SML, replacing the transitional bool.should mash with a proper RRF retriever and introducing the trust-boundary split between runtime-imposed scoping and agent-discoverable filters. Decisions per the implementation prompt §6: - §6.1 Retrieval: RRF over BM25 (title^2, description, content via best_fields multi_match) and semantic (unified_semantic). Filters mirrored to each child retriever so RRF can't pull unauthorized docs into the fused top-k. Defaults rank_constant=60, rank_window_size=50. - §6.2 Filter merging: option (2). Service signature splits `scoping?: SmlSearchScoping` (runtime-imposed, unconditional) from `filters?: SmlSearchFilters` (caller refinement). Trust boundary visible at the API layer; merge happens in one place. Sean's per-type id-allowlist shape (elastic#267333) survives unchanged as `SmlSearchScoping`. - §6.3 Agent-supplied dimensions: `types?: string[]`, `tags?: string[]`, each lowering into a `terms` clause. Time-range and free-form payload filters deferred. - §6.4 Compact LLM response: SmlSearchResult drops the `content` blob, full `payload`, dates, and discovery_labels; gains `more_content` (true when the indexed record has non-empty content worth fetching via sml_read). `permissions` retained internally for post-hoc filtering, stripped from the HTTP response. - §6.5 Reference resolution: URI strings only — SML does not dereference. Inline-expansion deferred until eval data justifies it (Reading 1; Sean's "SML shouldn't turn pointers into blobs"). - §6.6 Tool description rewrite with when-to-use guidance and four worked examples (plain query, types+tags filter, connector restriction, wildcard inventory). Runtime scoping is not exposed in the LLM input schema. Fixes the `total` bug: post-permission-filter `results.length` no longer overwrites ES `hits.total.value`. Frontend / HTTP route updates: - POST /sml/_search body accepts `scoping?` and `filters?` (replaces the old `filters` per-type id-allowlist field and `skip_content`). - POST /sml/_autocomplete renames `filters` → `scoping`; no agent-discoverable filters on this route (no LLM in the path). - Frontend SmlService, useSmlSearch, useSmlAutocomplete, usePrefetchSml, queryKeys, and buildSmlScopingFromAgent (formerly buildSmlFiltersFromAgent) all follow the new naming. Tests: - New unit tests for RRF body, filter threading through child retrievers, vocabulary-mismatch over the semantic leg, more_content on/off, compact result mapping, agent-filter builder, and tool wrapper scoping/filters mapping. - FTR + Scout integration tests updated to drop content/skip_content assertions and lock in the no-content invariant. Out of scope (deferred per epic): reranker, kneedle, per-type retriever profiles, HyDE/query expansion, reference inline-expansion, telemetry instrumentation (elastic#14366), recrawl/versioning (elastic#14367). Open coordination points (not blocking this PR): - Inference model on unified_semantic still relies on defaults (Kathleen Mar 19: pin Jina V5). Belongs to A. - Kathleen review on A's PR proposes splitting unified_semantic into per-field semantic_text fields. If A reverts unified, the semantic leg in this PR fans out from 1 to 2-3 child retrievers — small retriever-only change. - Pierre's configuration_override-aware tool handler context (elastic#267333 review) — wrapper currently reads agentConfiguration from the tool handler context, which `runner.ts` already resolves with overrides applied via `resolveConfiguration`. Note in code. Stacked on top of apmats/sml-autocomplete-followup (elastic#14364). Once autocomplete merges, this PR rebases to main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables scoping agents to a subset of configured connectors, and adjusts SML search to only the connectors that are enabled for the current Agent.
To test:
@(all your connectors auto-complete)@(only your configured connectors auto-complete)Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:*label is applied per the guidelinesbackport:*labels.Release Notes
Introduces an optional configuration for Agent Builder Agents:
connector_ids. If unset, a given agent has access to all configured kibana connectors. When set, the Agent will have access only to the listed connectors.