Skip to content

Add Views Security Model#141050

Merged
elasticsearchmachine merged 34 commits intoelastic:mainfrom
jfreden:views/add_security
Mar 6, 2026
Merged

Add Views Security Model#141050
elasticsearchmachine merged 34 commits intoelastic:mainfrom
jfreden:views/add_security

Conversation

@jfreden
Copy link
Copy Markdown
Contributor

@jfreden jfreden commented Jan 21, 2026

This PR wires ES|QL views into index authorization and view resolution so view expansion preserves the same security/error semantics as direct index access.

  • Adds EsqlResolveViewAction plus ViewResolutionService to resolve views from project metadata using ResolvedIndexExpressions, including strict handling for unauthorized and missing concrete targets.
  • Refactors ViewResolver to resolve views asynchronously, recurse through nested/wildcard view references, detect circular references/max depth, and keep unresolved non-view targets in the rewritten plan rather than dropping them.
  • Updates EsqlSession to run view replacement asynchronously and carry viewQueries in Configuration, so source/view context survives later planning and serialization.
  • Updates view CRUD/read request handling to resolve views via indices options (resolveViews=true) and reuse the same resolution path for GET view.
  • Aligns security behavior by persisting resolved expressions for view-resolving requests and by handling view abstractions correctly in index-permission resource accounting (including selector/failure-store edge cases).

@jfreden jfreden force-pushed the views/add_security branch 7 times, most recently from 773b37e to 3a9b417 Compare January 29, 2026 10:58
@github-actions
Copy link
Copy Markdown
Contributor

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

@jfreden jfreden force-pushed the views/add_security branch 2 times, most recently from 034bde1 to 30b08f9 Compare January 30, 2026 07:05
@jfreden jfreden force-pushed the views/add_security branch 6 times, most recently from 66887a4 to 18323bb Compare February 4, 2026 08:35
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 4, 2026

@jfreden jfreden force-pushed the views/add_security branch 5 times, most recently from a9dfacf to 8604bd1 Compare February 6, 2026 08:11
@jfreden jfreden force-pushed the views/add_security branch 5 times, most recently from c4c9c1a to 748c0aa Compare February 23, 2026 09:14
Copy link
Copy Markdown
Contributor

@n1v0lg n1v0lg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good! Sorry I ran out of time for the day. I just need to wrap up reviewing ViewResolver and the tests. Doing that first thing tomorrow morning.

* processing.
*/
@SuppressWarnings("unchecked")
public void transformDown(BiConsumer<? super T, ActionListener<T>> rule, ActionListener<T> listener) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I'm deferring to the other reviewers here since it's ES|QL code. It looks complicated but I'm assuming there's not way around it.

return false;
}

if (replaceable.getResolvedIndexExpressions() != null) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: It feels off that a boolean "check" style method would trip an assertion -- maybe we can combine shouldSetResolvedIndexExpressions and actually setting the expressions into one method and call it maybeSetResolvedIndexExpressions. WDYT?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going back and forth on this and landed on this approach with the motivation that we will eventually remove shouldSetResolvedIndexExpressions and #135799 will be fixed, and when it is, this is kind of what it would look like IMO.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think #135799 may stick around for some time, unfortunately. It's one of those things that isn't painful enough to get prioritized but maybe I'm overly pessimistic.

views.add(viewMetadata.getView(viewName));
}
var patterns = Arrays.stream(unresolvedRelation.indexPattern().indexPattern().split(","))
.filter(pattern -> Regex.isSimpleMatchPattern(pattern) == false || seenWildcards.add(pattern))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an exclusion edge where this breaks I think: if do something like GET view*,-view-1,view* you should get back results for views* (because exclusions are sensitive to order). That sort of index expression is weird and I can't come up with a good reason to write it so perhaps it's okay to leave it unhandled but ideally we'd fix it somehow.

Copy link
Copy Markdown
Contributor

@n1v0lg n1v0lg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core logic minus exclusions LGTM. Nice test coverage, too 👍

For exclusions, see my inline comments. The main problem is that we don't preserve order and also drop the occurrence of the same pattern incorrectly which breaks some exclusion edge cases. In the interest of progress, I think it's okay to merge this PR but I do think we need exclusion handling follow ups.

views.add(viewMetadata.getView(viewName));
}
var patterns = Arrays.stream(unresolvedRelation.indexPattern().indexPattern().split(","))
.filter(pattern -> Regex.isSimpleMatchPattern(pattern) == false || seenWildcards.add(pattern))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm it the edge case gets a little less far fetched once you write a view with an exclusion in it:

public void testViewBodyWithExclusionCombined() {
    addView("safe-logs", "FROM logs*,-logs-secret");
    addIndex("logs-public");
    addIndex("logs-secret");
    LogicalPlan plan = query("FROM safe-logs,logs-secret");
    assertThat(replaceViews(plan), matchesPlan(query("FROM logs*,-logs-secret,logs-secret")));
}

Is that I'd expect to happen. It would give us logs-secret.

Instead we get:

public void testViewBodyWithExclusionCombined() {
    addView("safe-logs", "FROM logs*,-logs-secret");
    addIndex("logs-public");
    addIndex("logs-secret");
    LogicalPlan plan = query("FROM safe-logs,logs-secret");
    assertThat(replaceViews(plan), matchesPlan(query("FROM logs-secret,logs*,-logs-secret")));
}

Meaning empty.

views.add(viewMetadata.getView(viewName));
}
var patterns = Arrays.stream(unresolvedRelation.indexPattern().indexPattern().split(","))
.filter(pattern -> Regex.isSimpleMatchPattern(pattern) == false || seenWildcards.add(pattern))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FROM safe-logs,logs-secret seems like a fairly natural query to write:

Maybe I started querying safe logs FROM safe-logs and now I'm interested in both safe logs and secret logs so I write:

FROM safe-logs,logs-secret

@jfreden jfreden requested review from a team as code owners March 5, 2026 15:47
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 5, 2026

✅ Vale Linting Results

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

@jfreden jfreden force-pushed the views/add_security branch from 4b46776 to 16d53ba Compare March 5, 2026 15:49
@jfreden jfreden removed request for a team March 5, 2026 15:50
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 6, 2026

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (2)
  • Team:Delivery
  • Team:Search - Inference

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 3889d5ce-5525-4c99-905f-7debfc641b4c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@jfreden jfreden added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Mar 6, 2026
@elasticsearchmachine elasticsearchmachine merged commit bcc0ae3 into elastic:main Mar 6, 2026
42 checks passed
@jfreden jfreden deleted the views/add_security branch March 6, 2026 12:12
spinscale pushed a commit to spinscale/elasticsearch that referenced this pull request Mar 6, 2026
This PR wires ES|QL views into index authorization and view resolution
so view expansion preserves the same security/error semantics as direct
index access.

- Adds `EsqlResolveViewAction` plus `ViewResolutionService` to resolve views from project metadata using `ResolvedIndexExpressions`, including strict handling for unauthorized and missing concrete targets.
- Refactors `ViewResolver` to resolve views asynchronously, recurse through nested/wildcard view references, detect circular references/max depth, and keep unresolved non-view targets in the rewritten plan rather than dropping them.
- Updates `EsqlSession` to run view replacement asynchronously and carry `viewQueries` in `Configuration`, so source/view context survives later planning and serialization.
- Updates view CRUD/read request handling to resolve views via indices options (`resolveViews=true`) and reuse the same resolution path for `GET view`.
- Aligns security behavior by persisting resolved expressions for view-resolving requests and by handling view abstractions correctly in index-permission resource accounting (including selector/failure-store edge cases).
szybia added a commit to szybia/elasticsearch that referenced this pull request Mar 6, 2026
…locations

* upstream/main: (153 commits)
  ES|QL: Update docs for TOP_SNIPPETS and DECAY (elastic#143739)
  Correctly include endpoint id in log msg in AuthorizationPoller (elastic#143743)
  Bar searching or sorting on _seq_no when disabled (elastic#143600)
  Generalize `testClientCancellation` test (elastic#143586)
  JSON_EXTRACT: zero-copy byte slicing for object, array, and number extraction (elastic#143702)
  Track recycler pages in circuit breaker (elastic#143738)
  [ESQL] Enable distributed pipeline breakers for external sources via FragmentExec (elastic#143696)
  Adding 'mode' and 'codec' fields to ES monitoring template (elastic#143673)
  [ESQL] Columnar I/O and vectorized block conversion for external sources (elastic#143703)
  Fix flaky MMR diversification YAML tests (elastic#143706)
  ES|QL codegen: check builder arguments for vector support (elastic#143724)
  Add Views Security Model (elastic#141050)
  ESQL: Prevent pushdown of unmapped fields in filters and sorts (elastic#143460)
  Don't run seq_no pruning tests in release CI (elastic#143725)
  ESQL: Support intra-row field references in ROW command (elastic#140217)
  ES|QL: Remove implicit limit in FORK branches in CSV tests (elastic#143601)
  IndexRoutingTests with and without synthetic id (elastic#143566)
  Synthetic id upgrade test in serverless (elastic#142471)
  Disable "Review skipped" comments for PRs without specified labels (elastic#143728)
  Cleanup ES|QL T-Digest code duplication, add memory accounting (elastic#143662)
  ...
sidosera pushed a commit to sidosera/elasticsearch that referenced this pull request Mar 6, 2026
This PR wires ES|QL views into index authorization and view resolution
so view expansion preserves the same security/error semantics as direct
index access.

- Adds `EsqlResolveViewAction` plus `ViewResolutionService` to resolve views from project metadata using `ResolvedIndexExpressions`, including strict handling for unauthorized and missing concrete targets.
- Refactors `ViewResolver` to resolve views asynchronously, recurse through nested/wildcard view references, detect circular references/max depth, and keep unresolved non-view targets in the rewritten plan rather than dropping them.
- Updates `EsqlSession` to run view replacement asynchronously and carry `viewQueries` in `Configuration`, so source/view context survives later planning and serialization.
- Updates view CRUD/read request handling to resolve views via indices options (`resolveViews=true`) and reuse the same resolution path for `GET view`.
- Aligns security behavior by persisting resolved expressions for view-resolving requests and by handling view abstractions correctly in index-permission resource accounting (including selector/failure-store edge cases).
elasticsearchmachine pushed a commit that referenced this pull request Mar 11, 2026
There has been several discussions on this topic since I opened
#141050 and @idegtiarenko
also mentioned this in
#143561 (comment).


The issue is that both `resolveViews` and `resolveAliases` are in
`WildcardOptions` but are used when resolving both concrete index
pattern targets and wildcard targets. This PR moves `resolveViews` from
`IndicesOptions.WildcardOptions` into a new
`IndicesOptions.IndexAbstractionOptions` record to address this
confusing inconsistency. 

This is in preparation for sending `resolveViews` as a parameter to
field caps for remote view resolving where more work will be done to
serialize `resolveViews` and not having this in place would make for
confusing code. See #143384
for more information on the upcoming serialization of `resolveViews`. 

**_Note_**: `resolveAliases` is moved in a follow up PR since it's
decoupled from the resolveViews changes.
#143953

**_Note_**: `IndicesRequest.includeDataStreams()` is the same type of
flag as `resolveAliases` and `resolveViews`, it controls whether data
streams participate in index resolution. It's a candidate for
`IndexAbstractionOptions` but currently lives on `IndicesRequest` (not
`IndicesOptions`) and is threaded separately through
`IndexNameExpressionResolver.Context` as a constructor parameter. Moving
it would touch 60+ files and change the `IndicesRequest` interface.
Because of how big that change would be, I have not considered doing
that in this PR (or at all).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >enhancement :Security/Security Security issues without another label Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Security Meta label for security team v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants