Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor IdentityAwarePlugin interface to be assigned a client for executing actions #16976

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Indexed IP field supports `terms_query` with more than 1025 IP masks [#16391](https://github.com/opensearch-project/OpenSearch/pull/16391)
- Make entries for dependencies from server/build.gradle to gradle version catalog ([#16707](https://github.com/opensearch-project/OpenSearch/pull/16707))
- Allow extended plugins to be optional ([#16909](https://github.com/opensearch-project/OpenSearch/pull/16909))
- Refactor IdentityAwarePlugin interface to be assigned a client for executing actions ([#16976](https://github.com/opensearch-project/OpenSearch/pull/16976))

### Deprecated
- Performing update operation with default pipeline or final pipeline is deprecated ([#16712](https://github.com/opensearch-project/OpenSearch/pull/16712))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import org.apache.shiro.SecurityUtils;
import org.apache.shiro.mgt.SecurityManager;
import org.opensearch.client.Client;
import org.opensearch.client.FilterClient;
import org.opensearch.client.node.NodeClient;
import org.opensearch.cluster.metadata.IndexNameExpressionResolver;
import org.opensearch.cluster.service.ClusterService;
Expand All @@ -23,8 +24,8 @@
import org.opensearch.core.xcontent.NamedXContentRegistry;
import org.opensearch.env.Environment;
import org.opensearch.env.NodeEnvironment;
import org.opensearch.identity.PluginSubject;
import org.opensearch.identity.Subject;
import org.opensearch.identity.noop.RunAsSystemClient;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I see noop I think it does nothing but this seems to be doing "something". Is this the right package for it?

Copy link
Member Author

@cwperks cwperks Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the rationale, and sorry if there's not enough background context on the PR description.

  1. IdentityPlugin - In practical terms, the Identity Plugin is the security plugin as its the plugin that authenticates a request and provides the identity of the caller.
  2. IdentityAwarePlugins - I'm not crazy about the name here, but these are plugins that are aware of the fact that they need to do privileged operations outside of the authenticated user context. For instance, needing system index access which currently requires the usage of ThreadContext.stashContext.

There are 2 different scenarios this PR needs to account for:

  1. Cluster running w/o security
  2. Cluster running w/ security

When security is not installed, there is no IdentityPlugin and what's provided to the IdentityAwarePlugins is this RunAsSystemClient. This client does the current system index access pattern seen across the plugins, it stashes the thread context before executing a transport action. This client will then restore back the original context before delegating back to the original actionListener.

When security is installed, what would be provided is not this class, but another client defined by the security plugin. Its not introduced yet, but the code may look similar to this:

public class RunAsClient extends FilterClient {
    private final NamedPrincipal pluginPrincipal;
    private final User pluginUser;

    public RunAsClient(Client delegate, Plugin plugin) {
        super(delegate);
        String principal = "plugin:" + plugin.getClass().getCanonicalName();
        this.pluginPrincipal = new NamedPrincipal(principal);
        // Convention for plugin username. Prefixed with 'plugin:'. ':' is forbidden from usernames, so this
        // guarantees that a user with this username cannot be created by other means.
        this.pluginUser = new User(principal);
    }

    public NamedPrincipal getPrincipal() {
        return pluginPrincipal;
    }

    @Override
    protected <Request extends ActionRequest, Response extends ActionResponse> void doExecute(
        ActionType<Response> action,
        Request request,
        ActionListener<Response> actionListener
    ) {
        ThreadContext threadContext = threadPool().getThreadContext();

        try (ThreadContext.StoredContext ctx = threadContext.stashContext()) {

            ActionListener<Response> wrappedListener = ActionListener.wrap(r -> {
                ctx.restore();
                actionListener.onResponse(r);
            }, e -> {
                ctx.restore();
                actionListener.onFailure(e);
            });

            threadContext.putTransient(ConfigConstants.OPENDISTRO_SECURITY_USER, pluginUser);
            super.doExecute(action, request, wrappedListener);
        }
    }
}

This client stashes the threadcontext, but then it injects an identity corresponding to the respective plugin that this client was assigned to. Security will use this identity to run authz checks which it does not currently do today. Currently, plugins can perform any action and are allowed to do so. The intent of this client is to allow system index access (to their own system indices) and prohibit other actions unless the cluster admin explicitly allows a plugin to perform an action outside the authenticated user context.

The one in this PR is in a package called noop because there's a notion of a NoopIdentityPlugin, but I agree that the naming is confusing.

Copy link
Member Author

@cwperks cwperks Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plugins already get a node client through createComponents. This particular client (the client provided in IdentityAwarePlugin.assignRunAsClient) is intended to perform operations outside the authenticated user context (in the context of the plugin if you will).

I want to work towards cluster administrators knowing explicitly what actions a plugin will perform outside the authenticated user context and have the cluster administrator sign-off at installation time. Similar to JSM.

For instance, one use-case the security plugin will need facilitated is the ability to write to the audit log index if a cluster is using an opensearch index for the audit log. The security plugin needs a guarantee that writes to this index will succeed regardless of the callers permissions and it stashes the ThreadContext to do this operation today.

import org.opensearch.identity.tokens.AuthToken;
import org.opensearch.identity.tokens.TokenManager;
import org.opensearch.plugins.ActionPlugin;
Expand Down Expand Up @@ -54,6 +55,7 @@
private final ShiroTokenManager authTokenHandler;

private ThreadPool threadPool;
private Client client;

/**
* Create a new instance of the Shiro Identity Plugin
Expand Down Expand Up @@ -83,6 +85,7 @@
Supplier<RepositoriesService> repositoriesServiceSupplier
) {
this.threadPool = threadPool;
this.client = client;

Check warning on line 88 in plugins/identity-shiro/src/main/java/org/opensearch/identity/shiro/ShiroIdentityPlugin.java

View check run for this annotation

Codecov / codecov/patch

plugins/identity-shiro/src/main/java/org/opensearch/identity/shiro/ShiroIdentityPlugin.java#L88

Added line #L88 was not covered by tests
return Collections.emptyList();
}

Expand Down Expand Up @@ -138,7 +141,7 @@
}
}

public PluginSubject getPluginSubject(Plugin plugin) {
return new ShiroPluginSubject(threadPool);
public FilterClient getRunAsClient(Plugin plugin) {
cwperks marked this conversation as resolved.
Show resolved Hide resolved
return new RunAsSystemClient(client);

Check warning on line 145 in plugins/identity-shiro/src/main/java/org/opensearch/identity/shiro/ShiroIdentityPlugin.java

View check run for this annotation

Codecov / codecov/patch

plugins/identity-shiro/src/main/java/org/opensearch/identity/shiro/ShiroIdentityPlugin.java#L145

Added line #L145 was not covered by tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the new multitenancy feature, we're using this wrapped execution with a different client interface. So the thread context stashing wraps other code elsewhere (with a client object included) using both clients (client for thread pools and sdkClient for the indices). How would this object enable such a pattern?

See https://github.com/opensearch-project/ml-commons/blob/3bbc70077bc76f790077cf46666569017a7032ed/plugin/src/main/java/org/opensearch/ml/action/connector/GetConnectorTransportAction.java#L83-L96 for an example of delegating to a utility class method

See https://github.com/opensearch-project/ml-commons/blob/3bbc70077bc76f790077cf46666569017a7032ed/plugin/src/main/java/org/opensearch/ml/action/connector/DeleteConnectorTransportAction.java#L121-L162 for a wrapped call using a different client interface

}
}

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@
package org.opensearch.identity.shiro;

import org.opensearch.OpenSearchException;
import org.opensearch.client.Client;
import org.opensearch.common.settings.Settings;
import org.opensearch.identity.IdentityService;
import org.opensearch.plugins.IdentityPlugin;
import org.opensearch.test.OpenSearchTestCase;
import org.opensearch.threadpool.ThreadPool;

import java.util.List;

Expand All @@ -28,7 +28,7 @@ public class ShiroIdentityPluginTests extends OpenSearchTestCase {
public void testSingleIdentityPluginSucceeds() {
IdentityPlugin identityPlugin1 = new ShiroIdentityPlugin(Settings.EMPTY);
List<IdentityPlugin> pluginList1 = List.of(identityPlugin1);
IdentityService identityService1 = new IdentityService(Settings.EMPTY, mock(ThreadPool.class), pluginList1);
IdentityService identityService1 = new IdentityService(Settings.EMPTY, mock(Client.class), pluginList1);
assertThat(identityService1.getTokenManager(), is(instanceOf(ShiroTokenManager.class)));
}

Expand All @@ -37,10 +37,7 @@ public void testMultipleIdentityPluginsFail() {
IdentityPlugin identityPlugin2 = new ShiroIdentityPlugin(Settings.EMPTY);
IdentityPlugin identityPlugin3 = new ShiroIdentityPlugin(Settings.EMPTY);
List<IdentityPlugin> pluginList = List.of(identityPlugin1, identityPlugin2, identityPlugin3);
Exception ex = assertThrows(
OpenSearchException.class,
() -> new IdentityService(Settings.EMPTY, mock(ThreadPool.class), pluginList)
);
Exception ex = assertThrows(OpenSearchException.class, () -> new IdentityService(Settings.EMPTY, mock(Client.class), pluginList));
assert (ex.getMessage().contains("Multiple identity plugins are not supported,"));
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.OpenSearchException;
import org.opensearch.client.Client;
import org.opensearch.common.annotation.InternalApi;
import org.opensearch.common.settings.Settings;
import org.opensearch.identity.noop.NoopIdentityPlugin;
import org.opensearch.identity.tokens.TokenManager;
import org.opensearch.plugins.IdentityAwarePlugin;
import org.opensearch.plugins.IdentityPlugin;
import org.opensearch.plugins.Plugin;
import org.opensearch.threadpool.ThreadPool;

import java.util.List;
import java.util.stream.Collectors;
Expand All @@ -30,14 +30,16 @@
private static final Logger log = LogManager.getLogger(IdentityService.class);

private final Settings settings;
private final Client client;
private final IdentityPlugin identityPlugin;

public IdentityService(final Settings settings, final ThreadPool threadPool, final List<IdentityPlugin> identityPlugins) {
public IdentityService(final Settings settings, final Client client, final List<IdentityPlugin> identityPlugins) {
this.settings = settings;
this.client = client;

if (identityPlugins.size() == 0) {
log.debug("Identity plugins size is 0");
identityPlugin = new NoopIdentityPlugin(threadPool);
identityPlugin = new NoopIdentityPlugin(client);
} else if (identityPlugins.size() == 1) {
log.debug("Identity plugins size is 1");
identityPlugin = identityPlugins.get(0);
Expand Down Expand Up @@ -66,8 +68,8 @@
public void initializeIdentityAwarePlugins(final List<IdentityAwarePlugin> identityAwarePlugins) {
if (identityAwarePlugins != null) {
for (IdentityAwarePlugin plugin : identityAwarePlugins) {
PluginSubject pluginSubject = identityPlugin.getPluginSubject((Plugin) plugin);
plugin.assignSubject(pluginSubject);
Client client = identityPlugin.getRunAsClient((Plugin) plugin);
plugin.assignRunAsClient(client);

Check warning on line 72 in server/src/main/java/org/opensearch/identity/IdentityService.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/IdentityService.java#L71-L72

Added lines #L71 - L72 were not covered by tests
}
}
}
Expand Down
19 changes: 0 additions & 19 deletions server/src/main/java/org/opensearch/identity/PluginSubject.java

This file was deleted.

8 changes: 0 additions & 8 deletions server/src/main/java/org/opensearch/identity/Subject.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
import org.opensearch.common.annotation.ExperimentalApi;

import java.security.Principal;
import java.util.concurrent.Callable;

/**
* An individual, process, or device that causes information to flow among objects or change to the system state.
Expand All @@ -22,11 +21,4 @@ public interface Subject {
* Get the application-wide uniquely identifying principal
* */
Principal getPrincipal();

/**
* runAs allows the caller to run a callable function as this subject
*/
default <T> T runAs(Callable<T> callable) throws Exception {
return callable.call();
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@

package org.opensearch.identity.noop;

import org.opensearch.identity.PluginSubject;
import org.opensearch.client.Client;
import org.opensearch.client.FilterClient;
import org.opensearch.identity.Subject;
import org.opensearch.identity.tokens.TokenManager;
import org.opensearch.plugins.IdentityPlugin;
import org.opensearch.plugins.Plugin;
import org.opensearch.threadpool.ThreadPool;

/**
* Implementation of identity plugin that does not enforce authentication or authorization
Expand All @@ -24,10 +24,10 @@
*/
public class NoopIdentityPlugin implements IdentityPlugin {

private final ThreadPool threadPool;
private final Client client;

public NoopIdentityPlugin(ThreadPool threadPool) {
this.threadPool = threadPool;
public NoopIdentityPlugin(Client client) {
this.client = client;
}

/**
Expand All @@ -49,7 +49,7 @@
}

@Override
public PluginSubject getPluginSubject(Plugin plugin) {
return new NoopPluginSubject(threadPool);
public FilterClient getRunAsClient(Plugin plugin) {
return new RunAsSystemClient(client);

Check warning on line 53 in server/src/main/java/org/opensearch/identity/noop/NoopIdentityPlugin.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/NoopIdentityPlugin.java#L53

Added line #L53 was not covered by tests
}
}

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.identity.noop;

import org.opensearch.action.ActionRequest;
import org.opensearch.action.ActionType;
import org.opensearch.client.Client;
import org.opensearch.client.FilterClient;
import org.opensearch.common.annotation.InternalApi;
import org.opensearch.common.util.concurrent.ThreadContext;
import org.opensearch.core.action.ActionListener;
import org.opensearch.core.action.ActionResponse;

/**
* Implementation of client that will run transport actions in a stashed context
* <p>
* This class and related classes in this package will not return nulls or fail permissions checks
*
* This class is used by the NoopIdentityPlugin to initialize IdentityAwarePlugins
*
* @opensearch.internal
*/
@InternalApi
public class RunAsSystemClient extends FilterClient {
Copy link
Member Author

@cwperks cwperks Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't an instance of a new client, its a wrapper around that local node client initialized in Node.java that overrides the doExecute method.

In particular, this is the default implementation that stashes the context prior to executing an action and restores it prior to delegating back to the original actionListener's onResponse or onFailure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the multitenant client mentioned earlier, the pattern is:

  • code stashes context
  • code calls the sdkClient (wrapper client)
  • for the NodeClient implementatoin, wrapper client then calls client.foo() which invokes the "protected" call

So for this to work we would need to change the default client implementation to use this RunAsSystemClient instance conditionally. Not sure how we do that... I'm sure it's possible, though.

See https://github.com/opensearch-project/opensearch-remote-metadata-sdk/blob/main/core/src/main/java/org/opensearch/remote/metadata/client/impl/LocalClusterIndicesClient.java for existing implementation

public RunAsSystemClient(Client delegate) {
super(delegate);
}

Check warning on line 33 in server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java#L32-L33

Added lines #L32 - L33 were not covered by tests

@Override
protected <Request extends ActionRequest, Response extends ActionResponse> void doExecute(
ActionType<Response> action,
Request request,
ActionListener<Response> actionListener
) {
ThreadContext threadContext = threadPool().getThreadContext();

Check warning on line 41 in server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java#L41

Added line #L41 was not covered by tests

try (ThreadContext.StoredContext ctx = threadContext.stashContext()) {

Check warning on line 43 in server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java#L43

Added line #L43 was not covered by tests

ActionListener<Response> wrappedListener = ActionListener.wrap(r -> {
ctx.restore();
Copy link
Member Author

@cwperks cwperks Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main reason for introducing this PR, to ensure that the original context is restored when an action is completed.

When the Security Plugin provides its implementation of a RunAsClient, it would inject a user corresponding to the plugin before doExecute and restore the original context (including authenticated user info) before calling the original actionListener's onResponse or onFailure.

Copy link
Member Author

@cwperks cwperks Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a PR on my own fork of the security plugin to demonstrate how the changes would be integrated into a sample plugin: cwperks/security#40

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rather than ActionListener.wrap( ... restore; onResponse ... restore; onFailure ) why not just use ActionListener.runBefore(listener, () -> context.restore()) (or better yet ActionListener.runBefore(listener, context::restore))?

actionListener.onResponse(r);
}, e -> {
ctx.restore();
actionListener.onFailure(e);
});

Check warning on line 51 in server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java#L45-L51

Added lines #L45 - L51 were not covered by tests

super.doExecute(action, request, wrappedListener);

Check warning on line 53 in server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java#L53

Added line #L53 was not covered by tests
}
}

Check warning on line 55 in server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java

View check run for this annotation

Codecov / codecov/patch

server/src/main/java/org/opensearch/identity/noop/RunAsSystemClient.java#L55

Added line #L55 was not covered by tests
}
Loading
Loading