diff --git a/docs/features/index.md b/docs/features/index.md index e4a2dc7cd..bc93c251f 100644 --- a/docs/features/index.md +++ b/docs/features/index.md @@ -142,6 +142,7 @@ - [Security Configuration](security/security-configuration.md) - [Security Debugging](security/security-debugging.md) - [Security Permissions](security/security-permissions.md) +- [Security Performance Improvements](security/security-performance-improvements.md) - [Security Plugin](security/security-plugin.md) - [Security Plugin Dependencies](security/security-plugin-dependencies.md) - [Security Role Mapping](security/security-role-mapping.md) diff --git a/docs/features/security/security-performance-improvements.md b/docs/features/security/security-performance-improvements.md new file mode 100644 index 000000000..aaa104472 --- /dev/null +++ b/docs/features/security/security-performance-improvements.md @@ -0,0 +1,158 @@ +# Security Performance Improvements + +## Summary + +The OpenSearch security layer includes performance optimizations that reduce the overhead of authentication, authorization, and access control operations. These improvements focus on reducing serialization costs during inter-node communication and optimizing privilege evaluation. The immutable User object, introduced in v3.1.0, enables caching of serialized data and eliminates thread synchronization overhead. + +## Details + +### Architecture + +```mermaid +graph TB + subgraph "Authentication Flow" + REQ[REST Request] --> BR[BackendRegistry] + BR --> AC[AuthenticationCache] + AC -->|Cache Miss| AB[AuthenticationBackend] + AB --> UF[UserFactory] + UF --> USER[Immutable User] + AC -->|Cache Hit| USER + end + + subgraph "Inter-Node Communication" + USER --> SER[Serialize Once] + SER --> CACHE[Serialization Cache] + CACHE --> N1[Node 1] + CACHE --> N2[Node 2] + CACHE --> NN[Node N] + end + + subgraph "Deserialization" + N1 --> DC[Deserialization Cache] + N2 --> DC + NN --> DC + DC --> RUSER[Reused User Object] + end +``` + +### Data Flow + +```mermaid +flowchart TB + A[HTTP Request] --> B{User Cached?} + B -->|Yes| C[Return Cached User] + B -->|No| D[Authenticate via Backend] + D --> E[Create Immutable User] + E --> F[Cache User + Serialized Form] + F --> C + C --> G{Inter-Node Request?} + G -->|Yes| H[Reuse Cached Serialization] + G -->|No| I[Process Locally] + H --> J[Remote Node] + J --> K{User in Cache?} + K -->|Yes| L[Return Cached User] + K -->|No| M[Deserialize + Cache] + M --> L +``` + +### Components + +| Component | Description | +|-----------|-------------| +| `User` | Immutable user object with builder-style modification methods | +| `UserFactory` | Factory interface for User object creation and deserialization | +| `UserFactory.Caching` | Production implementation with LRU cache for deserialized users | +| `UserFactory.Simple` | Simple implementation without caching for testing | +| `AuthenticationContext` | Context object passed through authentication/authorization flow | +| `ImpersonationBackend` | Interface for backends supporting user impersonation | +| `BackendRegistry` | Central authentication coordinator with user caching | + +### Configuration + +| Setting | Description | Default | +|---------|-------------|---------| +| `plugins.security.user_cache.max_size` | Maximum cached User objects in deserialization cache | 10000 | +| `plugins.security.user_cache.expire_after_access` | Time before cached entries expire | 1h | + +### Usage Example + +```java +// Creating an immutable User object +User user = new User("username") + .withRoles(ImmutableSet.of("role1", "role2")) + .withSecurityRoles(ImmutableSet.of("admin")) + .withAttributes(ImmutableMap.of("attr1", "value1")); + +// Modifying returns a new object (original unchanged) +User modifiedUser = user.withRoles("additional_role"); + +// Serialization is cached +String serialized = user.toSerializedBase64(); // Computed once, cached + +// Deserialization uses cache +UserFactory factory = new UserFactory.Caching(settings); +User deserializedUser = factory.fromSerializedBase64(serialized); // Cache lookup +``` + +### Custom Backend Migration + +```java +// Old AuthenticationBackend implementation +public class CustomAuthBackend implements AuthenticationBackend { + @Override + public User authenticate(AuthCredentials credentials) { + User user = new User(credentials.getUsername()); + user.addRoles(lookupRoles(credentials)); + return user; + } + + @Override + public boolean exists(User user) { + return userExists(user.getName()); + } +} + +// New implementation for v3.1.0+ +public class CustomAuthBackend implements AuthenticationBackend, ImpersonationBackend { + @Override + public User authenticate(AuthenticationContext context) { + AuthCredentials credentials = context.getCredentials(); + return new User(credentials.getUsername()) + .withRoles(lookupRoles(credentials)); + } + + @Override + public Optional impersonate(User user) { + if (userExists(user.getName())) { + return Optional.of(user.withAttributes(lookupAttributes(user.getName()))); + } + return Optional.empty(); + } +} +``` + +## Limitations + +- Memory overhead from caching serialized User data and deserialized objects +- Custom authentication/authorization backends require code changes for v3.1.0 compatibility +- Cache effectiveness depends on user session patterns; highly dynamic user populations may see reduced benefits + +## Related PRs + +| Version | PR | Description | +|---------|-----|-------------| +| v3.1.0 | [#5339](https://github.com/opensearch-project/security/pull/5339) | Remove unused custom User serialization | +| v3.1.0 | [#5212](https://github.com/opensearch-project/security/pull/5212) | Immutable user object | + +## References + +- [Issue #5168](https://github.com/opensearch-project/security/issues/5168): Make User object immutable +- [Issue #5200](https://github.com/opensearch-project/security/issues/5200): Serialization backward compatibility +- [Issue #3870](https://github.com/opensearch-project/security/issues/3870): Optimized privilege evaluation +- [Blog: Performance optimizations for the OpenSearch security layer](https://opensearch.org/blog/performance-optimizations-for-the-opensearch-security-layer/) + +## Change History + +- **v3.1.0** (2025-05-26): Immutable User object with serialization caching +- **v3.0.0**: Optimized privilege evaluation for document- and field-level security +- **v2.19.0**: Initial optimized privilege evaluation diff --git a/docs/releases/v3.1.0/features/security/security-performance-improvements.md b/docs/releases/v3.1.0/features/security/security-performance-improvements.md new file mode 100644 index 000000000..f535eaa04 --- /dev/null +++ b/docs/releases/v3.1.0/features/security/security-performance-improvements.md @@ -0,0 +1,139 @@ +# Security Performance Improvements + +## Summary + +OpenSearch 3.1.0 introduces significant performance improvements to the security layer through an immutable User object implementation. This optimization reduces serialization overhead during inter-node communication, resulting in improved throughput for clusters with users having many roles and attributes. + +## Details + +### What's New in v3.1.0 + +The User object has been redesigned to be immutable, enabling several performance optimizations: + +1. **Serialization Reuse**: Serialized binary data can be computed once and reused across multiple requests +2. **Deserialization Caching**: A cache maps serialized binary data to User objects, reducing deserialization operations +3. **Thread Safety**: Immutable objects are inherently thread-safe, eliminating synchronization overhead +4. **Reduced Memory Churn**: Fewer object allocations during request processing + +### Technical Changes + +#### Architecture Changes + +```mermaid +graph TB + subgraph "Before v3.1.0" + A1[REST Request] --> B1[BackendRegistry] + B1 --> C1[Mutable User Object] + C1 --> D1[Serialize per Node] + D1 --> E1[Node 1] + D1 --> F1[Node 2] + D1 --> G1[Node N] + E1 --> H1[Deserialize] + F1 --> I1[Deserialize] + G1 --> J1[Deserialize] + end + + subgraph "After v3.1.0" + A2[REST Request] --> B2[BackendRegistry] + B2 --> C2[Immutable User Object] + C2 --> D2[Serialize Once + Cache] + D2 --> E2[Node 1] + D2 --> F2[Node 2] + D2 --> G2[Node N] + E2 --> H2[Cache Lookup] + F2 --> I2[Cache Lookup] + G2 --> J2[Cache Lookup] + end +``` + +#### New Components + +| Component | Description | +|-----------|-------------| +| `UserFactory` | Factory for creating and deserializing User objects | +| `UserFactory.Caching` | Caching implementation that maps serialized data to User objects | +| `UserFactory.Simple` | Non-caching implementation for testing | +| `AuthenticationContext` | Context object for passing data between auth backends | +| `ImpersonationBackend` | New interface for user impersonation support | + +#### New Configuration + +| Setting | Description | Default | +|---------|-------------|---------| +| `plugins.security.user_cache.max_size` | Maximum number of cached User objects | 10000 | +| `plugins.security.user_cache.expire_after_access` | Cache entry expiration time | 1h | + +#### API Changes + +The `User` class now uses immutable builder-style methods: + +```java +// Old mutable API (deprecated) +User user = new User("username"); +user.addRoles(roles); +user.addAttributes(attributes); + +// New immutable API +User user = new User("username") + .withRoles(roles) + .withAttributes(attributes) + .withSecurityRoles(securityRoles); +``` + +The `AuthenticationBackend` interface signature changed: + +```java +// Old signature +User authenticate(AuthCredentials credentials); +boolean exists(User user); + +// New signature +User authenticate(AuthenticationContext context); +``` + +The `AuthorizationBackend` interface now returns a new User object: + +```java +// Old signature +void fillRoles(User user, AuthCredentials credentials); + +// New signature +User addRoles(User user, AuthenticationContext context); +``` + +### Performance Benchmarks + +Based on benchmarks from the PR, users with many attributes see significant improvements: + +| User Attributes | Throughput Improvement | +|-----------------|------------------------| +| 10 attributes | 21% | +| 100 attributes | 39% | + +### Migration Notes + +- Custom `AuthenticationBackend` implementations must be updated to use the new `AuthenticationContext` parameter +- Custom `AuthorizationBackend` implementations must return a new User object instead of modifying the input +- The `LdapUser` class is now only used for deserialization compatibility; new code should use the standard `User` class + +## Limitations + +- The caching mechanism adds memory overhead proportional to the number of unique users +- Custom authentication backends require code changes to support the new API + +## Related PRs + +| PR | Description | +|----|-------------| +| [#5339](https://github.com/opensearch-project/security/pull/5339) | Remove support for unused custom User serialization | +| [#5212](https://github.com/opensearch-project/security/pull/5212) | Immutable user object | + +## References + +- [Issue #5168](https://github.com/opensearch-project/security/issues/5168): Make User object immutable +- [Issue #5200](https://github.com/opensearch-project/security/issues/5200): When to retire serialization backward compat code +- [Blog: Performance optimizations for the OpenSearch security layer](https://opensearch.org/blog/performance-optimizations-for-the-opensearch-security-layer/) + +## Related Feature Report + +- [Full feature documentation](../../../../features/security/security-performance-improvements.md) diff --git a/docs/releases/v3.1.0/index.md b/docs/releases/v3.1.0/index.md index 227574c3a..182752738 100644 --- a/docs/releases/v3.1.0/index.md +++ b/docs/releases/v3.1.0/index.md @@ -51,6 +51,7 @@ - [Security Backend Bug Fixes](features/security/security-backend-bug-fixes.md) - Stale cache post snapshot restore, compliance audit log diff, DLS/FLS filter reader, auth header logging, password reset UI, forecasting permissions - [Security Debugging](features/security/security-debugging.md) - Enhanced error messages for "Security not initialized" with cluster manager status - [Security Dependency Updates](features/security/security-dependency-updates.md) - 24 dependency updates including Bouncy Castle 1.81, Kafka 4.0.0, and CVE-2024-52798 fix +- [Security Performance Improvements](features/security/security-performance-improvements.md) - Immutable User object with serialization caching for reduced inter-node communication overhead - [Security Role Mapping](features/security/security-role-mapping.md) - Fix mapped roles not included in ThreadContext userInfo after immutable User object change - [Security Permissions](features/security/security-permissions.md) - Add forecast roles and fix missing cluster:monitor and mapping get permissions