feat: Add AWS STS Session Tags support for credential vending

[obelix74]

Fixes #3325

Summary

Adds support for AWS STS Session Tags when vending S3 credentials, enabling correlation between Polaris catalog operations and S3 access in AWS CloudTrail. This feature is controlled by a new feature flag INCLUDE_SESSION_TAGS_IN_SUBSCOPED_CREDENTIAL (default: false).

Motivation

When Polaris vends temporary AWS credentials via STS AssumeRole, there's no way to correlate S3 access events in CloudTrail back to the originating Polaris catalog operation. This makes audit trails incomplete and security investigations difficult.

AWS STS Session Tags solve this by attaching metadata to the assumed role session that appears in CloudTrail events for all subsequent API calls made with those credentials.

Changes

New Components

• CredentialVendingContext - Immutable value class holding session tag context (catalog name, namespace, table name, request-id)
• INCLUDE_SESSION_TAGS_IN_SUBSCOPED_CREDENTIAL - Feature flag in FeatureConfiguration (default: false)

AWS Integration

• AwsCredentialsStorageIntegration - Adds 5 session tags to AssumeRoleRequest when feature enabled:
  • polaris:principal - The authenticated principal name
  • polaris:catalog - The catalog being accessed
  • polaris:namespace - The namespace being accessed
  • polaris:table - The table being accessed
  • polaris:request-id - The originating request ID for correlation
• Tags are marked as transitive for role chaining support
• Tag values truncated to 256 characters (AWS limit)
• Missing values use "unknown" placeholder for consistent CloudTrail presence

Cache Key Updates

• StorageCredentialCacheKey - Now includes CredentialVendingContext when session tags enabled
• StorageCredentialCache - Includes principal in cache key when session tags OR principal-name-in-session features are
  enabled (prevents credential cross-contamination)
• Added toSanitizedLogString() for safe debug logging without exposing sensitive context

Interface Updates

• Updated PolarisStorageIntegration, PolarisCredentialVendor, StorageCredentialsVendor to accept
  CredentialVendingContext
• Updated Azure and GCP integrations for signature compatibility (pass-through only)

Testing

• 6 new unit tests in AwsCredentialsStorageIntegrationTest:
  • testSessionTagsIncludedWhenFeatureEnabled
  • testSessionTagsNotIncludedWhenFeatureDisabled
  • testSessionTagsWithPartialContext
  • testSessionTagsWithLongValues
  • testSessionTagsWithEmptyContext
  • testSessionTagsAccessDeniedGracefulHandling
• All existing tests updated for signature compatibility
• Full test suites pass: polaris-core:test, polaris-runtime-service:test

Configuration

To enable session tags, set in your Polaris configuration:

   polaris:
     features:
       INCLUDE_SESSION_TAGS_IN_SUBSCOPED_CREDENTIAL: "true"

IAM Requirements

When enabled, the IAM role's trust policy must allow sts:TagSession:

   {
     "Effect": "Allow",
     "Principal": { "AWS": "arn:aws:iam::ACCOUNT:root" },
     "Action": ["sts:AssumeRole", "sts:TagSession"]
   }

Breaking Changes

None. Feature is off by default and all interface changes are backward compatible.

Checklist

• 🛡️ Don't disclose security issues! (contact security@apache.org)
• 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes [Feature] Add AWS STS Session Tags to Credential Vending for CloudTrail Correlation #3325
• 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
• 💡 Added comments for complex logic
• 🧾 Updated CHANGELOG.md (if needed)
• 📚 Updated documentation in site/content/in-dev/unreleased (if needed)

[singhpk234]

This is a cool feature ! looking forward to the change :) ~

[dimas-b]

Thanks for your contribution, @obelix74 !

The PR LGTM overall, my only real concern is relying on REST contexts.

Since this affects Polaris core, could you also send a discussion email to the dev ML for awareness?

[obelix74]

Thanks for your contribution, @obelix74 !

The PR LGTM overall, my only real concern is relying on REST contexts.

Since this affects Polaris core, could you also send a discussion email to the dev ML for awareness?

Thanks. Sent an email to the ML. I addressed most of your comments. I have a question on one of them. Thanks again.

[singhpk234]

Thanks for the change @obelix74 !

took an initial passs, i do agree with @dimas-b on sending rest context as part of tag, can you explain how do you wanna use this ?

[obelix74]

Thanks for the change @obelix74 !

took an initial passs, i do agree with @dimas-b on sending rest context as part of tag, can you explain how do you wanna use this ?

I removed using RestContext and used SLF4F#MDC instead to get the request id in a commit that I pushed about 30 mins ago. I am proposing using the request id (trace_id in the below picture) to tie in the request end to end from the client, all the way to the storage layer. If the clients to the catalog are willing to share information about the SQLs executed, we can use the request id to tie it all together.

Do I make sense?

[obelix74]

@singhpk234 as discussed, removed request_id from this PR.

[dimas-b]

Let's wait a few days for other reviewers to have a chance at commenting since it's a holiday season 😉

[adnanhemani]

LGTM, I think this will be an exciting feature that many will use!

[singhpk234]

LGTM, thanks @obelix74 for the all work !

[obelix74]

I see that @jbonofre wants to review. I will wait for them.

[dimas-b]

@jbonofre : I see you added yourself as a reviewer... Do you need more time or are you ok with merging?

[jbonofre]

I did the review and it looks good to me ! Thanks !