Bump SAML libs by willyborankin · Pull Request #2927 · opensearch-project/security

willyborankin · 2023-07-02T13:51:23Z

Description

Bump SAML libs

OpenSAML to version 4.3.0
One login SAML to 2.9.0

Issues Resolved

Testing

[Please provide details of testing done: unit testing, integration testing and manual testing]

Check List

New functionality includes testing
New functionality has been documented
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov · 2023-07-02T13:54:05Z

Codecov Report

Merging #2927 (5c12d0a) into main (4eef662) will decrease coverage by 0.06%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #2927      +/-   ##
============================================
- Coverage     62.31%   62.25%   -0.06%     
+ Complexity     3337     3332       -5     
============================================
  Files           266      266              
  Lines         19650    19648       -2     
  Branches       3329     3329              
============================================
- Hits          12244    12231      -13     
- Misses         5779     5786       +7     
- Partials       1627     1631       +4

Impacted Files	Coverage Δ
...dlic/auth/http/saml/AuthTokenProcessorHandler.java	`46.48% <100.00%> (-0.58%)`	⬇️
...zon/dlic/auth/http/saml/Saml2SettingsProvider.java	`60.18% <100.00%> (ø)`
.../dlic/auth/http/saml/SamlHTTPMetadataResolver.java	`62.96% <100.00%> (ø)`

... and 5 files with indirect coverage changes

stephen-crawford

Thanks for this @willyborankin, looks great!

cwperks · 2023-07-03T15:27:27Z

src/main/java/com/amazon/dlic/auth/http/saml/SamlHTTPMetadataResolver.java

@@ -15,6 +15,7 @@
 import java.security.AccessController;


Back when I migrated the main branch from Apache HttpClient 4 -> Apache HttpClient 5, I needed to retain the dependency on HttpClient 4 because of this class. Is there any version of OpenSAML available with Apache HttpClient 5 so that we can remove this last remaining dependeny on HttpClient 4?

This PR looks good to me.

Ahhh I will take a look for sure. Beside found out that Shibboleth has its own repository for OpenSAML and the latest version is 4.3.0. I will push changes

So 4.3.0 uses HTTP commons version 4. to exclude it completely I could try to prepare PR to the Shibboleth git repo (not sure how it works) or just implement our own soultion with HTTP commons 5 instead of 4. AFAU we need to extend org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver class for that. Wdyt?

I pushed 4.3.0 version

@cwperks added issue #2932

Thank you for filing an issue to track this @willyborankin! From the Shibboleth git, it looks like the HttpMetadataResolver class is using HttpClient 5 on their main branch: https://git.shibboleth.net/view/?p=java-opensaml.git;a=blob;f=opensaml-saml-impl/src/main/java/org/opensaml/saml/metadata/resolver/impl/HTTPMetadataResolver.java;hb=0d8f395fcca7923cb4d4cc5a98730b5f3fca3aa9

I'm looking to see if there is a release schedule to see when its expected that a published version of the jar will have the update.

4.3.0 is the way to go. 4.0.1 was published February 11, 2021 and 4.3.0 on January 17, 2023. It looks like they are only publishing new artifacts to the shibboleth repo.

@cwperks OK found https://shibboleth.atlassian.net/browse/JSSH-16 they are going to release it in version 9.0.0 of net.shibboleth.utilities:java-support. AFAIU this is part of IDP v5 release

- OpenSAML to version 4.3.0 - One login SAML tp 2.9.0 Signed-off-by: Andrey Pleskach <ples@aiven.io>

opensearch-trigger-bot · 2023-07-07T14:55:51Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2927-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 092e8f53e641fd09c1a7689edaaad1dadf5b1282
# Push it to GitHub
git push --set-upstream origin backport/backport-2927-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2927-to-2.x.

cwperks · 2023-07-07T15:04:53Z

The automatic backport failed :/. @willyborankin Would you be able to create a manual backport for this change? Let me know if you'd like any help with the creation of a manual backport.

cwperks · 2023-07-11T15:18:31Z

@willyborankin I think this PR is causing errors in the tests for the security-dashboards-plugin here: https://github.com/opensearch-project/security-dashboards-plugin/actions/runs/5510435982/jobs/10068087285?pr=1482

In testing locally, I am getting the following errors:

[2023-07-11T10:59:03,088][WARN ][o.o.s.s.ReflectionHelper ] [smoketestnode] Unable to enable 'com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator' due to java.lang.reflect.InvocationTargetException
[2023-07-11T10:59:03,089][ERROR][o.o.s.s.DynamicConfigModelV7] [smoketestnode] Unable to initialize auth domain saml_auth_domain=AuthcDomain [http_enabled=true, transport_enabled=false, order=5, http_authenticator=HttpAuthenticator [challenge=true, type=saml, config={idp={metadata_url=http://localhost:7000/metadata, entity_id=urn:example:idp}, sp={entity_id=https://localhost:9200/}, kibana_url=http://localhost:5601/, exchange_key=6aff3042-1327-4f3d-82f0-40a157ac4464}], authentication_backend=AuthcBackend [type=noop, config={}], description=null] due to OpenSearchException[java.lang.reflect.InvocationTargetException]; nested: InvocationTargetException; nested: RuntimeException[java.security.AccessControlException: access denied ("java.util.PropertyPermission" "*" "read,write")]; nested: AccessControlException[access denied ("java.util.PropertyPermission" "*" "read,write")];
org.opensearch.OpenSearchException: java.lang.reflect.InvocationTargetException

but after I wrap the call to ReflectionHelper.instantiateAAA with AccessController.doPrivileged I end up getting a ClassNotFoundException:

Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_2: Error occurred while attempting to refresh metadata from 'http://localhost:7000/metadata'
java.lang.NoClassDefFoundError: Could not initialize class org.opensaml.xmlsec.signature.impl.X509CertificateImpl

willyborankin · 2023-07-11T15:24:50Z

@willyborankin I think this PR is causing errors in the tests for the security-dashboards-plugin here: https://github.com/opensearch-project/security-dashboards-plugin/actions/runs/5510435982/jobs/10068087285?pr=1482

In testing locally, I am getting the following errors:

[2023-07-11T10:59:03,088][WARN ][o.o.s.s.ReflectionHelper ] [smoketestnode] Unable to enable 'com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator' due to java.lang.reflect.InvocationTargetException
[2023-07-11T10:59:03,089][ERROR][o.o.s.s.DynamicConfigModelV7] [smoketestnode] Unable to initialize auth domain saml_auth_domain=AuthcDomain [http_enabled=true, transport_enabled=false, order=5, http_authenticator=HttpAuthenticator [challenge=true, type=saml, config={idp={metadata_url=http://localhost:7000/metadata, entity_id=urn:example:idp}, sp={entity_id=https://localhost:9200/}, kibana_url=http://localhost:5601/, exchange_key=6aff3042-1327-4f3d-82f0-40a157ac4464}], authentication_backend=AuthcBackend [type=noop, config={}], description=null] due to OpenSearchException[java.lang.reflect.InvocationTargetException]; nested: InvocationTargetException; nested: RuntimeException[java.security.AccessControlException: access denied ("java.util.PropertyPermission" "*" "read,write")]; nested: AccessControlException[access denied ("java.util.PropertyPermission" "*" "read,write")];
org.opensearch.OpenSearchException: java.lang.reflect.InvocationTargetException

but after I wrap the call to ReflectionHelper.instantiateAAA with AccessController.doPrivileged I end up getting a ClassNotFoundException:

Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_2: Error occurred while attempting to refresh metadata from 'http://localhost:7000/metadata'
java.lang.NoClassDefFoundError: Could not initialize class org.opensaml.xmlsec.signature.impl.X509CertificateImpl

@cwperks Ok I will investigate. Is it part of 2.9?

willyborankin · 2023-07-11T15:25:35Z

Ahhh it is simple fix

cwperks · 2023-07-11T15:27:15Z

@willyborankin Yes, it is blocking PRs in the security-dashboards-plugin repo that ideally go into 2.9.

- OpenSAML to version 4.3.0 - One login SAML tp 2.9.0 Signed-off-by: Andrey Pleskach <ples@aiven.io>

This change combines the many updates from the following commits: * 5f62e8a dependabot: bump commons-io:commons-io from 2.11.0 to 2.13.0 (opensearch-project#3074) * 2f69a10 bump com.github.wnameless.json:json-base from 2.4.0 to 2.4.1 (opensearch-project#3062) * c0e50da dependabot: bump org.cryptacular:cryptacular from 1.2.4 to 1.2.5 (opensearch-project#3071) * d3488e8 dependabot: bump kafka_version from 3.5.0 to 3.5.1 (opensearch-project#3041) * ab6778d Update ospackage, checker-qual, zcxvbn and error_prone_annotations, camel-xmlsecurity (opensearch-project#3023) * 0e6608d Bump JSON libs (opensearch-project#2926) * df07bea SAML 4.3.0 addition persmission (opensearch-project#2987) * e5348eb Change maven repo location for compatibility check (opensearch-project#2980) * 4a1ec53 Bump jaxb to 2.3.8 (opensearch-project#2977) * 9599155 Bump guava to 32.1.1-jre (opensearch-project#2976) * 06eed60 dependabot: bump org.glassfish.jaxb:jaxb-runtime from 2.3.4 to 4.0.3 (opensearch-project#2970) * 1113244 Bump eventbus to 3.3.1 (opensearch-project#2965) * 99ff7b3 dependabot: bump org.apache.bcel:bcel from 6.6.0 to 6.7.0 (opensearch-project#2969) * 0794c3f dependabot: bump jakarta.xml.bind:jakarta.xml.bind-api (opensearch-project#2968) * 9e6aab3 dependabot: bump com.google.j2objc:j2objc-annotations from 1.3 to 2.8 (opensearch-project#2963) * 8227f64 dependabot: bump com.sun.istack:istack-commons-runtime (opensearch-project#2960) * 8e044a6 dependabot: bump org.apiguardian:apiguardian-api from 1.0.0 to 1.1.2 (opensearch-project#2964) * 49cbf52 Remove commons-collections 3.2.2 (opensearch-project#2924) * 092e8f5 Bump SAML libs (opensearch-project#2927) * 8ab7cb4 Resolve CVE-2023-2976 by forcing use of Guava 32.0.1 (opensearch-project#2937) * 4eef662 Clean up and bump Apache libs (opensearch-project#2925) * 9a72355 Bump BouncyCastle from jdk15on to jdk15to18 (opensearch-project#2901) * e4f4817 [Enhancement] Parallel test jobs for CI (opensearch-project#2861) * d871af3 Update snappy to 1.1.10.1 and guava to 32.0.1-jre (opensearch-project#2886) * c808692 Format everything (opensearch-project#2866) Signed-off-by: Peter Nied <petern@amazon.com>

- OpenSAML to version 4.3.0 - One login SAML tp 2.9.0 Signed-off-by: Andrey Pleskach <ples@aiven.io>

Manually backported PRs: - opensearch-project#2987 - opensearch-project#2927 Signed-off-by: Andrey Pleskach <ples@aiven.io>

Backported PRs: - #2987 - #2927 - #3651 - #3690 into 2.x --------- Signed-off-by: Andrey Pleskach <ples@aiven.io>

willyborankin force-pushed the bump-saml-libs branch from 8c326e8 to 9cadd9e Compare July 2, 2023 14:32

willyborankin changed the title ~~Bump SAML libs~~ Bump SAML and LDAP libs Jul 2, 2023

willyborankin force-pushed the bump-saml-libs branch from 9cadd9e to db48f93 Compare July 2, 2023 15:12

willyborankin changed the title ~~Bump SAML and LDAP libs~~ Bump SAML libs Jul 2, 2023

willyborankin marked this pull request as ready for review July 2, 2023 16:21

willyborankin requested review from DarshitChanpura, RyanL1997, cliu123, cwperks, davidlago, peternied, reta and stephen-crawford as code owners July 2, 2023 16:21

stephen-crawford previously approved these changes Jul 3, 2023

View reviewed changes

cwperks reviewed Jul 3, 2023

View reviewed changes

stephen-crawford dismissed their stale review via ec7be81 July 3, 2023 16:55

willyborankin force-pushed the bump-saml-libs branch from ec7be81 to 8bfc827 Compare July 4, 2023 17:03

Bump SAML libs

5c12d0a

- OpenSAML to version 4.3.0 - One login SAML tp 2.9.0 Signed-off-by: Andrey Pleskach <ples@aiven.io>

willyborankin force-pushed the bump-saml-libs branch from 8bfc827 to 5c12d0a Compare July 4, 2023 17:27

willyborankin mentioned this pull request Jul 4, 2023

[Maintenance] Review use of SamlHTTPMetadataResolver #2932

Open

stephen-crawford approved these changes Jul 5, 2023

View reviewed changes

cwperks approved these changes Jul 5, 2023

View reviewed changes

cwperks added the backport 2.x label Jul 7, 2023

cwperks merged commit 092e8f5 into opensearch-project:main Jul 7, 2023

cwperks mentioned this pull request Jul 11, 2023

[BUG] SAML integration test has trouble for locating sign-in button opensearch-project/security-dashboards-plugin#1506

Closed

peternied pushed a commit to peternied/security that referenced this pull request Aug 2, 2023

Bump SAML libs (opensearch-project#2927)

56d8fbb

- OpenSAML to version 4.3.0 - One login SAML tp 2.9.0 Signed-off-by: Andrey Pleskach <ples@aiven.io>

peternied pushed a commit to peternied/security that referenced this pull request Aug 3, 2023

[Backport 2.x] Bump SAML libs (opensearch-project#2927)

8ba90ed

- OpenSAML to version 4.3.0 - One login SAML tp 2.9.0 Signed-off-by: Andrey Pleskach <ples@aiven.io>

willyborankin added a commit to willyborankin/security that referenced this pull request Nov 8, 2023

Bump OpenSAML to 4.3.0

79104bd

Manually backported PRs: - opensearch-project#2987 - opensearch-project#2927 Signed-off-by: Andrey Pleskach <ples@aiven.io>

willyborankin mentioned this pull request Nov 8, 2023

[Backport 2.x] Backport OpenSAML 4.3.0 to 2.x #3671

Merged

willyborankin added a commit to willyborankin/security that referenced this pull request Nov 13, 2023

Bump OpenSAML to 4.3.0

16a7a76

Manually backported PRs: - opensearch-project#2987 - opensearch-project#2927 Signed-off-by: Andrey Pleskach <ples@aiven.io>

willyborankin added a commit to willyborankin/security that referenced this pull request Nov 20, 2023

Bump OpenSAML to 4.3.0

aa07f0e

Manually backported PRs: - opensearch-project#2987 - opensearch-project#2927 Signed-off-by: Andrey Pleskach <ples@aiven.io>

willyborankin added a commit to willyborankin/security that referenced this pull request Nov 30, 2023

Bump OpenSAML to 4.3.0

61ca2ac

Manually backported PRs: - opensearch-project#2987 - opensearch-project#2927 Signed-off-by: Andrey Pleskach <ples@aiven.io>

cwperks pushed a commit that referenced this pull request Dec 6, 2023

[Backport 2.x] Backport OpenSAML 4.3.0 to 2.x (#3671)

d7b10bb

Backported PRs: - #2987 - #2927 - #3651 - #3690 into 2.x --------- Signed-off-by: Andrey Pleskach <ples@aiven.io>

Conversation

willyborankin commented Jul 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Testing

Check List

Uh oh!

codecov bot commented Jul 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

stephen-crawford left a comment

Choose a reason for hiding this comment

Uh oh!

cwperks Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

willyborankin Jul 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willyborankin Jul 4, 2023

Choose a reason for hiding this comment

Uh oh!

willyborankin Jul 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willyborankin Jul 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cwperks Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

cwperks Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

willyborankin Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

opensearch-trigger-bot bot commented Jul 7, 2023

Uh oh!

cwperks commented Jul 7, 2023

Uh oh!

cwperks commented Jul 11, 2023

Uh oh!

willyborankin commented Jul 11, 2023

Uh oh!

willyborankin commented Jul 11, 2023

Uh oh!

cwperks commented Jul 11, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

willyborankin commented Jul 2, 2023 •

edited

Loading

codecov bot commented Jul 2, 2023 •

edited

Loading

willyborankin Jul 3, 2023 •

edited

Loading

willyborankin Jul 4, 2023 •

edited

Loading

willyborankin Jul 4, 2023 •

edited

Loading