Skip to content

[BACKPORT] Backports Plugin Install Workflow and action for windows#2288

Closed
stephen-crawford wants to merge 18 commits intoopensearch-project:1.3from
stephen-crawford:backportsForWindows
Closed

[BACKPORT] Backports Plugin Install Workflow and action for windows#2288
stephen-crawford wants to merge 18 commits intoopensearch-project:1.3from
stephen-crawford:backportsForWindows

Conversation

@stephen-crawford
Copy link
Contributor

Description

Backports the plugin_install workflow and associated action.

Backports PR #2271

Successfully ran CI and Plugin Install for Linux runner. Windows cannot be tested further until there is a Windows Min. Distribution available.

Check List

  • New functionality includes testing
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
@stephen-crawford stephen-crawford requested a review from a team December 1, 2022 19:00
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
@codecov-commenter
Copy link

codecov-commenter commented Dec 1, 2022

Codecov Report

Merging #2288 (183183c) into 1.3 (7eff736) will increase coverage by 0.14%.
The diff coverage is n/a.

❗ Current head 183183c differs from pull request most recent head f7e840a. Consider uploading reports for the commit f7e840a to get more accurate results

@@             Coverage Diff              @@
##                1.3    #2288      +/-   ##
============================================
+ Coverage     64.59%   64.73%   +0.14%     
- Complexity     3223     3226       +3     
============================================
  Files           247      247              
  Lines         17376    17377       +1     
  Branches       3088     3097       +9     
============================================
+ Hits          11224    11249      +25     
+ Misses         4593     4578      -15     
+ Partials       1559     1550       -9     
Impacted Files Coverage Δ
...a/org/opensearch/security/tools/SecurityAdmin.java 47.59% <0.00%> (+0.06%) ⬆️
...zon/dlic/auth/ldap2/LDAPAuthorizationBackend2.java 28.73% <0.00%> (+0.37%) ⬆️
...ic/auth/ldap/backend/LDAPAuthorizationBackend.java 57.95% <0.00%> (+0.37%) ⬆️
.../org/opensearch/security/auth/BackendRegistry.java 59.37% <0.00%> (+0.52%) ⬆️
...security/configuration/DlsFlsFilterLeafReader.java 62.11% <0.00%> (+0.70%) ⬆️
...earch/security/auditlog/impl/AbstractAuditLog.java 74.93% <0.00%> (+1.58%) ⬆️
...com/amazon/dlic/auth/ldap2/PrivilegedProvider.java 50.81% <0.00%> (+1.63%) ⬆️
.../org/opensearch/security/support/ConfigHelper.java 82.35% <0.00%> (+1.96%) ⬆️
...ava/com/amazon/dlic/auth/ldap2/MakeJava9Happy.java 43.47% <0.00%> (+4.34%) ⬆️
...org/opensearch/security/rest/TenantInfoAction.java 88.23% <0.00%> (+10.29%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@peternied
Copy link
Member

Windows cannot be tested further until there is a Windows Min. Distribution available.

@scrawfor99 Please reach out to the Engineering Effectiveness team to get help finding the min distro

@peternied peternied mentioned this pull request Dec 1, 2022
3 tasks
Copy link
Member

@peternied peternied left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this is still in draft while getting the windows min build, want to make that switch? Note; I think it might be more useful to lump the changes for the plugin install workflow into a single PR that we approve at once rather than one at a time

Comment on lines 36 to 42
ls
cd build
echo at $(pwd)
ls
cd distributions
echo at $(pwd)
ls
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug statements?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops will remove.

peternied and others added 8 commits December 2, 2022 09:38
Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow.  Until retries have been enabled they will automatically pass - but still run and report logs.  As soon as we have full confidence we will allow them to start blocking pull requests from merging.  opensearch-project#2184

Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape.  Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure.  I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11.  opensearch-project#2195

Signed-off-by: Peter Nied <petern@amazon.com>
Adds spotbugs [1] to detect internalization before they are added to the codebase, also fixed several encoding bugs that impact windows users.

[1] https://spotbugs.readthedocs.io/en/stable/index.html

Closes opensearch-project#2194

Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Stephen Crawford <steecraw@amazon.com>
@stephen-crawford
Copy link
Contributor Author

I rebased onto your branch and then PR'd against it. I am reaching out to the Engineering Effectiveness team now.

@stephen-crawford stephen-crawford deleted the backportsForWindows branch April 11, 2023 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants