Skip to content

Live tests #2812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
weshaggard merged 19 commits into from
Jan 17, 2019
Merged

Live tests #2812

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
d9901ca
Adding Keyvault SDK to CI pipeline - (Includes changes from author: l…
headshot786 Dec 12, 2018
083fef5
live tests for keyvault sdk (includes changes from author: lenala as …
headshot786 Dec 12, 2018
6154137
live test updates
headshot786 Dec 18, 2018
d204813
Updated Test Recordings ran under new live test congiuration.
headshot786 Dec 20, 2018
e767308
Merge remote-tracking branch 'upstream/master' into liveTests
headshot786 Jan 10, 2019
f2655db
Bringing back changes which got overwritten with master branch merge
headshot786 Jan 10, 2019
585e5c2
yml mvn command update.
g2vinay Jan 10, 2019
c3c4e88
Updated Test recordings
headshot786 Jan 10, 2019
e226db2
Updated Secret Operations Test Test recordings
headshot786 Jan 10, 2019
2682bd8
updated yml with variable group
g2vinay Jan 16, 2019
efc825b
reverting changes in the files
g2vinay Jan 16, 2019
e221646
Updated Yml with Maven Task
headshot786 Jan 17, 2019
9022fb9
Updating yml
headshot786 Jan 17, 2019
9faef73
adding variables as maven options
headshot786 Jan 17, 2019
805ed55
changing environment variables as options
headshot786 Jan 17, 2019
fdaa2f2
Update yml with Bash script
headshot786 Jan 17, 2019
0232b36
cleaninig maven properties
g2vinay Jan 17, 2019
e22b3e5
Merge branch 'master' into liveTests
weshaggard Jan 17, 2019
addfe30
Merge branch 'liveTests' of https://github.com/g2vinay/azure-sdk-for-…
g2vinay Jan 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .azure-pipelines/client.test.live.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
variables:
- group: KeyVault Test Secrets
- name: ARM_CLIENTID
value: $(java-keyvault-test-arm-client-id)
- name: ARM_CLIENTKEY
value: $(java-keyvault-test-arm-client-key)

queue:
name: Hosted VS2017
steps:
- bash: |

@weshaggard weshaggard Jan 16, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you aren't using the Maven@3 task to run the tests similar to https://github.com/Azure/azure-sdk-for-java/blob/master/.azure-pipelines/client.test.yml#L17?

In general we should try to do that to help be consistent and also that allows us to capture the test results into the test tab of the build which gives us a good view of the tests than ran and a history of them.

@g2vinay g2vinay Jan 17, 2019 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The environment variables were not read during the build when tried to configure as Maven task.

LOG_PARAMS='-Dorg.slf4j.simpleLogger.defaultLogLevel=error -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn'
mvn test -Dhttp.keepAlive=false -Dsurefire.rerunFailingTestsCount=3 $LOG_PARAMS -f pom.client.build.xml
displayName: 'Run Live Tests'
env:
ARM_CLIENTID: $(ARM_CLIENTID)

@weshaggard weshaggard Jan 8, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these 2 env variables the only difference between live vs not live?

@g2vinay g2vinay Jan 10, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are more Variables defined in the variables tab of dev ops pipeline.

ARM_CLIENTKEY: $(ARM_CLIENTKEY)
AZURE_TEST_MODE: RECORD

- task: PublishTestResults@2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/TEST-*.xml'

7 changes: 1 addition & 6 deletions keyvault/data-plane/azure-keyvault-extensions/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,7 @@
<artifactId>adal4j</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure</artifactId>
<scope>test</scope>
</dependency>


<!-- Other Dependencies -->
<dependency>
<groupId>junit</groupId>
Expand Down
11 changes: 5 additions & 6 deletions .../java/com/microsoft/azure/keyvault/extensions/test/KeyVaultClientIntegrationTestBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import com.microsoft.aad.adal4j.AuthenticationResult;
import com.microsoft.aad.adal4j.ClientCredential;
import com.microsoft.azure.AzureResponseBuilder;
import com.microsoft.azure.credentials.ApplicationTokenCredentials;
import com.microsoft.azure.keyvault.KeyVaultClient;
import com.microsoft.azure.keyvault.authentication.KeyVaultCredentials;
import com.microsoft.azure.keyvault.models.Attributes;
Expand Down Expand Up @@ -78,11 +77,11 @@ public static String getSecondaryVaultUri() {
}

private static String getLiveVaultUri1() {
return getenvOrDefault("keyvault.vaulturi", "https://javasdktestvault.vault.azure.net");
return getenvOrDefault("KEYVAULT_VAULTURI", "https://javasdktestvault.vault.azure.net");

@JonathanGiles JonathanGiles Jan 6, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity - what motivated the changing of all key names to uppercase with underscores rather that periods?

@lenala lenala Jan 8, 2019 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like periods are not processed correctly on linux system, and VSTS converts them to underscore, and converts environment variable names to uppercase anyway. Please see https://developercommunity.visualstudio.com/content/problem/328064/vsts-silently-converts-variable-names-to-uppercase.html for uppercase and https://stackoverflow.com/questions/50020314/escaping-environment-variable-names-in-vsts for underscores.

}

private static String getLiveVaultUri2() {
return getenvOrDefault("keyvault.vaulturi.alt", "https://javasdktestvault2.vault.azure.net");
return getenvOrDefault("KEYVAULT_VAULTURI_ALT", "https://javasdktestvault2.vault.azure.net");
}

private static String getenvOrDefault(String varName, String defValue) {
Expand All @@ -100,13 +99,13 @@ protected static void compareAttributes(Attributes expectedAttributes, Attribute

private static AuthenticationResult getAccessToken(String authorization, String resource) throws Exception {

String clientId = System.getenv("arm.clientid");
String clientId = System.getenv("ARM_CLIENTID");

if (clientId == null) {
throw new Exception("Please inform arm.clientid in the environment settings.");
}

String clientKey = System.getenv("arm.clientkey");
String clientKey = System.getenv("ARM_CLIENTKEY");
String username = System.getenv("arm.username");
String password = System.getenv("arm.password");

Expand Down Expand Up @@ -351,7 +350,7 @@ protected static DeletedKeyBundle pollOnKeyDeletion(String vaultBaseUrl, String

protected static DeletedSecretBundle pollOnSecretDeletion(String vaultBaseUrl, String secretName) throws Exception {
int pendingPollCount = 0;
while (pendingPollCount < 50) {
while (pendingPollCount < 70) {
DeletedSecretBundle deletedSecretBundle = keyVaultClient.getDeletedSecret(vaultBaseUrl, secretName);
if (deletedSecretBundle == null) {
if (isRecordMode()) {
Expand Down
6 changes: 3 additions & 3 deletions .../com/microsoft/azure/keyvault/extensions/test/KeyVaultKeyResolverDefaultProviderTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ public void KeyVault_KeyVaultKeyResolver_Secret128Base64_KeyVaultKeyResolverDefa
byte[] CEK = { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, (byte) 0x88, (byte) 0x99, (byte) 0xAA, (byte) 0xBB, (byte) 0xCC, (byte) 0xDD, (byte) 0xEE, (byte) 0xFF };
byte[] EK = { 0x1F, (byte) 0xA6, (byte) 0x8B, 0x0A, (byte) 0x81, 0x12, (byte) 0xB4, 0x47, (byte) 0xAE, (byte) 0xF3, 0x4B, (byte) 0xD8, (byte) 0xFB, 0x5A, 0x7B, (byte) 0x82, (byte) 0x9D, 0x3E, (byte) 0x86, 0x23, 0x71, (byte) 0xD2, (byte) 0xCF, (byte) 0xE5 };

String TEST_SECRET_NAME = SECRET_NAME + "1";
String TEST_SECRET_NAME = SECRET_NAME + "2";

try {
SetSecretRequest request = new SetSecretRequest.Builder(getVaultUri(), TEST_SECRET_NAME, _base64.encodeAsString(keyBytes)).withContentType("application/octet-stream").build();
Expand Down Expand Up @@ -181,7 +181,7 @@ public void KeyVault_KeyVaultKeyResolver_Secret192Base64_KeyVaultKeyResolverDefa
byte[] CEK = { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, (byte) 0x88, (byte) 0x99, (byte) 0xAA, (byte) 0xBB, (byte) 0xCC, (byte) 0xDD, (byte) 0xEE, (byte) 0xFF };
byte[] EK = { (byte) 0x96, 0x77, (byte) 0x8B, 0x25, (byte) 0xAE, 0x6C, (byte) 0xA4, 0x35, (byte) 0xF9, 0x2B, 0x5B, (byte) 0x97, (byte) 0xC0, 0x50, (byte) 0xAE, (byte) 0xD2, 0x46, (byte) 0x8A, (byte) 0xB8, (byte) 0xA1, 0x7A, (byte) 0xD8, 0x4E, 0x5D };

String TEST_SECRET_NAME = SECRET_NAME + "2";
String TEST_SECRET_NAME = SECRET_NAME + "4";

@JonathanGiles JonathanGiles Jan 6, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It always makes me a little nervous when I see magic numbers being changed in unit tests. Why is this required?

@g2vinay g2vinay Jan 9, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests involve creating and deleting secrets.

Multiple Tests using same magic number for the secret name conflicts when a test is trying to create a secret and the previous Test requested a deletion. And the deletion is still in queue.
To avoid this conflict, changed the magic number for the test.


try {
SetSecretRequest request = new SetSecretRequest.Builder(getVaultUri(), TEST_SECRET_NAME, _base64.encodeAsString(keyBytes)).withContentType("application/octet-stream").build();
Expand Down Expand Up @@ -258,7 +258,7 @@ public void KeyVault_KeyVaultKeyResolver_Secret256Base64_KeyVaultKeyResolverDefa
byte[] CEK = { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, (byte) 0x88, (byte) 0x99, (byte) 0xAA, (byte) 0xBB, (byte) 0xCC, (byte) 0xDD, (byte) 0xEE, (byte) 0xFF };
byte[] EK = { 0x64, (byte) 0xE8, (byte) 0xC3, (byte) 0xF9, (byte) 0xCE, 0x0F, 0x5B, (byte) 0xA2, 0x63, (byte) 0xE9, 0x77, 0x79, 0x05, (byte) 0x81, (byte) 0x8A, 0x2A, (byte) 0x93, (byte) 0xC8, 0x19, 0x1E, 0x7D, 0x6E, (byte) 0x8A, (byte) 0xE7 };

String TEST_SECRET_NAME = SECRET_NAME + "3";
String TEST_SECRET_NAME = SECRET_NAME + "1";

try {
SetSecretRequest request = new SetSecretRequest.Builder(getVaultUri(), TEST_SECRET_NAME, _base64.encodeAsString(keyBytes)).withContentType("application/octet-stream").build();
Expand Down
212 changes: 0 additions & 212 deletions ...ault-extensions/target/test-classes/session-records/KeyVault_KeyVaultKeyResolver_Key.json
View file
Open in desktop

This file was deleted.

Loading