Skip to content

Update blobs migration guide to include GenerateSas; Fix migration test bugs #22380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amnguye merged 5 commits into from
Jul 2, 2021
Merged

Update blobs migration guide to include GenerateSas; Fix migration test bugs #22380

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2e1d4c2
Update migration guide to include GenerateSas; Fix migrationt test bugs
amnguye Jul 1, 2021
847eec6
One more fix
amnguye Jul 1, 2021
51800cd
PR Comments - make generateSas the more preferred pattern to create a…
amnguye Jul 1, 2021
4601e2e
Add snippet for using basic GenerateSas
amnguye Jul 2, 2021
2a9733b
Merge branch 'main' into feature/storage/migration-sas
amnguye Jul 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 30 additions & 10 deletions sdk/storage/Azure.Storage.Blobs/AzureStorageNetMigrationV12.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ v12
The new library only supports constructing a client with a fully constructed SAS URI. Note that since client URIs are immutable once created, a new client instance with a new SAS must be created in order to rotate a SAS.

```C# Snippet:SampleSnippetsBlobMigration_SasUri
BlobClient blob = new BlobClient(new Uri(blobLocationWithSas));
BlobClient blob = new BlobClient(sasUri);
```

#### Connection string
Expand Down Expand Up @@ -446,7 +446,7 @@ v12

v12 has explicit methods for listing by hierarchy.
```C# Snippet:SampleSnippetsBlobMigration_ListHierarchy
IAsyncEnumerable<BlobHierarchyItem> results = containerClient.GetBlobsByHierarchyAsync(prefix: blobPrefix);
IAsyncEnumerable<BlobHierarchyItem> results = containerClient.GetBlobsByHierarchyAsync(prefix: blobPrefix, delimiter: delimiter);
await foreach (BlobHierarchyItem item in results)
{
MyConsumeBlobItemFunc(item);
Expand Down Expand Up @@ -550,15 +550,12 @@ The modern SDK uses a builder pattern for constructing a SAS token. Clients are

```C# Snippet:SampleSnippetsBlobMigration_SasBuilder
// Create BlobSasBuilder and specify parameters
BlobSasBuilder sasBuilder = new BlobSasBuilder()
BlobSasBuilder sasBuilder = new BlobSasBuilder(BlobSasPermissions.Read, DateTimeOffset.UtcNow.AddHours(1))
{
// with no url in a client to read from, container and blob name must be provided if applicable
BlobContainerName = containerName,
BlobName = blobName,
ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
BlobName = blobName
};
// permissions applied separately, using the appropriate enum to the scope of your SAS
sasBuilder.SetPermissions(BlobSasPermissions.Read);

// Create full, self-authenticating URI to the resource
BlobUriBuilder uriBuilder = new BlobUriBuilder(StorageAccountBlobUri)
Expand All @@ -570,14 +567,37 @@ BlobUriBuilder uriBuilder = new BlobUriBuilder(StorageAccountBlobUri)
Uri sasUri = uriBuilder.ToUri();
```

You can also create a SAS from the client object.
Copy link
Contributor

@kasobol-msft kasobol-msft Jul 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we restructure the guide in such a way that GenerateSasUri(BlobSasPermissions, DateTimeOffset) is promoted (i.e. mentioned first) and more advanced cases follow?


```C# Snippet:SampleSnippetsBlobMigration_GenerateSas
// Create a BlobClient with a shared key credential
BlobClient blobClient = new BlobClient(blobUri, sharedKeyCredential);
// Create BlobSasBuilder and specify parameters
BlobSasBuilder sasBuilder = new BlobSasBuilder(BlobSasPermissions.Read, DateTimeOffset.UtcNow.AddHours(1))
{
// Since we are generating from the client, the client will have the container and blob name
// Specify any optional paremeters here
StartsOn = DateTimeOffset.UtcNow.AddHours(-1)
};

Uri sasUri;
// Ensure our client has the credentials required to generate a SAS
if (blobClient.CanGenerateSasUri)
{
// Create full, self-authenticating URI to the resource from the BlobClient
sasUri = blobClient.GenerateSasUri(sasBuilder);

// Use newly made as SAS URI to download the blob
await new BlobClient(sasUri).DownloadToAsync(new MemoryStream());
}
```

If using a stored access policy, construct your `BlobSasBuilder` from the example above as follows:

```C# Snippet:SampleSnippetsBlobMigration_SasBuilderIdentifier
// Create BlobSasBuilder and specify parameters
BlobSasBuilder sasBuilder = new BlobSasBuilder()
BlobSasBuilder sasBuilder = new BlobSasBuilder
{
BlobContainerName = containerName,
BlobName = blobName,
Identifier = "mysignedidentifier"
};
```
Expand Down
163 changes: 106 additions & 57 deletions sdk/storage/Azure.Storage.Blobs/samples/Sample03_Migrations.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,27 +55,14 @@ public void AuthWithSasCredential()
try
{
container.Create();
container.GetBlobClient(blobName).Upload(BinaryData.FromString("hello world"));
BlobClient blobClient = container.GetBlobClient(blobName);
blobClient.Upload(BinaryData.FromString("hello world"));

// build SAS URI for sample
BlobSasBuilder sas = new BlobSasBuilder
{
BlobContainerName = containerName,
BlobName = blobName,
ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
};
sas.SetPermissions(BlobSasPermissions.All);

StorageSharedKeyCredential credential = new StorageSharedKeyCredential(StorageAccountName, StorageAccountKey);

UriBuilder sasUri = new UriBuilder(this.StorageAccountBlobUri);
sasUri.Path = $"{containerName}/{blobName}";
sasUri.Query = sas.ToSasQueryParameters(credential).ToString();

string blobLocationWithSas = sasUri.Uri.ToString();
Uri sasUri = blobClient.GenerateSasUri(BlobSasPermissions.All, DateTimeOffset.UtcNow.AddHours(1));

#region Snippet:SampleSnippetsBlobMigration_SasUri
BlobClient blob = new BlobClient(new Uri(blobLocationWithSas));
BlobClient blob = new BlobClient(sasUri);
#endregion

var stream = new MemoryStream();
Expand Down Expand Up @@ -584,17 +571,18 @@ void MyConsumeBlobItemFunc(BlobHierarchyItem item)

// show in snippet where the prefix goes, but our test doesn't want a prefix for its data set
string blobPrefix = null;
string delimiter = "/";

#region Snippet:SampleSnippetsBlobMigration_ListHierarchy
IAsyncEnumerable<BlobHierarchyItem> results = containerClient.GetBlobsByHierarchyAsync(prefix: blobPrefix);
IAsyncEnumerable<BlobHierarchyItem> results = containerClient.GetBlobsByHierarchyAsync(prefix: blobPrefix, delimiter: delimiter);
await foreach (BlobHierarchyItem item in results)
{
MyConsumeBlobItemFunc(item);
}
#endregion

Assert.IsTrue(expectedBlobNamesResult.SetEquals(downloadedBlobNames));
Assert.IsTrue(new HashSet<string> { virtualDirName }.SetEquals(downloadedPrefixNames));
Assert.IsTrue(new HashSet<string> { virtualDirName + '/' }.SetEquals(downloadedPrefixNames));
}
finally
{
Expand Down Expand Up @@ -700,15 +688,12 @@ public async Task SasBuilder()

#region Snippet:SampleSnippetsBlobMigration_SasBuilder
// Create BlobSasBuilder and specify parameters
BlobSasBuilder sasBuilder = new BlobSasBuilder()
BlobSasBuilder sasBuilder = new BlobSasBuilder(BlobSasPermissions.Read, DateTimeOffset.UtcNow.AddHours(1))
{
// with no url in a client to read from, container and blob name must be provided if applicable
BlobContainerName = containerName,
BlobName = blobName,
ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
BlobName = blobName
};
// permissions applied separately, using the appropriate enum to the scope of your SAS
sasBuilder.SetPermissions(BlobSasPermissions.Read);

// Create full, self-authenticating URI to the resource
BlobUriBuilder uriBuilder = new BlobUriBuilder(StorageAccountBlobUri)
Expand All @@ -729,13 +714,67 @@ public async Task SasBuilder()
}
}

[Test]
public async Task GenerateSas()
{
string accountName = StorageAccountName;
string accountKey = StorageAccountKey;
string containerName = Randomize("sample-container");
string blobName = Randomize("sample-blob");
StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(StorageAccountName, StorageAccountKey);

// setup blob
var container = new BlobContainerClient(ConnectionString, containerName);
BlobUriBuilder uriBuilder = new BlobUriBuilder(container.Uri) { BlobName = blobName };
Uri blobUri = uriBuilder.ToUri();

try
{
await container.CreateAsync();
await container.GetBlobClient(blobName).UploadAsync(BinaryData.FromString("hello world"));

#region Snippet:SampleSnippetsBlobMigration_GenerateSas
// Create a BlobClient with a shared key credential
BlobClient blobClient = new BlobClient(blobUri, sharedKeyCredential);
// Create BlobSasBuilder and specify parameters
BlobSasBuilder sasBuilder = new BlobSasBuilder(BlobSasPermissions.Read, DateTimeOffset.UtcNow.AddHours(1))
{
// Since we are generating from the client, the client will have the container and blob name
// Specify any optional paremeters here
StartsOn = DateTimeOffset.UtcNow.AddHours(-1)
};

Uri sasUri;
// Ensure our client has the credentials required to generate a SAS
if (blobClient.CanGenerateSasUri)
{
// Create full, self-authenticating URI to the resource from the BlobClient
sasUri = blobClient.GenerateSasUri(sasBuilder);

// Use newly made as SAS URI to download the blob
await new BlobClient(sasUri).DownloadToAsync(new MemoryStream());
}
#endregion
else
{
Assert.Fail("Unable to create SAS URI");
}
}
finally
{
await container.DeleteIfExistsAsync();
}
}

[Test]
public async Task SasBuilderIdentifier()
{
string accountName = StorageAccountName;
string accountKey = StorageAccountKey;
string containerName = Randomize("sample-container");
string blobName = Randomize("sample-blob");
DateTimeOffset expiresOn = DateTimeOffset.UtcNow.AddDays(1);
DateTimeOffset startsOn = DateTimeOffset.UtcNow.AddHours(-1);
StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(StorageAccountName, StorageAccountKey);

// setup blob
Expand All @@ -744,6 +783,7 @@ public async Task SasBuilderIdentifier()
try
{
await container.CreateAsync();
BlobClient blobClient = container.GetBlobClient(blobName);
await container.GetBlobClient(blobName).UploadAsync(BinaryData.FromString("hello world"));

// Create one or more stored access policies.
Expand All @@ -754,8 +794,8 @@ public async Task SasBuilderIdentifier()
Id = "mysignedidentifier",
AccessPolicy = new BlobAccessPolicy
{
StartsOn = DateTimeOffset.UtcNow.AddHours(-1),
ExpiresOn = DateTimeOffset.UtcNow.AddDays(1),
StartsOn = startsOn,
ExpiresOn = expiresOn,
Permissions = "rw"
}
}
Expand All @@ -765,22 +805,14 @@ public async Task SasBuilderIdentifier()

#region Snippet:SampleSnippetsBlobMigration_SasBuilderIdentifier
// Create BlobSasBuilder and specify parameters
BlobSasBuilder sasBuilder = new BlobSasBuilder()
BlobSasBuilder sasBuilder = new BlobSasBuilder
{
BlobContainerName = containerName,
BlobName = blobName,
Identifier = "mysignedidentifier"
};
#endregion

// Create full, self-authenticating URI to the resource
BlobUriBuilder uriBuilder = new BlobUriBuilder(StorageAccountBlobUri)
{
BlobContainerName = containerName,
BlobName = blobName,
Sas = sasBuilder.ToSasQueryParameters(sharedKeyCredential)
};
Uri sasUri = uriBuilder.ToUri();
Uri sasUri = blobClient.GenerateSasUri(sasBuilder);

// successful download indicates pass
await new BlobClient(sasUri).DownloadToAsync(new MemoryStream());
Expand Down Expand Up @@ -926,18 +958,26 @@ public async Task RetryPolicy()
string containerName = Randomize("sample-container");
string blobName = Randomize("sample-file");
var containerClient = new BlobContainerClient(ConnectionString, containerName);
await containerClient.GetBlobClient(blobName).UploadAsync(BinaryData.FromString(data));

#region Snippet:SampleSnippetsBlobMigration_RetryPolicy
BlobClientOptions blobClientOptions = new BlobClientOptions();
blobClientOptions.Retry.Mode = RetryMode.Exponential;
blobClientOptions.Retry.Delay = TimeSpan.FromSeconds(10);
blobClientOptions.Retry.MaxRetries = 6;
BlobServiceClient service = new BlobServiceClient(connectionString, blobClientOptions);
BlobClient blobClient = service.GetBlobContainerClient(containerName).GetBlobClient(blobName);
Stream targetStream = new MemoryStream();
await blobClient.DownloadToAsync(targetStream);
#endregion
try
{
await containerClient.CreateIfNotExistsAsync();
await containerClient.GetBlobClient(blobName).UploadAsync(BinaryData.FromString(data));

#region Snippet:SampleSnippetsBlobMigration_RetryPolicy
BlobClientOptions blobClientOptions = new BlobClientOptions();
blobClientOptions.Retry.Mode = RetryMode.Exponential;
blobClientOptions.Retry.Delay = TimeSpan.FromSeconds(10);
blobClientOptions.Retry.MaxRetries = 6;
BlobServiceClient service = new BlobServiceClient(connectionString, blobClientOptions);
BlobClient blobClient = service.GetBlobContainerClient(containerName).GetBlobClient(blobName);
Stream targetStream = new MemoryStream();
await blobClient.DownloadToAsync(targetStream);
#endregion
}
finally
{
await containerClient.DeleteIfExistsAsync();
}

Assert.Pass();
}
Expand All @@ -953,15 +993,24 @@ public async Task MaximumExecutionTime()
string containerName = Randomize("sample-container");
string blobName = Randomize("sample-file");
var containerClient = new BlobContainerClient(ConnectionString, containerName);
await containerClient.GetBlobClient(blobName).UploadAsync(BinaryData.FromString(data));

#region Snippet:SampleSnippetsBlobMigration_MaximumExecutionTime
BlobClient blobClient = containerClient.GetBlobClient(blobName);
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
cancellationTokenSource.CancelAfter(TimeSpan.FromSeconds(30));
Stream targetStream = new MemoryStream();
await blobClient.DownloadToAsync(targetStream, cancellationTokenSource.Token);
#endregion

try
{
await containerClient.CreateIfNotExistsAsync();
await containerClient.GetBlobClient(blobName).UploadAsync(BinaryData.FromString(data));

#region Snippet:SampleSnippetsBlobMigration_MaximumExecutionTime
BlobClient blobClient = containerClient.GetBlobClient(blobName);
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
cancellationTokenSource.CancelAfter(TimeSpan.FromSeconds(30));
Stream targetStream = new MemoryStream();
await blobClient.DownloadToAsync(targetStream, cancellationTokenSource.Token);
#endregion
}
finally
{
await containerClient.DeleteIfExistsAsync();
}

Assert.Pass();
}
Expand Down