Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions sdk/storage/Azure.Storage.Blobs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,76 @@ Get started with our [Blob samples][samples]:
1. [Hello World](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/samples/Sample01a_HelloWorld.cs): Upload, download, and list blobs (or [asynchronously](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/samples/Sample01b_HelloWorldAsync.cs))
2. [Auth](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/samples/Sample02_Auth.cs): Authenticate with connection strings, public access, shared keys, shared access signatures, and Azure Active Directory.

## Advanced Scenarios using Azure.DataMovement.Blobs

For more advanced scenarios like transferring blob virtual directories, we recommend looking into our [Azure.Storage.DataMovement](https://www.nuget.org/packages/Azure.Storage.DataMovement) and [Azure.Storage.DataMovement.Blob](https://www.nuget.org/packages/Azure.Storage.DataMovement.Blobs) packages. Get started with our [DataMovement Blob Samples](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement.Blobs/samples/Sample01b_HelloWorldAsync.cs).

Upload a local directory to the root of the `BlobContainerClient`.
```C# Snippet:ExtensionMethodSimpleUploadToRoot
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath);

await transfer.WaitForCompletionAsync();
```

Upload a local directory to a virtual blob directory in the `BlobContainerClient` by specifying a directory prefix
```C# Snippet:ExtensionMethodSimpleUploadToDirectoryPrefix
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, blobDirectoryPrefix);

await transfer.WaitForCompletionAsync();
```

Upload a local directory to a virtual blob directory in the `BlobContainerClient` specifying more advanced options
```C# Snippet:ExtensionMethodSimpleUploadWithOptions
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
{
BlobContainerOptions = new BlobStorageResourceContainerOptions
{
BlobDirectoryPrefix = blobDirectoryPrefix
},
TransferOptions = new DataTransferOptions()
{
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
}
};

DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, options);

await transfer.WaitForCompletionAsync();
```

Download the entire `BlobContainerClient` to a local directory
```C# Snippet:ExtensionMethodSimpleDownloadContainer
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath);

await transfer.WaitForCompletionAsync();
```

Download a virtual blob directory in the `BlobContainerClient` by specifying a directory prefix
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectory
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, blobDirectoryPrefix);

await transfer.WaitForCompletionAsync();
```

Download from the `BlobContainerClient` specifying more advanced options
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectoryWithOptions
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
{
BlobContainerOptions = new BlobStorageResourceContainerOptions
{
BlobDirectoryPrefix = blobDirectoryPrefix
},
TransferOptions = new DataTransferOptions()
{
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
}
};

DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, options);

await transfer.WaitForCompletionAsync();
```

## Contributing

See the [Storage CONTRIBUTING.md][storage_contrib] for details on building,
Expand Down
172 changes: 88 additions & 84 deletions sdk/storage/Azure.Storage.DataMovement.Blobs/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,16 @@
# Azure Storage Data Movement Blobs client library for .NET

> Server Version: 2020-04-08, 2020-02-10, 2019-12-12, 2019-07-07, and 2020-02-02

## Project Status: Beta

This product is in beta. Some features will be missing or have significant bugs. Please see [Known Issues](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement/KnownIssues.md) for detailed information.

---

Azure Storage is a Microsoft-managed service providing cloud storage that is
highly available, secure, durable, scalable, and redundant. Azure Storage
includes Azure Blobs (objects), Azure Data Lake Storage Gen2, Azure Files,
and Azure Queues.
highly available, secure, durable, scalable, and redundant.

The Azure Storage Data Movement library is optimized for uploading, downloading and
copying customer data.
The Azure Storage Data Movement Blobs library is optimized for uploading, downloading and
copying blobs.

The Azure.Storage.DataMovement.Blobs library provides infrastructure shared by the other
Azure Storage client libraries.
Expand Down Expand Up @@ -72,82 +68,10 @@ We guarantee that all client instance methods are thread-safe and independent of

This section demonstrates usage of Data Movement for interacting with blob storage.

### Extensions on `BlobContainerClient`

For applications with preexisting code using Azure.Storage.Blobs, this package provides extension methods for `BlobContainerClient` to get some of the benefits of the `TransferManager` with minimal extra code.

Instantiate the BlobContainerClient
```C# Snippet:ExtensionMethodCreateContainerClient
BlobServiceClient service = new BlobServiceClient(serviceUri, credential);

BlobContainerClient container = service.GetBlobContainerClient(containerName);
```

Upload a local directory to the root of the container
```C# Snippet:ExtensionMethodSimpleUploadToRoot
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath);

await transfer.WaitForCompletionAsync();
```

Upload a local directory to a virtual directory in the container by specifying a directory prefix
```C# Snippet:ExtensionMethodSimpleUploadToDirectoryPrefix
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, blobDirectoryPrefix);

await transfer.WaitForCompletionAsync();
```

Upload a local directory to a virtual directory in the container specifying more advanced options
```C# Snippet:ExtensionMethodSimpleUploadWithOptions
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
{
BlobContainerOptions = new BlobStorageResourceContainerOptions
{
BlobDirectoryPrefix = blobDirectoryPrefix
},
TransferOptions = new DataTransferOptions()
{
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
}
};

DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, options);
### Using the TransferManager for Blob Transfers

await transfer.WaitForCompletionAsync();
```
The `TransferManager` is the primary class for managing data transfers between storage resources. See [Setup the TransferManager sample](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement#setup-the-transfermanager).

Download the entire container to a local directory
```C# Snippet:ExtensionMethodSimpleDownloadContainer
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath);

await transfer.WaitForCompletionAsync();
```

Download a directory in the container by specifying a directory prefix
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectory
DataTransfer tranfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, blobDirectoryPrefix);

await tranfer.WaitForCompletionAsync();
```

Download from the container specifying more advanced options
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectoryWithOptions
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
{
BlobContainerOptions = new BlobStorageResourceContainerOptions
{
BlobDirectoryPrefix = blobDirectoryPrefix
},
TransferOptions = new DataTransferOptions()
{
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
}
};

DataTransfer tranfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, options);

await tranfer.WaitForCompletionAsync();
```

### Initializing Blob Storage `StorageResource`

Expand Down Expand Up @@ -302,13 +226,92 @@ destinationResource: blobs.FromContainer(
await dataTransfer.WaitForCompletionAsync();
```

### Extensions on `BlobContainerClient`

For applications with preexisting code using Azure.Storage.Blobs, this package provides extension methods for `BlobContainerClient` to get some of the benefits of the `TransferManager` with minimal extra code.

Instantiate the BlobContainerClient
```C# Snippet:ExtensionMethodCreateContainerClient
BlobServiceClient service = new BlobServiceClient(serviceUri, credential);

BlobContainerClient container = service.GetBlobContainerClient(containerName);
```

Upload a local directory to the root of the container
```C# Snippet:ExtensionMethodSimpleUploadToRoot
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath);

await transfer.WaitForCompletionAsync();
```

Upload a local directory to a virtual directory in the container by specifying a directory prefix
```C# Snippet:ExtensionMethodSimpleUploadToDirectoryPrefix
DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, blobDirectoryPrefix);

await transfer.WaitForCompletionAsync();
```

Upload a local directory to a virtual directory in the container specifying more advanced options
```C# Snippet:ExtensionMethodSimpleUploadWithOptions
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
{
BlobContainerOptions = new BlobStorageResourceContainerOptions
{
BlobDirectoryPrefix = blobDirectoryPrefix
},
TransferOptions = new DataTransferOptions()
{
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
}
};

DataTransfer transfer = await container.StartUploadDirectoryAsync(localPath, options);

await transfer.WaitForCompletionAsync();
```

Download the entire container to a local directory
```C# Snippet:ExtensionMethodSimpleDownloadContainer
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath);

await transfer.WaitForCompletionAsync();
```

Download a directory in the container by specifying a directory prefix
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectory
DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, blobDirectoryPrefix);

await transfer.WaitForCompletionAsync();
```

Download from the container specifying more advanced options
```C# Snippet:ExtensionMethodSimpleDownloadContainerDirectoryWithOptions
BlobContainerClientTransferOptions options = new BlobContainerClientTransferOptions
{
BlobContainerOptions = new BlobStorageResourceContainerOptions
{
BlobDirectoryPrefix = blobDirectoryPrefix
},
TransferOptions = new DataTransferOptions()
{
CreationPreference = StorageResourceCreationPreference.OverwriteIfExists,
}
};

DataTransfer transfer = await container.StartDownloadToDirectoryAsync(localDirectoryPath2, options);

await transfer.WaitForCompletionAsync();
```

## Troubleshooting

***TODO***
See [Handling Failed Transfers](#handling-failed-transfers) and [Enabling Logging](https://learn.microsoft.com/en-us/dotnet/azure/sdk/logging) to assist with any troubleshooting.

## Next steps

***TODO***
Get started with our [Share Files Samples][share_samples].

For more base Transfer Manager scenarios see [DataMovement samples][datamovement_base].

## Contributing

Expand Down Expand Up @@ -342,7 +345,8 @@ additional questions or comments.
[azure_sub]: https://azure.microsoft.com/free/dotnet/
[RequestFailedException]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/core/Azure.Core/src/RequestFailedException.cs
[error_codes]: https://learn.microsoft.com/rest/api/storageservices/common-rest-api-error-codes
[samples]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement.Blobs/samples
[datamovement_base]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement
[share_samples]: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.DataMovement.Files.Shares/samples
[storage_contrib]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/CONTRIBUTING.md
[cla]: https://cla.microsoft.com
[coc]: https://opensource.microsoft.com/codeofconduct/
Expand Down
Loading
Loading