[Storage] [DataMovement] Change TransferOperation to IDisposable; Remove unnecessary Disposable inheritance on other internal classes

[amnguye]

part 1 to address #51327

In this PR we attempt to alleviate some of the memory leak issues caused by TransferOperation:

• Remove reference to TransferOperation after the Transfer has reached a "Completed" state
• Properly Dispose of the CancellationTokenSource object held by TransferOperation
• Remove need to IDisposable on internal JobPart, TransferJobs, chunk handlers
• Update tests to reflect changes to TransferOperation and other changes above

[github-actions]

API Change Check

APIView identified API level changes in this PR and created the following API reviews

Azure.Storage.DataMovement

[amnguye]

This PR will most likely not address all the memory leak issue, but we are still going to be hunting them down. But to prevent this PR from getting too impossible to review, I won't be making more big changes to resolve the memory leak in this PR. Instead I will make another PR to resolve further memory leak issues I find.

This is definitely a step in the right direction of tackling this issue though.

[Copilot]

Pull Request Overview

This PR addresses memory leak issues in the Azure Storage DataMovement library by making TransferOperation implement IDisposable and removing unnecessary IDisposable inheritance from internal classes. The changes focus on properly disposing CancellationTokenSource objects and removing completed/paused transfers from TransferManager to prevent memory leaks.

Key changes include:

• Made TransferOperation properly dispose its CancellationTokenSource when reaching completed/paused states
• Removed IDisposable inheritance from internal job part and chunk handler classes
• Updated TransferManager to remove completed transfers from its internal collection
• Modified processor interfaces to use simpler completion patterns without IAsyncDisposable

Reviewed Changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
TransferOperation.cs	Added removeTransferDelegate parameter to constructor for cleanup
TransferInternalState.cs	Added logic to dispose CancellationTokenSource and remove transfers on completion
TransferManager.cs	Removed CancellationTokenSource linking and added transfer removal logic
ChannelProcessing.cs	Simplified processor interface by removing IAsyncDisposable and CancellationToken parameters
JobPartInternal.cs	Removed CancellationToken parameter from QueueChunkDelegate
Various test files	Updated tests to use proper CancellationTokenSource disposal patterns

Comments suppressed due to low confidence (1)

sdk/storage/Azure.Storage.DataMovement/tests/TransferOperationTests.cs:1

• Missing space after comma between parameters. Should be 'It.IsAny<TransferInternalState.RemoveTransferDelegate>(), It.IsAny()'.

// Copyright (c) Microsoft Corporation. All rights reserved.