Skip to content
This repository has been archived by the owner on Jul 19, 2024. It is now read-only.

Continuously call of TransferManager methods makes not to work blob accesses #554

Open
tsbttk opened this issue Oct 19, 2020 · 0 comments
Open

Comments

@tsbttk
Copy link

tsbttk commented Oct 19, 2020

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the SDK was used?

10.1.0, 10.5.0

What problem was encountered?

In our application, we use azure-storage-blob-10.1.0.jar to access to the Azure blob service.

We use "TransferManager.downloadBlobToFile(downloadFileChannel, blob, null, null).blockingGet()" method
and "TransferManager.uploadFileToBlockBlob(fileChannel, blob, 8 * 1024 * 1024, null).blockingGet()" method.

After we call these methods continuously (for about 13 days or call these methods thousands times), we can not access to the blob service via these methods. After certain moment these methods become to return timeout.

We could get a heapdump while the phenomenon was happening. From the heapdump we notice SharedChannelPool#requests instance that is instance of ConcurrentLinkedDeque has many requests that is not transacted.
we also notice it is because size of SharedChannelPool#leased instance is 64, and SharedChannelPool#poolSize is also 64.
So SharedChannelPool thread stops at L.109

                        while (leased.size() >= poolSize && !closed) {
                            sync.wait();	// L.109
                        }

(We know now 64 is the number of core (2) * 32 from source code of libraries.)

As a trial, while the application is running, we stop calling thse download and upload methods.
After we stop calling these download and upload methods, the size of SharedChannelPool#leased map stays 64.
So we think that something like leak may exist in SharedChannelPool#leased map.

We want to know how we can call these 2 methods continuously.

Have you found a mitigation/solution?

No, not yet.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant