azure-servicebus.AutoLockRenewer change ThreadPool usage

[m-credera]

Is your feature request related to a problem? Please describe.
The azure.servicebus.AutoLockRenewer is implicitly limited to renewing locks for at most max_workers number of messages. For an application that takes many messages off of the queue for parallel processing, this can present problems.

The azure.servicebus.AutoLockRenewer uses a single future inside of a ThreadPoolExecutor to monitor messages and renew locks. This means it's limited to monitoring at most max_workers messages at a time (since a thread pool only runs one future per-thread until the future completes).

Describe the solution you'd like
An alternative implementation could instead use max_workers futures monitoring a set of messages per future rather than a single message per future.

Describe alternatives you've considered
An alternative solution would be to implement parallelism with each worker taking a message off of the queue for itself to process. This is certainly possible, but there are still cases where taking more than max_workers messages off of the queue could be desirable (e.g. prefetching messages).

[yunhaoling]

hey @michaelcredera , thanks for your feedback, I'll take a look into it.

--

You understanding is correct. The current implementation indeed has the implicit limitation -- one message per worker. This should be addressed.

An alternative implementation could instead use max_workers futures monitoring a set of messages per future rather than a single message per future.

thanks for the suggestions, they both sound like good plan. I'll allocate some time thinking about them, will get back to you after I figure it out.

[yunhaoling]

hey @michaelcredera , sorry we have limited resourcing currently. would you like to try making a PR for the change? I would be glad to review it😊 (It's completely fine if you don't have the time, just FYI we're an open-source library and we welcome user contributions)

[m-credera]

Hey @yunhaoling - I can try to get a PR up this week

[m-credera]

@yunhaoling Are there branching permissions on the repo? I didn't see anything in the contribution guide, but I'm getting a 403 error pushing my branch:

git push --set-upstream origin feature/19362-threadpool-improvements-for-servicebus
...
fatal: unable to access 'https://github.com/Azure/azure-sdk-for-python.git/': The requested URL returned error: 403

[yunhaoling]

hey @michaelcredera , creating a branch directly in the azure-sdk-for-python repo is not allowed for external contributors.
instead, you could fork the azure-sdk-for-python repo and create a PR from your repo branch.