Query regarding SQS polling behaviour of Spring Cloud AWS 3.0.X

[bala-sophos]

Hi,

Have a query around the below example scenario from the docs

Example:

Let’s consider a scenario where the container is configured for: maxConcurrentMessages = 20, maxMessagesPerPoll = 10, maxDelayBetweenPolls = 5 seconds, and a pollTimeout = 10 seconds.

The container starts in low throughput mode, meaning it’ll attempt a single poll for 10 messages. If any messages are returned, it’ll switch to high throughput mode, and will make up to 2 simultaneous polls for 10 messages each. If all 20 messages are retrieved, it’ll not attempt any more polls until messages are processed. If after the 5 seconds for maxDelayBetweenPolls 6 messages have been processed, the framework will poll for the 6 messages. If the queue is depleted and a poll returns no messages, it’ll enter low throughput mode again and perform only one poll at a time.

Query:
In the above example,
i) What happens if all the 20 messages polled are processed within 1 second. Will it wait for 5 seconds (which is the value for maxDelayBetweenPolls) , to poll for the next set of messages?
ii) What happens if 19 out of 20 messages polled were processed within 1 second . But 20th message took 3 seconds. Will it wait until the 20 message is processed and only then poll for the next set of messages?

If this is the case, is there any way to not wait for maxDelayBetweenPolls seconds and instead as soon as a worker thread is available we poll for as many messages as the number of worker threads available.

Is there any particular component, for which I can provide my own implementation to achieve this.

[tomazfernandes]

i) What happens if all the 20 messages polled are processed within 1 second. Will it wait for 5 seconds (which is the value for maxDelayBetweenPolls) , to poll for the next set of messages?

No, as soon as 10 permits are available the framework will perform another poll for 10 messages. The maxDelayBetweenPolls configuration is for partial polls - if let's say only 6 permits were returned, the framework would poll for 6 messages after 5 seconds.

ii) What happens if 19 out of 20 messages polled were processed within 1 second . But 20th message took 3 seconds. Will it wait until the 20 message is processed and only then poll for the next set of messages?

In this case it would poll for 10 additional messages in one second, and wait until the 20th message was processed before polling for the second batch of 10 messages. If the 20th took longer to process - let's say 10 seconds - it would poll for 9 messages after 5 seconds.

The idea is waiting for maxDelayBetweenPolls so it always tries to fetch 10 messages per batch, but after that threshold poll for any available permits.

Please let me know if this clarifies the behavior for you.

[bala-sophos]

Thanks @tomazfernandes for the clarification. This was helpful.