Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] pop batch ack for pushConsumer #6841

Closed
f1amingo opened this issue May 31, 2023 · 0 comments · Fixed by #6842
Closed

[Feature] pop batch ack for pushConsumer #6841

f1amingo opened this issue May 31, 2023 · 0 comments · Fixed by #6842

Comments

@f1amingo
Copy link
Contributor

Is Your Feature Request Related to a Problem?

Currently, the POP consumer can pull up to 32 messages in batch, but after consuming one message, it needs to immediately ack once without achieving batching, which is inefficient and consumes bandwidth and CPU resources for both consumers and brokers.
To address this issue, we propose a solution of POP batch ack.

Describe the Solution You'd Like

The client consumption logic remains unchanged. When preparing to send an ACK request after successfully consuming a message, only cache the ACK in memory and the request to Broker is not actually made at this time.

Meanwhile, when the number of cached ACKs exceeds MAX_COUNT_IN_BATCH, or approaches the msg reviveTime, or the memory retention time exceeds MAX_STAY_TIME_MS, a service thread will request a batch ack to the corresponding broker.

[consumerThread1] -----(ackInMem)----------↓
[consumerThread2]---(ackInMem)---> [BatchAckService] ---(batchAckAsync)---> Broker
... ...
[consumerThreadn] -----(ackInMem)---------↑

Describe Alternatives You've Considered

None.

Additional Context

I will submit a pull request later.
The submitted code has been thoroughly tested and has a large number of unit tests.
This pop batch ack implementation can fully support the existing single ack and FIFO message ack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant