Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

polling time wile using consumer iterator interface #1905

Closed
mirko0x5f opened this issue Sep 19, 2019 · 2 comments
Closed

polling time wile using consumer iterator interface #1905

mirko0x5f opened this issue Sep 19, 2019 · 2 comments

Comments

@mirko0x5f
Copy link

Hello,
I'm pretty new to kafka-python, I'm currently using the kafka-python consumer with the iterator interface.
I'm trying to understand if there is a way to change the time between poll() calls while using the iterator interface, or if this time is influenced by any setting in the consumer initialization. Can someone help?
Thank you in advance!

@dpkp
Copy link
Owner

dpkp commented Sep 28, 2019

I am going to land a large change to the iterator interface for the upcoming 1.4.7 release. With that change you can set consumer_timeout_ms to alter the timeout passed to poll() and use max_poll_records to set the maximum number of records to process via iteration between calls to poll(). So, for example, if your iteration process takes 1 second to process a single message (obviously unlikely, but it's a round number) and you set max_poll_records=10 then you should expect that consumer.poll() will be called once every 10 seconds during iteration. Does that make sense? See #1902

@dpkp dpkp closed this as completed Sep 28, 2019
@mirko0x5f
Copy link
Author

That makes sense, thank you for your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants