new feature

[vtolstov]

what you think about semi-autocommit feature?
you already have CommitRecords func, but if i want to specify autocommit interval and CommitRecords not commit passed records immediately but in provided interval. So i can call CommitRecords with 1,2,3,5 records that not commited right now, but accumulated. For example i'm specify autocommit interval 5s, in this case i can call CommitRecords multiple times with one or many records and after interval happens - client send commit offsets request to kafka with accumulated offsets.
What you think ?

[twmb]

I think this is a good idea, but I don't want to merge the behavior with CommitRecords, because merging it would mean no synchronous usage CommitRecords.

I think this can be done with a new option and a new client method:

func AutoCommitMarks() GroupOpt

func (cl *Client) MarkCommitRecords(...*Record)

I'm not yet 100% settled on the names, but I'm leaning heavily towards what's proposed above. I'll commit this probably tomorrow, as a part of v0.11.0, which will come with a few API breakages (detailed in #62).

[vtolstov]

nice, will look at it after merging

[twmb]

Is the goal here to also opt out of the current behavior that allows any prior polled records to be committed during the next poll?

Right now if you poll offsets 0 through 5, the next poll will allow 5 to be committed. I can't decide whether to opt out of this such that you must MarkCommitRecords, or whether to keep this behavior so that you only need to use MarkCommitRecords if you want to have records available for commit while processing a batch slowly.

[twmb]

I'm leaning towards making MarkCommitRecords required, I think that feels more understandable and gives more control.

[twmb]

This has been implemented in 31ed46b

[twmb]

And tagged in v0.10.3