output_kafka2 plugin is missing parameters ?

[simonasr]

Hello,

Readme states there are parameters kafka_agg_max_bytes and kafka_agg_max_messages available in kafka2 output plugin. But this is not true. I found them only in kafka_buffered output plugin source code.
It would be nice to have these parameters in kafka2 output plugin.. 🤞

[repeatedly]

I found them only in kafka_buffered output plugin source code.

Ah, this is README mistake. We will fix soon.

It would be nice to have these parameters in kafka2 output plugin..

These features are mainly for v0.12 fluentd buffer. I think not useful parameters in v1.

[simonasr]

@repeatedly Thanks for replying!

I would say it would be useful - in our environment (~2000records/s) we sometimes get Kafka::MessageSizeTooLarge exception with buffer flush set to 5s. Currently we mitigated this with changing flush time to 1s. But it would be better if there would be ability to set kafka_agg_max_messages and flush messages when certain limit reached.
Or is there better way preventing Kafka::MessageSizeTooLarge exception?

[simonasr]

closing issue. got my answers in #323