Support host based partition-key for Kafka output, Important for ordering.

[ShahSunny]

Support host based partition-key for Kafka output, Important to ensure the ordering of logs per host.

For confirmed bugs, please report:

• Version: 5.0.0 Alpha 4
• Operating System: Linux
• Steps to Reproduce:

[ruflin]

@ShahSunny As far as I understand, that is a feature request for the Kafka output and not a bug?

[urso]

Yes, it's a feature request.

[urso]

What exactly you mean by 'host based partition-key'? You care about the actual partitioning, or do you care about message ordering?

#2284 makes partitioner configurable add support for hash-based partitioning (either using configurable message key or configure list of event fields used to compute hash).

some beats (filebeat and winlogbeat) do require infinite retry. Due to limitations in current architecture, if publishing fails too often due to network errors (default set to 1000 to make it somewhat unlikely), message order can not guaranteed anymore. Setting the kafka version to 0.10 when using kafka 10 will add the event timestamp (for filebeat/winlogbeat moment data has been acquired) to published messages.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue doesn't have a Team:<team> label.