Add template_id sort to patterned_text track#825
Conversation
|
@kkrik-es I need to do some systematic testing of different ways to sort, but this seems like a good first bet. |
| {% endif %} | ||
| {% if patterned_text_message_field | default(false) is true %} | ||
| "sort": { | ||
| "field": [ "host.name", "message.template_id", "@timestamp" ], |
There was a problem hiding this comment.
I wonder if we should have template_id first. It's fine to submit it as is, but you can also test with template_id first and see if it's better.
|
I ran elastic/logs with the sort config added in this PR (host.name, message.template_id, timestamp), as well as with "message.template_id,host.name,timestamp". Here are the results: I was expecting the first config to get better compression, but this was not the case. Each of the message fields do get slightly higher compression, but this was not enough to make up for other fields. For example, There are some good gains to be made by improving how |
Use the mapping parameter added in elastic/elasticsearch#136571 to sort on message.template_id instead of manually specifying with index.sort.fields. Relates to #825.
|
@parkertimmins
Backporting entails:
Thank you! |
No description provided.