Unable to parse Beats payload

[Elastic-James]

As a result of upgrading Logstash from 8.9.0 to 8.10.4 the below error was continually observed in Logstash's logs, and which caused partial pipeline flow issues. We have had to roll back to Logstash 8.9.x

“BeatsHandler - Handling exception: java.land.RuntimeException: Unable to parse beats payload (caused by: com.fasterxml.jackson.core.exc.StreamContraintException: String length (20,050,912 exceed the maximum length (20,000,000))."

After doing some digging the implication here is the DEFAULT_MAX_STRING_LEN is no longer being set and thus picking up the default of 20MB

DEFAULT_MAX_STRING_LEN
public static final int DEFAULT_MAX_STRING_LEN
Default setting for maximum string length: see StreamReadConstraints.Builder.maxStringLength(int) for details.

See:
https://docshoster.org/p/com.fasterxml.jackson/jackson-core/latest/com/fasterxml/jackson/core/exc/StreamConstraintsException.html

https://docshoster.org/p/com.fasterxml.jackson/jackson-core/latest/com/fasterxml/jackson/core/StreamReadConstraints.html#DEFAULT_MAX_STRING_LEN

[jsvd]

Users hitting this can upgrade to the newly released 8.11.1 https://www.elastic.co/guide/en/logstash/current/logstash-8-11-1.html