Skip to content

[BUG] Unable to send data to OpenSearch when batch size is large and http compression is enabled #182

@oeyh

Description

@oeyh

Describe the bug
When batch size is large, data will be sent out in multiple partial bulk requests. If http compression is enabled in this case, we get 500 error when sending data to OpenSearch.

To Reproduce
Steps to reproduce the behavior:

  1. Create a sample log file with 1000 characters per line and 100 lines
  2. Set pipeline.batch.size: 50 in logstash.yml; set target_bulk_bytes => 10240 and http_compression => true for logstash-output-opensearch plugin in logstash.conf. This will make batch byte size larger than target_bulk_bytes so that the plugin will send multiple partial bulk requests.
  3. Start logstash to read the sample log and send logs to an Amazon OpenSearch domain
  4. See errors like this and keep retry:
    [2022-12-22T12:21:32,103][ERROR][logstash.outputs.opensearch][main][67a1dea2c259b674532fcabe49e3d0dd7401d7cab38266ee76e8920ddf9faca3] Encountered a retryable error (will retry with exponential backoff) {:code=>500, :url=>"https://xxx-domain-blnft6zvvl64fq2v43jn4b3hky.us-east-1.es.amazonaws.com:443/_bulk", :content_length=>3412, :body=>""}
  5. See server side error: java.io.EOFException: Unexpected end of ZLIB input stream

Expected behavior
Be able to send data to OpenSearch even when batch size is large and http compression is enabled

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions