Manticore transport unable to send custom headers with perform_request

[kaisecheng]

elasticsearch-transport gem is unable to send custom headers client.search(q: '*', size: 2, headers: {"Elastic-Api-Version"=>"2023-10-31"}) in Manticore implementation probably since 7.2.

The headers are ignored and overwritten by the initial constructor headers.
Faraday and curb do merge with input headers, hence exclude in this bug

Reproduce steps

Test it against serverless endpoint which should return 400 when the header is not "Elastic-Api-Version"=>"2023-10-31"

require "elasticsearch"
require "elasticsearch/transport/transport/http/manticore"

client = Elasticsearch::Client.new(
    {
        :hosts=>["https://YOUR_SERVERLESS_ENDPOINT:443"], 
        :transport_options=>
            {
                :headers=>{
                    "Authorization"=>"Basic YOUR_TOKEN",
                    "Elastic-Api-Version"=>"2023-10-31"
                }, 
                :request_timeout=>60, 
                :connect_timeout=>10, 
                :socket_timeout=>60
            }, 
        :transport_class=>Elasticsearch::Transport::Transport::HTTP::Manticore, 
        :ssl=>{:ssl=>true, :verify=>:default, :trust_strategy=>nil}
    })

# expect 200
puts client.info

# expect 400 but get 200
puts client.search(q: '*', size: 2, headers: {"Elastic-Api-Version"=>"2024-10-31"})

The last line is expected to get 400 but got 200 because the headers didn't send. You can initialize the Elasticsearch::Clientwith the wrong header "2024-10-31", the server returns 400.

[kaisecheng]

/cc @picandocodigo

[picandocodigo]

Hi @kaisecheng, I'll look into this and make a patch release soon. Thanks!

[kaisecheng]

@picandocodigo Could you tell me approximately the delivery schedule of the patch?

[picandocodigo]

@kaisecheng I'll try to get it out as soon as possible this week time permitting or early next week if not.

[picandocodigo]

@kaisecheng I think I've managed to fix it. If you can test out the branch, that'd be great. If not I can merge and prepare a patch release soon.

• Fixes instantiating Client in Manticore implementation. #69

[kaisecheng]

@picandocodigo Thanks for the patch. It works as expected

[picandocodigo]

@kaisecheng thanks for testing it! I'll publish a release soon 👍

[kaisecheng]

@picandocodigo I'll be on PTO for three weeks. I am hoping to get a new release version of elasticsearch-transport for logstash-plugin this week before I go. It would be great if you could publish a release soon ❤️

[picandocodigo]

@kaisecheng 8.2.4 is out with this fix, thanks!

[kaisecheng]

@picandocodigo thanks! Is it possible to backport the fix to elasticsearch-ruby 7.17 branch? Logstash plugins are still depending on 7.17 🙇

[picandocodigo]

@kaisecheng elasticsearch-ruby v7.17.9 is now available with the fix to elasticsearch-transport 🙂