Add Elasticsearch Scaler based on search template

[orphaner]

I am pleased to propose you the implementation of an Elasticsearch scaler based on a search template. Using a search template make things easy on the scaler configuration. The user must provide a search template name and eventually its params. The search result can be parsed using the gjson library like the metrics api scaler does.
For now, the authentication is based on username/password ; on the future we could imagine provide a CA certificate to validate a self signed certificate or ECE credentials.

I had a great time working on this scaler and I would be very pleased to get your code review's feedback.

Checklist

• Commits are signed with Developer Certificate of Origin (DCO - learn more)
• Tests have been added
• A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
• A PR is opened to update the documentation on (repo) (if applicable)
• Changelog has been updated

Fixes #1756

[JorTurFer]

Hi,
another small nits:
Could you review the comment about the metric name and add atl least 1 test with ScalerIndex > 0?
Thanks! ❤️

[JorTurFer]

/run-e2e elasticsearch*

[orphaner]

/run-e2e elasticsearch*

looks like the e2e test is KO; how do you get access to the logs?

[JorTurFer]

/run-e2e elasticsearch*

looks like the e2e test is KO; how do you get access to the logs?

https://github.com/kedacore/keda/runs/4284418478?check_suite_focus=true#step:7:1

[orphaner]

/run-e2e elasticsearch*

looks like the e2e test is KO; how do you get access to the logs?

https://github.com/kedacore/keda/runs/4284418478?check_suite_focus=true#step:7:1

https://github.com/kedacore/keda/runs/4284418478?check_suite_focus=true#step:7:94
it timeouts again. I've increased the timeout to 600s

[JorTurFer]

/run-e2e elasticsearch*

[JorTurFer]

Same error I think:
https://github.com/kedacore/keda/runs/4284976427?check_suite_focus=true#step:7:1

[orphaner]

I've add a kubectl describe command to help me understand why does the sts does not start

[orphaner]

write permissions are required to run /run-e2e elasticsearch*

ok ; could you run it for me please?

[JorTurFer]

/run-e2e elasticsearch*

[JorTurFer]

@orphaner , these are the logs:
https://github.com/kedacore/keda/runs/4285561916?check_suite_focus=true#step:7:1

[orphaner]

hello @JorTurFer I really need to understand what's going on. I add more kubectl command to debug that stuff.
Can you relaunch e2e tests please?
Thanks :)

[JorTurFer]

hello @JorTurFer I really need to understand what's going on. I add more kubectl command to debug that stuff. Can you relaunch e2e tests please? Thanks :)

Sure, all the times that you need :)

[JorTurFer]

/run-e2e elasticsearch*
Update: You can check the progres here

[orphaner]

hello @JorTurFer I really need to understand what's going on. I add more kubectl command to debug that stuff. Can you relaunch e2e tests please? Thanks :)

Sure, all the times that you need :)

many thanks!
BTW, now I've figured out how to get access to the e2e job logs ;)

[JorTurFer]

hello @JorTurFer I really need to understand what's going on. I add more kubectl command to debug that stuff. Can you relaunch e2e tests please? Thanks :)

Sure, all the times that you need :)

many thanks! BTW, now I've figured out how to get access to the e2e job logs ;)

Nice!! but it's not needed xD. Today we have merged a PR which edit the triggering comment with the execution url automatically:

it's the GitHub Action by itself who edit the comment with its own execution url :)
On-demand e2e test is a recently added feature, and we still be working on improving it xD

[JorTurFer]

/run-e2e elasticsearch*
Update: You can check the progres here

[orphaner]

hello @JorTurFer I really need to understand what's going on. I add more kubectl command to debug that stuff. Can you relaunch e2e tests please? Thanks :)

Sure, all the times that you need :)

many thanks! BTW, now I've figured out how to get access to the e2e job logs ;)

Nice!! but it's not needed xD. Today we have merged a PR which edit the triggering comment with the execution url automatically:

it's the GitHub Action by itself who edit the comment with its own execution url :) On-demand e2e test is a recently added feature, and we still be working on improving it xD

this is a nice improvement 👍

I've understood the problem. Elasticsearch use memory mapping - mmap. The sysctl vm.max_map_count parameter needs to be set to 262144 which is not the case on the github runner.
https://github.com/kedacore/keda/runs/4296384285?check_suite_focus=true#step:8:318

Can you run the test again? I've disabled mmap on the elasticsearch sts.

[JorTurFer]

the execution is in proges: #2311 (comment)
:)

[orphaner]

/run-e2e elasticsearch* Update: You can check the progres here

good new, the e2e tests are OK: https://github.com/kedacore/keda/runs/4296798019?check_suite_focus=true#step:8:414 👍
let me cleanup the debug outputs

[JorTurFer]

/run-e2e elasticsearch*
Update: You can check the progres here

[zroubalik]

Looking good, just a very minor nits

[JorTurFer]

/run-e2e elasticsearch*
Update: You can check the progres here

[zroubalik]

LGTM, thanks for the contribution!

[orphaner]

thanks for the merge 👍