Add custom ECH deployment + smoke test

[michel-laterman]

What does this PR do?

Use the image produced in the packaging step as the integrations server image for an ECH deployment and run a basic smoke test against that deployment. This allows for the changes made in a PR to be tested as a part of the cloud image in addition to the current test coverage.

Why is it important?

Removes the 1 day delay from producing a SNAPSHOT image to testing the image.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

N/A

[michel-laterman]

This PR copies the approach taken by #8035, but applies it to the normal integration test pipeline

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[v1v]

LGTM, at least the CI specifics. Thanks Michel.

It’s concerning how much duplicated configuration is needed here. We plan to revisit this in the future; however, for now, this is our reality.

[swiatekm]

Looks good enough to me, though I think some of the env variable manipulation in cloud:load is unnecessary.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b integration-tests-custom-image upstream/integration-tests-custom-image
git merge upstream/main
git push upstream integration-tests-custom-image

[v1v]

Thanks for cleaning up some of the cosmetic leftovers. I left a few comments

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 68339f6

Failed CI Steps

• Custom ECH Testing
• x86_64:sudo: fleet-privileged

History

• 💔 Build #23886 failed ea71e70
• 💔 Build #23835 failed fa88b18
• 💔 Build #23833 failed a57debb
• 💚 Build #23769 succeeded 7ed4952
• 💔 Build #23766 failed a6c10cc
• 💔 Build #23762 failed d89c99d

cc @michel-laterman

[pkoutsovasilis]

LGTM, @v1v please have another look at your comments, we need this PR as it increases the testing footprint of elastic-agent

[v1v]

Besides what we discussed in the thread, I approve the CI part