[8.18](backport #7023) Remove filebeatreceiver and metricbeat receiver to remove dependency on Beats

[mergify]

What does this PR do?

Removes the filebeatreceiver and metricbeatreceiver and associated tests from the 9.0 branch.

Why is it important?

We need to update our dependency on otel-collector-contrib to get fixes to the elasticsearchexporter that the EDOT collector needs to go GA in 9.0. This update cannot be done because the otel dependency versions in Beats conflicts with the one we need. We can't update the otel dependency versions in Beats without upgraded Beats' dependency on the Prometheus packages, which introduce breaking changes we need time to adapt to. For full details see elastic/beats#42748.

There is no fast fix to this on the Beats side, this is IMO the least bad option. The other quick alternative would be to have Elastic Agent's 9.0 branch point to a feature branch in Beats with the dependency update (so 9.0 here would not point to 9.0 in Beats). That is less disruptive in this repository, but creates another unofficial branch to maintain as long as this situation goes on, and does not prevent this problem happening again with another dependency later that would require more work in the unofficial Beats branch. If it turns out this was the wrong decision, we can switch to the other approach at any time in the future by reverting this commit.

The intent is we'll revert this and bring the beat receiver components back once Beats can safely be updated to the appropriate OTel dependencies.

Disruptive User Impact

None, beats receivers are experimental and unannounced. This change isolates any impact to functionality that isn't meant to be usable externally in 9.0.

Related / blocked issues

• Update OTel components to v0.120.x #6912
• [chore] Update prometheus to match OTel v0.120.0 beats#42748
• Use elastic apm connector to replace other apm components #6937

  ---

  This is an automatic backport of pull request [9.0] Remove filebeatreceiver and metricbeat receiver to remove dependency on Beats #7023 done by Mergify.

[mergify]

Cherry-pick of 7f18476 has failed:

On branch mergify/bp/8.18/pr-7023
Your branch is up to date with 'origin/8.18'.

You are currently cherry-picking commit 7f1847629.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   internal/pkg/agent/cmd/testdata/otel/components-output.yml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   NOTICE.txt
	both modified:   go.mod
	both modified:   go.sum
	both modified:   internal/pkg/otel/components.go
	both modified:   testing/integration/otel_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[cmacknz]

We are going to vendor the Prometheus dependency in Beats to hopefully prevent us from needing to do this in this branch elastic/beats#42890

[mergify]

This pull request has not been merged yet. Could you please review and merge it @swiatekm? 🙏

[mergify]

This pull request has not been merged yet. Could you please review and merge it @swiatekm? 🙏

[pchila]

We are going to vendor the Prometheus dependency in Beats to hopefully prevent us from needing to do this in this branch elastic/beats#42890

@cmacknz PR elastic/beats#42890 has been closed. Should this be merged then or we are evaluating alternatives?

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.18/pr-7023 upstream/mergify/bp/8.18/pr-7023
git merge upstream/8.18
git push upstream mergify/bp/8.18/pr-7023

[mergify]

This pull request has not been merged yet. Could you please review and merge it @swiatekm? 🙏

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: b5858eb

History

• 💚 Build #18440 succeeded 0cafafa
• 💛 Build #17890 was flaky 07b47d2
• 💔 Build #17766 failed 39a7ba4
• 💔 Build #17645 failed c6bfadd
• 💔 Build #17613 failed fe90900
• 💔 Build #17583 failed ed78b13

cc @swiatekm

[swiatekm]

Closing in favor of #7443.