feat: replace apmelasticsearch with otel tracing

[kruskall]

Motivation/summary

replace go agent apmelasticsearch module with otel

replace go agent apmtest with otel manualreader in tests

set span attributes to ensure span type is set in kibana

Checklist

• Update CHANGELOG.asciidoc
• Documentation has been updated

For functional changes, consider:

• Is it observable through the addition of either logging or metrics?
• Is its use being published in telemetry to enable product improvement?
• Have system tests been added to avoid regression?

How to test these changes

Related issues

Closes #18949

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label. Could you fix it @kruskall? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-7.17 is the label to automatically backport to the 7.17 branch.
• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• backport-9./d is the label to automatically backport to the 9./d branch. /d is the digit.
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 8243ae5

[lahsivjar]

Can we use the OTel's propogator API to do this?

[kruskall]

Sorry, can you expand on this ? I'm not sure how this would work (keep in mind we're not using upstream otel tracer (yet) but the apmotel abstraction in the go agent)