metrics: add gauge with compression goal if enabled

[yaauie]

Release notes

[rn: skip]

What does this PR do?

Adds a gauge to pipeline queue compression metrics to indicate the encode goal

Why is it important/What is the impact to the user?

When a user is analyzing the cost and benefit of queue compression, including the current setting helps contextualize the values.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

1. create an ndjson file example-input.ndjson:

   {"this":"that","banana":["orange",{"elephant":1}]}
   {"event":{"original":"this is an original event"}}
   

2. add my feed utility to your path
3. Invoke Logstash with options that activate the PQ and compression, feeding your input to stdin:

   bin/logstash --log.level=info \
   -Squeue.type=persisted \
   -Squeue.drain=true \
   -Squeue.compression=balanced \
   --config.string 'input { stdin { codec => json_lines } } output { sink {} }' \
   < <(feed example-input.ndjson)
   

4. while that runs, query the HTTP API:

   curl --silent 'localhost:9600/_node/stats?pretty=true' | jq .pipelines.main.queue.compression
   

5. observe that the encode.goal matches the provided queue.compression setting.

Related issues

Pared off of: #18227

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label. Could you fix it @yaauie? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• If no backport is necessary, please add the backport-skip label

[yaauie]

I made the decision to sample the available values instead of exploding the matrix by 4x.

[robbavey]

LGTM

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 8b1da6d

Failed CI Steps

• 🥼 IT Persistent Queues / part 2-of-6