[8.19](backport #44782) [beatreceiver] - Add status reporting

[mergify]

Proposed commit message

This PR adds status reporting for beatreceivers. The status reporting is added while creating the runners. The first PR (#44528) was quite "hacky" and it had go deep down to inject status reporters.

This PR adds a runner factory wrapper that will:

1. Call the parent factory to create the runner
2. Inject status reporter

The code responsible for doing the above tasks will live in libbeat and we will only enable it for beatreceivers. From an the beat receiver high level, it will do following:

1. The beater will be created in createReceiver
2. We will add the factory wrapper

   beats/x-pack/libbeat/cmd/instance/receiver.go

   Lines 80 to 83 in 344bbce

   	if w, ok := br.beater.(cfgfile.WithFactoryWrapper); ok {
   	groupReporter := status.NewGroupStatusReporter(host)
   	w.WithFactoryWrapper(status.StatusReporterFactory(groupReporter))
   	}

3. The receiver will kick off the beater

   beats/x-pack/libbeat/cmd/instance/receiver.go

   Lines 76 to 81 in 6286492

   	func (br *BeatReceiver) Start() error {
   	if err := br.beater.Run(&br.beat.Beat); err != nil {
   	return fmt.Errorf("beat receiver run error: %w", err)
   	}
   	return nil
   	}

Note:

To accomplish the above steps, it is essential that we create the runners in beater.Run(...). Currently, metricbeat creates runners during the beater creation phase and starts them in beater.Run(...). This PR moves the runner creation code in beater.Run(...) to closely align with filebeat's implementation.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Related Beat receivers do not correctly report status back to the Elastic Agent elastic-agent#8210

Screenshots

Output

Here's output of running two streams (degraded) together:

┌─ fleet
│  └─ status: (STOPPED) Not enrolled into Fleet
└─ elastic-agent
   ├─ status: (DEGRADED) 1 or more components/units in a degraded state
   ├─ pipeline:logs/_agent-component/filestream-default
   │  ├─ status: StatusRecoverableError [error while running harvester: cannot read from file source: /var/log/elasticAgent-install-20240625_133733.log]
   │  ├─ exporter:elasticsearch/_agent-component/default
   │  │  └─ status: StatusOK
   │  └─ receiver:filebeatreceiver/_agent-component/filestream-default
   │     └─ status: StatusRecoverableError [error while running harvester: cannot read from file source: /var/log/elasticAgent-install-20240625_133733.log]
   └─ pipeline:logs/_agent-component/system/metrics-default
      ├─ status: StatusRecoverableError [Error fetching data for metricset system.process: error fetching process list: non fatal error; reporting partial metrics: error fetching PID metrics for 607 processes, most likely a "permission denied" error. Enable debug logging to determine the exact cause.]
      ├─ exporter:elasticsearch/_agent-component/default
      │  └─ status: StatusOK
      └─ receiver:metricbeatreceiver/_agent-component/system/metrics-default
         └─ status: StatusRecoverableError [Error fetching data for metricset system.process: error fetching process list: non fatal error; reporting partial metrics: error fetching PID metrics for 607 processes, most likely a "permission denied" error. Enable debug logging to determine the exact cause.]

Testing

1. Checkout this PR locally
2. Go to elastic-agent and follow this guide to test local beats changes
3. Package agent with mage package
4. Follow steps on Beat receivers do not correctly report status back to the Elastic Agent elastic-agent#8210 to install agent and verify the status

Closes elastic/elastic-agent#8210

---

This is an automatic backport of pull request #44782 done by Mergify.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[mergify]

This pull request has not been merged yet. Could you please review and merge it @VihasMakwana? 🙏

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.19/pr-44782 upstream/mergify/bp/8.19/pr-44782
git merge upstream/8.19
git push upstream mergify/bp/8.19/pr-44782