[8.19] (backport #11998) (bugfix) log level does not change when standalone agent is reloaded or when otel runtime is used

[mergify]

What does this PR do?

When agent.logging.level config is changed during runtime in standalone elastic-agent, the new log level was not respected. (Note: This works in managed mode. This is the code path followed by managed mode for ref https://github.com/khushijain21/elastic-agent/blob/loglevel/internal/pkg/agent/application/actions/handlers/handler_action_settings.go#L54-L94 )

This PR fixes this bug and ensures this works both for process and OTel runtime.

Note: If both agent.logging.level and service::telemetry::logs::level are set - then the agent log level is applied to process runtime and telemetry::logs is applied to OTel runtime. If not, all components use agent.logging.level

Why is it important?

This is important so that we don't have to restart elastic-agent to update its logging level

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

None

How to test this PR locally

Start standalone elastic-agent with monitoring enabled and change the logging level without restarting. Observe the debug logs.

I tested this PR manually for both process and otel mode. And by switching from INFO-DEBUG-INFO.

Related issues

• Closes Log level changes do not affect the elastic-otel-collector sub-process #11959

---

This is an automatic backport of pull request #11998 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of 85b7e99 has failed:

On branch mergify/bp/8.19/pr-11998
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 85b7e9932.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1766556100-reload-log-level.yaml
	modified:   internal/pkg/agent/application/periodic.go
	modified:   internal/pkg/agent/cmd/run.go
	modified:   internal/pkg/otel/manager/execution.go
	modified:   internal/pkg/otel/manager/execution_subprocess.go
	modified:   internal/pkg/otel/translate/otelconfig.go
	modified:   testing/integration/ess/otel_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/agent/application/application.go
	both modified:   internal/pkg/agent/application/coordinator/coordinator.go
	both modified:   internal/pkg/agent/application/coordinator/coordinator_test.go
	both modified:   internal/pkg/agent/application/coordinator/coordinator_unit_test.go
	both modified:   internal/pkg/otel/manager/manager.go
	both modified:   internal/pkg/otel/manager/manager_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: fadc286

Failed CI Steps

• check-ci
• Unit tests - Ubuntu 22.04
• Unit tests - Ubuntu 22.04
• Unit tests - Ubuntu 22.04 with requirefips build tag
• Unit tests - Ubuntu 22.04 with requirefips build tag
• Unit tests - fips140=only Ubuntu 22.04
• Unit tests - fips140=only Ubuntu 22.04
• Unit tests - Ubuntu 22.04 ARM64
• Unit tests - Ubuntu 22.04 ARM64
• Unit tests - Windows 2022
• Unit tests - Windows 2022
• Unit tests - Windows 2016
• Unit tests - Windows 2016
• Unit tests - macOS 15 ARM
• Unit tests - macOS 15 ARM
• Unit tests - Windows 10
• Unit tests - Windows 10
• Unit tests - Windows 11
• Unit tests - Windows 11
• :junit: Junit annotate
• 📦 amd64: zip,tar.gz,rpm,deb
• 📦 amd64: zip,tar.gz,rpm,deb
• 📦 amd64: OTEL_COMPONENT zip,tar.gz,rpm,deb
• 📦 amd64: OTEL_COMPONENT zip,tar.gz,rpm,deb
• 📦 amd64: FIPS tar.gz
• 📦 amd64: FIPS tar.gz
• 📦 arm64: zip,tar.gz
• 📦 arm64: zip,tar.gz
• 📦 arm64: OTEL_COMPONENT zip,tar.gz
• 📦 arm64: OTEL_COMPONENT zip,tar.gz
• 📦 arm64: FIPS tar.gz
• 📦 arm64: FIPS tar.gz
• 📦 amd64: Containers
• 📦 arm64: Containers
• 📦 amd64: FIPS Containers
• 📦 arm64: FIPS Containers
• 📦 amd64: OTEL_COMPONENT Containers
• 📦 arm64: OTEL_COMPONENT Containers
• Aggregate test reports
• ESS FIPS stack cleanup
• Aggregate test reports

History

cc @khushijain21

[cmacknz]

Let's not backport this to active minors, it's a reasonably complex change and we don't want to release it before we've had a chance to see if any problems come up.