otelcol: fix windows event log core placement #11051

braydonk · 2024-09-04T18:17:40Z

Description

In the CollectorSettings LoggingOptions, when the options are zap.WrapCore, the final core wrapped will end up running first. For this reason, the windowsEventLogCore must be the first option in the list of user supplied options, to ensure user supplied options are run before the core that writes to the Windows Event Log.

Prior art: I fixed this before in #5298.

Link to tracking issue

#5297

Testing

Testing was manual by running the Collector as a service after this change. I tried to add a unit test but could not find an effective way to do that due to all the type indirection done by the zap package. I was hoping to find an easy way to tell that the windowsEventLogCore is the first core wrapped in the options but I could not find a way to pull that off. Open to ideas if anyone can find a way.

In the CollectorSettings LoggingOptions, when the options are `zap.WrapCore`, the final core wrapped will end up running first. For this reason, the windowsEventLogCore must be the first option in the list of user supplied options, to ensure user supplied options are run before the core that writes to the Windows Event Log.

braydonk · 2024-09-04T18:23:50Z

otelcol/collector_windows.go

@@ -141,6 +138,19 @@ func openEventLog(serviceName string) (*eventlog.Log, error) {
 	return elog, nil
 }

+func loggingOptionsWithEventLogCore(


I extracted this to a function for 2 reasons:

I was trying to write a unit test for it (failed to find a way).

I wanted the comment explaining why the prepend is necessary to really stand out for any future contributors looking at this.

codecov · 2024-09-04T18:29:43Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.23%. Comparing base (65bebec) to head (e60fb92).
Report is 25 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #11051   +/-   ##
=======================================
  Coverage   92.23%   92.23%           
=======================================
  Files         414      414           
  Lines       19810    19810           
=======================================
  Hits        18271    18271           
  Misses       1167     1167           
  Partials      372      372

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

braydonk · 2024-09-04T19:18:02Z

I don't yet understand why this breaks the LogToFile test. It seems like this test is painful to run locally so might take a bit for me to get it set up and figure out what's going on.

braydonk · 2024-09-10T14:08:10Z

This test is broken for me on main, doesn't seem related to my changes. I think I'll need help to figure out why, I am super lost at this point. @pjanotti any chance you can help me figure out why this test appears to be busted?

EDIT: The test is not broken on main, I just hadn't rebuilt the binary. However I could still use assistance understanding how this change affects the logger in such a way to break file logging. At the spot I changed, there are no other logging options provided in the test anyway, so I don't understand at all what is going on.

EDIT 2: Wow, so it was the double pointer argument all along. Insane that I didn't see that. 😡

I should have passed the service config as a double pointer, not as a pointer that I further reference within the function.

pjanotti · 2024-09-10T22:49:55Z

Thanks for spotting this @braydonk

the windowsEventLogCore must be the first option in the list of user supplied options, to ensure user supplied options are run before the core that writes to the Windows Event Log.

So there were some options in telemetry, e.g.: log level, being ignored when logging to the event log?

pjanotti · 2024-09-10T22:56:22Z

otelcol/collector_test.go

@@ -564,6 +564,8 @@ func newEnvProvider() confmap.ProviderFactory {
 			return confmap.NewRetrieved(float64(6.4))
 		case "env:BOOL":
 			return confmap.NewRetrieved(true)
+		case "env:ProgramData":


Why is this needed? Was it added for the manual tests?

Left in by accident from manual tests. Removed in d8e89c2

pjanotti · 2024-09-10T23:02:34Z

otelcol/collector_windows.go

+	userOptions []zap.Option,
+) []zap.Option {
+	return append(
+		// The event log core must run last, so it needs to be the first option


Given that I already changed that order in the past (and can't recall if by accident or intentionally) explicit wording seems advisable:

Suggested change

// The event log core must run last, so it needs to be the first option

// The order below must be preserved - see PR #11051

// The event log core must run last, so it needs to be the first option

Made the comment more explicit in cf2b4c5

pjanotti · 2024-09-10T23:05:20Z

.chloggen/fix_windows_service_event_log_core.yaml

+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: []


Isn't the result of the change visible to user?

…lector into event_log_fix

braydonk · 2024-09-11T14:30:42Z

So there were some options in telemetry, e.g.: log level, being ignored when logging to the event log?

This specifically affects users who actually use the otelcol package to create their own Collector service. In my use case, I added LoggingOptions to change certain Collector self-logs to DEBUG level (effectively muting them in our case). When the order of the event log core was changed in the LoggingOptions, it meant that it always ran before these options we provided.

Isn't the result of the change visible to user?

Not for users of an OpenTelemetry-project-provided Collector as far as I know. This change is only noticeable for people in my very specific case who provide LoggingOptions in their custom-built collector that runs on Windows as a service. 😄

pjanotti

LGTM

braydonk requested review from a team and bogdandrutu September 4, 2024 18:17

fix changelog issue and PR number

52b9909

braydonk commented Sep 4, 2024

View reviewed changes

braydonk and others added 2 commits September 10, 2024 17:48

Fix double pointer argument

a334381

I should have passed the service config as a double pointer, not as a pointer that I further reference within the function.

Merge branch 'main' into event_log_fix

1ed4bcb

pjanotti reviewed Sep 10, 2024

View reviewed changes

atoulme added the os:windows Windows specific issues label Sep 10, 2024

braydonk added 3 commits September 11, 2024 14:19

remove unused test code

d8e89c2

make informative comment more explicit

cf2b4c5

Merge branch 'event_log_fix' of github.com:braydonk/opentelemetry-col…

e60fb92

…lector into event_log_fix

pjanotti approved these changes Sep 11, 2024

View reviewed changes

mx-psi approved these changes Sep 16, 2024

View reviewed changes

mx-psi merged commit 6029f31 into open-telemetry:main Sep 16, 2024
49 checks passed

github-actions bot added this to the next release milestone Sep 16, 2024

braydonk deleted the event_log_fix branch September 16, 2024 13:32

braydonk mentioned this pull request Sep 20, 2024

Prepend windowsEventLogCore instead of appending #5297

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

otelcol: fix windows event log core placement #11051

otelcol: fix windows event log core placement #11051

braydonk commented Sep 4, 2024 •

edited

Loading

braydonk Sep 4, 2024

codecov bot commented Sep 4, 2024 •

edited

Loading

braydonk commented Sep 4, 2024

braydonk commented Sep 10, 2024 •

edited

Loading

pjanotti commented Sep 10, 2024 •

edited

Loading

pjanotti Sep 10, 2024

braydonk Sep 11, 2024

pjanotti Sep 10, 2024

braydonk Sep 11, 2024

pjanotti Sep 10, 2024

braydonk commented Sep 11, 2024 •

edited

Loading

pjanotti left a comment

	// The event log core must run last, so it needs to be the first option
	// The order below must be preserved - see PR #11051
	// The event log core must run last, so it needs to be the first option

otelcol: fix windows event log core placement #11051

otelcol: fix windows event log core placement #11051

Conversation

braydonk commented Sep 4, 2024 • edited Loading

Description

Link to tracking issue

Testing

braydonk Sep 4, 2024

Choose a reason for hiding this comment

codecov bot commented Sep 4, 2024 • edited Loading

Codecov Report

braydonk commented Sep 4, 2024

braydonk commented Sep 10, 2024 • edited Loading

pjanotti commented Sep 10, 2024 • edited Loading

pjanotti Sep 10, 2024

Choose a reason for hiding this comment

braydonk Sep 11, 2024

Choose a reason for hiding this comment

pjanotti Sep 10, 2024

Choose a reason for hiding this comment

braydonk Sep 11, 2024

Choose a reason for hiding this comment

pjanotti Sep 10, 2024

Choose a reason for hiding this comment

braydonk commented Sep 11, 2024 • edited Loading

pjanotti left a comment

Choose a reason for hiding this comment

braydonk commented Sep 4, 2024 •

edited

Loading

codecov bot commented Sep 4, 2024 •

edited

Loading

braydonk commented Sep 10, 2024 •

edited

Loading

pjanotti commented Sep 10, 2024 •

edited

Loading

braydonk commented Sep 11, 2024 •

edited

Loading