Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Datadog reserved Status field, send error messages if no added fields #19

Merged
merged 5 commits into from
Jan 17, 2023

Conversation

ewucc
Copy link
Contributor

@ewucc ewucc commented Dec 23, 2022

This is for #6

Theres 2 changes here

  • use the datadog Status reserved field which does the logging color coding. It doesn't support anything outside of info, warn, error so anything above error we just use error
  • If there are no added zap fields, which currently get added by the plugins and anything that's interaction with user code files, then just send the error message.

We might want to chat about the second point. My memory is kind of fuzzy on this but I believe yuval did a pass on error messages and added the sanitization process for things that touch user code because we didn't want to leak anything via error messages. This change would mean that if we forget to add the zap field to sanitize user code info in a plugin or something, we would just send the error message.

Standard checks

  • Unit tests: Any special considerations?
  • Docs: Do we need to update any docs, internal or public?
  • Backwards compatibility: Will this break existing apps? If so, what would be the extra work required to keep them working?

@@ -78,16 +79,20 @@ func (t *Client) Debug(event string) {

func (t *Client) Warn(event string) {
t.Properties[datadogLogLevel] = Warn
t.Properties[datadogStatus] = Warn
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

were we always sending the logs, just classifying them as Info? I thought we send the "klotho compiling failed" as info, but also with the error logs and thats what was also missing

pkg/analytics/fields_listener.go Outdated Show resolved Hide resolved
pkg/analytics/client.go Outdated Show resolved Hide resolved
return nil
}

func SendEntryMessage() zap.Field {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some godoc for this explaining what it does / when to use it?

Also, this could be simplified I think to just

var SendEntryMessage = zap.Object("entryMessage", struct{}{})

(and checking f == SendEntryMessage in the listener instead of the Key)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep will do 👍

@@ -34,6 +34,7 @@ var (
)

var datadogLogLevel = "_logLevel"
var datadogStatus = "status"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how does status differ from _logLevel? From the docs, it seems like `status is the correct one to use so why send both?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeh I was confused by this in their docs initially also. So _loglevel allows you to specify whatever level you want and you can filter on it from the DD console. The status one though only allows the pre-defined (info, warn, error) and we can't modify that, but that causes the colorized fields in DD
image

So it's not entirely necessary since we filter on the _loglevel, but it gives you a quicker at a glance view since it will color the actual log entries instead of treating everything as info and blue by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we switch our usage of _loglevel to status? I don't think we really care about the levels it doesn't support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could, but it doesn't have things like debug/panic which currently i set to the closest relevant so debug -> info and panic -> error. If we're ok with losing granularity I can change that to just use status, but it didn't seem like we needed to consolidate and afaik it doesnt change costs in any way either so i left both.

@github-actions
Copy link

Package Line Rate Health
github.com/klothoplatform/klotho/pkg/analytics 2%
github.com/klothoplatform/klotho/pkg/annotation 23%
github.com/klothoplatform/klotho/pkg/core 20%
github.com/klothoplatform/klotho/pkg/env_var 82%
github.com/klothoplatform/klotho/pkg/exec_unit 45%
github.com/klothoplatform/klotho/pkg/infra/kubernetes 58%
github.com/klothoplatform/klotho/pkg/infra/kubernetes/helm 52%
github.com/klothoplatform/klotho/pkg/input 63%
github.com/klothoplatform/klotho/pkg/lang 37%
github.com/klothoplatform/klotho/pkg/lang/dockerfile 0%
github.com/klothoplatform/klotho/pkg/lang/golang 9%
github.com/klothoplatform/klotho/pkg/lang/javascript 47%
github.com/klothoplatform/klotho/pkg/lang/python 60%
github.com/klothoplatform/klotho/pkg/lang/yaml 0%
github.com/klothoplatform/klotho/pkg/logging 7%
github.com/klothoplatform/klotho/pkg/multierr 95%
github.com/klothoplatform/klotho/pkg/provider/aws 60%
github.com/klothoplatform/klotho/pkg/runtime 75%
github.com/klothoplatform/klotho/pkg/static_unit 32%
github.com/klothoplatform/klotho/pkg/validation 73%
Summary 42% (3593 / 8613)

@ewucc ewucc merged commit acafab9 into main Jan 17, 2023
@ewucc ewucc deleted the dd-log-update branch January 17, 2023 21:43
atorres-klo added a commit that referenced this pull request Aug 14, 2024
fixed some minor go linting errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants