Experimental: zap backend for go-log

[Kubuxu]

Builds and works with go-ipfs, exposes zap functionality.

[frrist]

👍 This looks good to me. Excited to use the new functionality this adds if we decide to move forward with this!

[Kubuxu]

Updated and rebased on master

[frrist]

Hey @Kubuxu is there anything I could do to help move this forward? I have been using it on a test branch for go-filecoin. Having the capability to produce structured logs with this would be great.

[lanzafame]

@Kubuxu @frrist with all the projects that use go-log now moved over to using go modules, can we tag this as a new major release and move on?

[Stebalien]

As far as I can tell, the only real breaking changes are around setting log-levels, right? In that case, bumping a minor (this is pre-1.0) should be fine. Yeah, it's a breaking change but it's easy to deal with (and there's no good way to avoid it).

Well, technically, we could release a version of go-logging that uses zap.

---

@lanzafame this project is pre-1.0 so bumping the major to v1 doesn't really help (and we're not ready for that anyways).

[frrist]

Looks good, couple minor comments that I will leave up to you.

[Stebalien]

Is the plan to eventually switch to zap directly? It seems odd to hard-code zap into this library and expose it like this if that isn't the case.

However, if that is the case, we should have a migration plan in place to get rid of the facade.

[Kubuxu]

Zap's interface is big but very useful for structured logging. I don't see any benefit of writing a full interface facade for it (it won't allow us to switch to something else easier).

The migration is mostly limited to stopping the use of deprecated functions.

[Stebalien]

But is there a plan to eventually drop this library and directly depend on Zap.

[raulk]

I too was in the camp that logging is so pervasive that we shouldn't introduce indirections. Also, changing the logging backend happens once in a blue moon -- so it's a questionable risk/reward gradient. However, there are some legit use cases where you'd want to replace the logging backend with something else, e.g. WASM runtimes.

[Kubuxu]

No, as Zap doesn't provide centralised log level facilities.

[raulk]

However, there are some legit use cases where you'd want to replace the logging backend with something else, e.g. WASM runtimes.

I guess we'd do that using build tags anyway. As long as the callers don't need to reference zap or zapcore explicitly, we should be able to mimic its public API?

[Kubuxu]

We can change how logging is done for WASM by changing formatters and outputs in Zap.

Problem with mimicking Zap's API is that they are at least 93 functions we would have to define.

In some cases (performance) one would use zap.Field and connected functions to achieve 0-overhead structured logging.

[Stebalien]

LGTM (would still like to think through the rest of the upgrade path but this looks like a reasonable step forward).

[Stebalien]

I'd like to get @raulk to give a signoff as this affects go-libp2p pretty significantly. Also, @hsanjuan.

Breaking changes:

• SetAllLoggers now takes a string level and go-logging is no longer a dep.
• Warning has been officially deprecaed in favor of Warn (may cause linters to fail).
• edit: it breaks custom log formats

The benefit is:

• Structured logging support.
• Switching to someone else's logging library (instead of our own go-logging).

[hsanjuan]

I have customized LogFormats because I could not read things in blue. This will also break that. But ok..

[raulk]

I too was in the camp that logging is so pervasive that we shouldn't introduce indirections. Also, changing the logging backend happens once in a blue moon -- so it's a questionable risk/reward gradient. However, there are some legit use cases where you'd want to replace the logging backend with something else, e.g. WASM runtimes.

[raulk]

LGTM. @Kubuxu I took the liberty to add some godocs and update deps; revert if inappropriate.

[raulk]

Quick alignment comment

@whyrusleeping: IIRC in the "exposing zap vs. abstraction with a zap backend" discussion, you advocated for keeping an abstraction layer as a way to substitute the backend in alternative runtime environments, like WASM. Zap is pretty modular and we should be able to do that replacement within Zap itself. Are you OK going in this direction?

@albrow: since 0x is already running go-libp2p on WASM, are you happy with this change? It should bring important optimisations.

@Kubuxu: do you have bandwidth to introduce a few go benchmarks to quantify the gains under different conditions (console, file, structured logging, different levels, etc.)?

---

Looks like the tests provoke a deadlock and a few data races: https://travis-ci.com/ipfs/go-log/builds/125061936

[Kubuxu]

Quick benchmarks:

benchmark                      old ns/op     new ns/op     delta
BenchmarkSimpleInfo-4          5598          4015          -28.28%
BenchmarkFormatInfo-4          5819          4523          -22.27%
BenchmarkFormatInfoMulti-4     4482          2633          -41.25%

benchmark                      old allocs     new allocs     delta
BenchmarkSimpleInfo-4          27             10             -62.96%
BenchmarkFormatInfo-4          27             11             -59.26%
BenchmarkFormatInfoMulti-4     28             12             -57.14%

benchmark                      old bytes     new bytes     delta
BenchmarkSimpleInfo-4          928           368           -60.34%
BenchmarkFormatInfo-4          1488          569           -61.76%
BenchmarkFormatInfoMulti-4     1464          562           -61.61%

[Kubuxu]

Looks like the tests provoke a deadlock and a few data races:

Those were there, it is a race inside a test (not the event writer) and race detector causes the lockup.

[Kubuxu]

I probably won't have time to benchmark it more (at least right now).

[albrow]

@raulk we aren't enabling go-log except for some rare cases where we need to debug or understand libp2p internals. As long as the new changes still allow libp2p to compile to Wasm it won't really affect us.

[Kubuxu]

@Stebalien @hsanjuan @raulk
If there are no objections, I will merge this tomorrow.

[Stebalien]

Please do.

[raulk]

Benchmarks look great.