Questions about memory usage

[youngledo]

After reading the official performance report, the conclusion is that log4j2 is more powerful, but why isn't it the case when I test it myself?

1. Do nothing

• log4j2
  Firstly, without doing anything, the application's memory is about 160M:
  

• logback
  But I only have around 30MB of memory when using Logback:
  

2. Write log

It was tested in the same way: http://127.0.0.1:8080/log4j2/pressure?minLengthInclusive=102400&maxLengthExclusive=409600&times=10000

• log4j2
  Although the memory usage is not very high, it is more than twice as much as logback.
  

• logback
  

Did I do something wrong?

This is my test project(Spring Boot):
log-sample.zip

[youngledo]

I roughly understand the reason, it's because of the disruptor. If I exclude it, the memory will decrease.

Then do the pressure test in the same way as above:

It feels similar to Logback.

So, I'm confused now, do I need to use a disruptor? Can you help me answer this?

[rgoers]

The disruptor is only required when using async loggers or the AsyncAppender is configured to use it.

[tcmot]

你想测试log4j2的内存使用情况，应该参考这篇文档https://logging.apache.org/log4j/2.x/manual/garbagefree.html.
最好使用普通项目来测试，而不是SpringBoot项目。想节省内存disruptor是必须的，这是因为disruptor会预创建很多的对象，避免使用了临时对象。而logback却没有这个特性，理论上，logback内存占用情况一定是高于log4j2的。

我自己也写了一个日志库，https://github.com/jdkstack/nogc4j-ce.
这是一个无GC日志库，内存占用上是固定的，不会发生改变。

[ppkarwasz]

1. Do nothing

* log4j2
  Firstly, without doing anything, the application's memory is about 160M:

This graph isn't really reliable: you don't see any garbage-collection event, so it is entirely possible that most of the 160 MiB could be collected.

2. Write log

It was tested in the same way: http://127.0.0.1:8080/log4j2/pressure?minLengthInclusive=102400&
maxLengthExclusive=409600&times=10000

What does minLengthInclusive and maxLengthExclusive parameters mean? If these are the sizes of log messages, their values greatly exceed the size of a typical log event. Log4j Core by default expects log messages up to 518 bytes and log events up to 8192 bytes in size. Besides these limits you start creating a lot of temporary objects and you are no longer garbage-free. See Garbage Free logging for more details.

the conclusion is that log4j2 is more powerful, but why isn't it the case when I test it myself?

If by "powerful" you mean that Log4j Core always provides a better performance and lower memory usage than Logback, the conclusion is wrong. Everything depends on how your application is configured and how you use logging. Note that the astonishing 20M messages/s is a peak throughput that you can keep for very small periods of time (a couple of ms). You can compare this with the performance of SSDs: writing the first 8 MiB or so is lightning fast, but the sustainable rate is much lower.

Regarding memory usage:

• The garbage-free mode increases your memory usage, since many reusable buffers need to be allocated. You pay an initial memory price, but later on you will not experience GC pauses due to logging. Nowadays garbage collection is very fast, but a couple of years ago it was a "stop the world" event, which was very costly for financial applications. See log4j2.enableThreadlocals for details on how to disable garbage-free logging.
• Using the Disruptor also costs memory, since it pre-allocates up to 256 × 1024 buffers. See asynchronous loggers on how to enable or disable the usage of the LMAX Disruptor.