Memory Leak？

[zhangnian]

Question：

Why does the memory usage of BigCache keep growing in the following code? The function handling HTTP requests does not involve any BigCache read or write operations.
Every time this program runs, the memory usage quickly increases to around 10GB.

Go Version：1.23
Go Mod：

go 1.23

require (
	github.com/allegro/bigcache/v3 v3.1.0
	github.com/gin-gonic/gin v1.10.0
)

My Code：

package main

import (
	"context"
	"github.com/allegro/bigcache/v3"
	"github.com/gin-gonic/gin"
	"net/http"
)

var localBigCache *bigcache.BigCache

func init() {
	lbc, _ := bigcache.New(context.Background(), bigcache.Config{
		Shards:             1024,
		LifeWindow:         0,
		CleanWindow:        0,
		MaxEntriesInWindow: 1024 * 6,
		MaxEntrySize:       1024 * 100,
		StatsEnabled:       false,
		Verbose:            true,
		HardMaxCacheSize:   0,
	})

	localBigCache = lbc
}

func main() {
	r := gin.New()
	r.GET("/test", func(c *gin.Context) {
		c.String(200, "hello world")
	})

	srv := &http.Server{
		Handler: r,
	}

	if err := srv.ListenAndServe(); err != nil {
		panic(err)
	}
}

[janisz]

Based on napkin calculations: 1024 (shards) * 10 (initialShardSize) * 1024 * 100 (MaxEntrySize) = 1GB

bigcache/shard.go

Lines 434 to 454 in a2f05d7

	func initNewShard(config Config, callback onRemoveCallback, clock clock) *cacheShard {
	bytesQueueInitialCapacity := config.initialShardSize() * config.MaxEntrySize
	maximumShardSizeInBytes := config.maximumShardSizeInBytes()
	if maximumShardSizeInBytes > 0 && bytesQueueInitialCapacity > maximumShardSizeInBytes {
	bytesQueueInitialCapacity = maximumShardSizeInBytes
	}
	return &cacheShard{
	hashmap: make(map[uint64]uint64, config.initialShardSize()),
	hashmapStats: make(map[uint64]uint32, config.initialShardSize()),
	entries: *queue.NewBytesQueue(bytesQueueInitialCapacity, maximumShardSizeInBytes, config.Verbose),
	entryBuffer: make([]byte, config.MaxEntrySize+headersSizeInBytes),
	onRemove: callback,
	isVerbose: config.Verbose,
	logger: newLogger(config.Logger),
	clock: clock,
	lifeWindow: uint64(config.LifeWindow.Seconds()),
	statsEnabled: config.StatsEnabled,
	cleanEnabled: config.CleanWindow > 0,
	}
	}

How did you measure used memory?
How do you define quickly? It's seconds? Minutes?
Could you generate profiles of this app?

File: main
Type: inuse_space
Time: Feb 15, 2025 at 10:17pm (CET)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1122.09MB, 100% of 1122.60MB total
Dropped 2 nodes (cum <= 5.61MB)
      flat  flat%   sum%        cum   cum%
 1011.65MB 90.12% 90.12%  1011.65MB 90.12%  github.com/allegro/bigcache/v3/queue.NewBytesQueue (inline)
  110.45MB  9.84%   100%  1122.09MB   100%  github.com/allegro/bigcache/v3.initNewShard
         0     0%   100%  1122.09MB   100%  github.com/allegro/bigcache/v3.New (inline)
         0     0%   100%  1122.09MB   100%  github.com/allegro/bigcache/v3.newBigCache
         0     0%   100%  1122.09MB   100%  main.init.0
         0     0%   100%  1122.60MB   100%  runtime.doInit (inline)
         0     0%   100%  1122.60MB   100%  runtime.doInit1
         0     0%   100%  1122.60MB   100%  runtime.main

File: main
Type: alloc_space
Time: Feb 15, 2025 at 10:16pm (CET)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1122.09MB, 99.69% of 1125.57MB total
Dropped 18 nodes (cum <= 5.63MB)
      flat  flat%   sum%        cum   cum%
 1011.65MB 89.88% 89.88%  1011.65MB 89.88%  github.com/allegro/bigcache/v3/queue.NewBytesQueue (inline)
  110.45MB  9.81% 99.69%  1122.09MB 99.69%  github.com/allegro/bigcache/v3.initNewShard
         0     0% 99.69%  1122.09MB 99.69%  github.com/allegro/bigcache/v3.New (inline)
         0     0% 99.69%  1122.09MB 99.69%  github.com/allegro/bigcache/v3.newBigCache
         0     0% 99.69%  1122.09MB 99.69%  main.init.0
         0     0% 99.69%  1122.60MB 99.74%  runtime.doInit (inline)
         0     0% 99.69%  1122.60MB 99.74%  runtime.doInit1
         0     0% 99.69%  1122.60MB 99.74%  runtime.main

[zhangnian]

Sorry, it was my mistake. The correct value should be 1GB.

Showing nodes accounting for 1093.10MB, 99.59% of 1097.61MB total
Dropped 21 nodes (cum <= 5.49MB)
      flat  flat%   sum%        cum   cum%
 1002.54MB 91.34% 91.34%  1002.54MB 91.34%  github.com/allegro/bigcache/v3/queue.NewBytesQueue (inline)
   90.56MB  8.25% 99.59%  1093.10MB 99.59%  github.com/allegro/bigcache/v3.initNewShard
         0     0% 99.59%  1093.10MB 99.59%  github.com/allegro/bigcache/v3.New (inline)
         0     0% 99.59%  1093.10MB 99.59%  github.com/allegro/bigcache/v3.newBigCache
         0     0% 99.59%  1093.10MB 99.59%  main.init.0
         0     0% 99.59%  1093.10MB 99.59%  runtime.doInit (inline)
         0     0% 99.59%  1093.10MB 99.59%  runtime.doInit1
         0     0% 99.59%  1093.10MB 99.59%  runtime.main

What I’m confused about is that I initialized a 1GB cache, but it wasn’t actually used right away. Moreover, after the program started, the memory didn’t increase immediately. Instead, the cache only grew to 1GB as the number of processed HTTP requests increased.

I measure used memory by continuous observation windows 11's Process Monitor Software working set memory, The working set refers to the physical memory usage, not the virtual memory usage.

I understand that 1GB of virtual memory was allocated when initializing the cache, but during the actual HTTP request processing, this cache wasn’t immediately utilized. Why, then, does the physical memory usage rapidly and continuously increase to 1GB?

Within a few seconds, the physical memory usage increases.

There is my stress test script:

wrk -t 100 -c 1000 -d 100s http://localhost:8292/test

Thank you for your patient reply.

[janisz]

The correct value should be 1GB.

That's good to hear as that's something I got with my napkin math.

What I’m confused about is that I initialized a 1GB cache, but it wasn’t actually used right away.

bigcache eagerly allocate memory it estimates to be used based on passed config. This is done to minimise new allocations later.

I understand that 1GB of virtual memory was allocated when initializing the cache, but during the actual HTTP request processing, this cache wasn’t immediately utilized. Why, then, does the physical memory usage rapidly and continuously increase to 1GB?

That's great question but I'm afraid beyond my knowledge limits :( I believe that as a program you have no idea what type of memory you get from OS and the OS decided about virtual/physical allocations. 1GB is not much theses days so if it fits your RAM then there is no reason to allocate it somewhere else. You can try filling your ram and then running code. You need to keep in mind that Go manages memory for you, so there is a chance that GC or runtime "touches" allocated slices preventing them from swaping 🤷