Skip to content

Commit

Permalink
statistics: avoid large CMSketch affecting the latency of normal query (
Browse files Browse the repository at this point in the history
  • Loading branch information
sre-bot authored and bobotu committed Aug 11, 2020
1 parent 8f13cf1 commit 0d97ac0
Showing 1 changed file with 10 additions and 1 deletion.
11 changes: 10 additions & 1 deletion statistics/cmsketch.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,17 @@ type TopNMeta struct {
// NewCMSketch returns a new CM sketch.
func NewCMSketch(d, w int32) *CMSketch {
tbl := make([][]uint32, d)
// Background: The Go's memory allocator will ask caller to sweep spans in some scenarios.
// This can cause memory allocation request latency unpredictable, if the list of spans which need sweep is too long.
// For memory allocation large than 32K, the allocator will never allocate memory from spans list.
//
// The memory referenced by the CMSketch will never be freed.
// If the number of table or index is extremely large, there will be a large amount of spans in global list.
// The default value of `d` is 5 and `w` is 2048, if we use a single slice for them the size will be 40K.
// This allocation will be handled by mheap and will never have impact on normal allocations.
arena := make([]uint32, d*w)
for i := range tbl {
tbl[i] = make([]uint32, w)
tbl[i] = arena[i*int(w) : (i+1)*int(w)]
}
return &CMSketch{depth: d, width: w, table: tbl}
}
Expand Down

0 comments on commit 0d97ac0

Please sign in to comment.