You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Marshalling go map type to CBOR is a common operation that can be expensive memory-wise of the map has a lot of fields. The code in CBOR uses map iteration that will allocate memory for each field and value. This excessive memory allocation is described in this issue on golang [1].
Describe the solution you'd like
Starting with go 1.18, there are two new methods Value.SetIterKey and Value.SetIterValue that will do fewer allocations.
Plus, the sync.Pool is also another tool that can be used to further reduce memory allocation by reusing temporary memory allocation.
Lastly, the Value.SetZero method (available since go 1.20) is helpful to release memory allocation to the GC when is no longer needed.
Additional context
The new methods SetIterKey and SetIterValue are introduced in go 1.18 and unavailable for go 1.17 or before while SetZero is available starting go 1.20. In order to maintain the backward compatibility for the CBOR codebase (if this is a concern), the code for encode map can be refractored into separate files where a go:build tag can be used to specify which files can be included in compilation for go 1.18+/1.20+ and what not.
The performance/memory improvement can be seen here (collected and analyzed by go benchmark and benchstat):
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Marshalling go map type to CBOR is a common operation that can be expensive memory-wise of the map has a lot of fields. The code in CBOR uses map iteration that will allocate memory for each field and value. This excessive memory allocation is described in this issue on golang [1].
Describe the solution you'd like
Starting with go 1.18, there are two new methods Value.SetIterKey and Value.SetIterValue that will do fewer allocations.
Plus, the sync.Pool is also another tool that can be used to further reduce memory allocation by reusing temporary memory allocation.
Lastly, the Value.SetZero method (available since go 1.20) is helpful to release memory allocation to the GC when is no longer needed.
Additional context
The new methods
SetIterKey
andSetIterValue
are introduced in go 1.18 and unavailable for go 1.17 or before whileSetZero
is available starting go 1.20. In order to maintain the backward compatibility for the CBOR codebase (if this is a concern), the code for encode map can be refractored into separate files where ago:build
tag can be used to specify which files can be included in compilation for go 1.18+/1.20+ and what not.The performance/memory improvement can be seen here (collected and analyzed by go benchmark and benchstat):
The text was updated successfully, but these errors were encountered: