Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GSRPC Registry high memory consumption #382

Open
khssnv opened this issue May 16, 2024 · 0 comments
Open

GSRPC Registry high memory consumption #382

khssnv opened this issue May 16, 2024 · 0 comments

Comments

@khssnv
Copy link

khssnv commented May 16, 2024

Hello. I experience high memory consumption by GSRPC Registry events decoder which tries to allocate about 28GiB in the following case.

In an attempt to decode Cere Network mainnet block 15 256 918 it tries to create a slice with 969 133 067 elements each more than 20 bytes long. Please find the screenshot with the debugger window below (it is hard to provide a minimal reproducible example, but please let me know if it is required). We can see that decoder gets the wrong length at go-substrate-rpc-client/scale/codec.go#L245. It then tries to allocate a large slice and panics with "out of memory" error if the system does not provide enough RAM or continues until a decoding error and updates metadata on the decoder fallback flow.

A naive fix which eliminates the issue is to update the metadata for each block as implemented in the patch: khssnv@ab97fff. But it adds an unnecessary RPC call on each block.

reflect.unsafe_NewArray (caution, light theme)

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant