Migrate to loupe for determining the size of modules in memory

[uint]

Currently, we use our own algorithm for guesstimating the size of WASM modules used in packages/vm/src/cache.rs. (Search for module_size and module.size.)

We should see if we can migrate to loupe instead.

[webmaster128]

Here is the line that panics because of the overflow: https://github.com/wasmerio/wasmer/blob/2.0.0-rc2/lib/types/src/entity/boxed_slice.rs#L159. Now we need to get a stack trace to understand how this is called.

[uint]

@Hywan I boiled it down to a MWE and it seems reproducible with just built-in wasmer types.

use wasmer::{Cranelift, Module, Store, Universal};

static CONTRACT: &[u8] = include_bytes!("../hackatom.wasm");

#[test]
fn loupe_overflow_tests() {
    let engine = Universal::new(Cranelift::default()).engine();
    let store = Store::new(&engine);
    let foo = Module::new(&store, &CONTRACT).unwrap();

    loupe::size_of_val(&foo); // DON'T STAY CALM, DO PANIC!
}

[dependencies]
loupe = "0.1.2"
wasmer = { git = "https://github.com/wasmerio/wasmer", tag = "2.0.0-rc2" }

The Wasm file can be found here:
https://github.com/CosmWasm/cosmwasm/blob/main/packages/vm/testdata/hackatom.wasm

I don't think we're going to investigate further right now since it's not a critical issue for us, but if we can help you along, let us know!

[Hywan]

OK I understand what's happening. Let me fix this :-). Thank you for your investigation!

[Hywan]

Universal has a field BoxedSlice<…, FunctionBodyPtr>. This FunctionBodyPtr holds a pointer. The first time a pointer is visited to collect the memory usage, it returns POINTER_BYTE_SIZE. The second time a pointer is visited, it returns 0. In this particular case, it's been already visited, so it returns 0, where we substract 8 with mem::size_of, and boom. I'll think about a fix.

[webmaster128]

@Hywan could you create a ticket in Wasmer to track this and make visible to other users? This is not blocking us for now. Thanks!

[Hywan]

The 0.1.3 version of wasmerio/loupe is fixing your issue. I'm opening a PR on wasmerio/wasmer.

[webmaster128]

Amazing, thanks!

[maurolacy]

Revisiting this, just out of curiosity. Here is a comparison of our guesstimate of module sizes, versus loupe's provided value:

$ ct --tests pin_unpin -- --nocapture
    Finished test [unoptimized + debuginfo] target(s) in 0.05s
     Running /home/mauro/work/cosmwasm/target/debug/deps/cosmwasm_vm-73bd49b65fcd8938

running 1 test
module.size: 2953616
loupe: 4971300
test cache::tests::pin_unpin_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 249 filtered out; finished in 0.80s

$ 

So, either we're underestimating module sizes, or loupe is over-estimating them. And who knows which is which and who is who.

I think that, in any case, we should use loupe's value, as it's better to over-estimate the cache usage, than the opposite.

[webmaster128]

Could you check the loupe values of module.store() and module.artifact()?

[maurolacy]

Here are the numbers:

module.size: 2953616
loupe module: 4971300
loupe module.store: 1773460
loupe module.artifact: 3219840

I think this makes sense. We're (probably) guesstimating the artifact size more or less accurately, and missing on the store size.

I also noticed that module.store + module.artifact (4993300) is bigger than module (4971300) by about 22KB. Which doesn't make much sense... the opposite should be the case. In any case, it's a small amount (though it speaks of "overestimation". Or better, "estimation variance").

[Hywan]

One think you must be careful about when comparing sizes is that MemoryUsage keeps track of already visited pointers. For instance, if you get the size of Store, you will visit all pointers once. Then, you get the size of Artifact, it will start with a fresh new “memory tracker”, so it will include Store if there is somewhere a pointer to the Store, because it recursively traverse all types. What I'm trying to explain is that module.store + module.artifact is wrong; either you calculate module, either module.store, either module.artifact, but you can't sum things.

loupe::size_of_val is a shortcut that uses a new “memory tracker” everytime:

pub fn size_of_val<T: MemoryUsage>(value: &T) -> usize {
    <T as MemoryUsage>::size_of_val(value, &mut BTreeSet::new())
}

Otherwise, you will need to use the same “memory tracker”, see the MemoryUsage::size_of_val definition:

fn size_of_val(&self, tracker: &mut dyn MemoryUsageTracker) -> usize

You can do that like this (not tested, almost pseudo-rust-ish-code):

let mut tracker = BTree::new();
dbg!(module.store().size_of_val(&mut tracker));
dbg!(module.size_of_val(&mut tracker));

That way, you will re-use the same “memory tracker”, and you can more or less compare memory usages. I'm saying more or less because already visited pointers count for 1, so you are likely to notice small differences in your maths.

[maurolacy]

Does loupe has a way to differentiate between heap and stack memory?

My guesstimate only considers heap memory. That may very well be the reason for the divergence in the reported used memory.

[maurolacy]

OK, I was playing with this these days, and I found a series of issues that may be causing the discrepancies:

• massif uses snapshots, and misses small allocations (less than 1kb) sometimes. If there are a lot of them, that may explain the differences, or part of it.
• The massif documentation specifically mentions that some memory allocations are not considered. So, if wasmer is using mmap or friends, by example to allocate the store, those bytes will not be counted. I tried enabling the --pages-as-heap=yes option, but then memory numbers go to the roof, as this counts stack and data segments as used memory too.

All in all, this shows that massif provides an initial estimate which is not bad (i.e. it's of the right order of magnitude), but if your program is either doing a lot of small allocations, and / or using non-conventional allocation methods, the reported memory usage will be somewhat lower than the real one.

[maurolacy]

So, I think we must clearly change the module size estimation method to loupe.

[webmaster128]

Thank you for looking into this – sounds good! Is anything blocking us from wapping iut the implementation? Do you have the chance to open a PR for this, @maurolacy?

[maurolacy]

OK, I will cut a PR with the changes.

[Hywan]

I confirm that we do take into account the mapped memory, https://github.com/wasmerio/wasmer/blob/7e914488383e2c0f3086b1b3aa60f277d60f5095/lib/vm/src/mmap.rs#L283-L289.

[maurolacy]

I confirm that we do take into account the mapped memory, https://github.com/wasmerio/wasmer/blob/7e914488383e2c0f3086b1b3aa60f277d60f5095/lib/vm/src/mmap.rs#L283-L289.

Thanks for taking the time to check this.