Remove the max from the module's memory declaration.

[sunfishcode]

Currently, wasm modules declare a maximum linear memory size that they'll ever request via resize_memory.

However, applications that know how much memory they'll use up front can just allocate that much memory up front and avoid resizing. And applications that don't know how much memory they'll use up front won't often be able to declare a meaningful maximum.

Having a maximum value in wasm, and thus implementations that optimize based on the maximum value, will create an incentive for applications to declare a maximum even when there is no natural absolute maximum, or when they don't know what it is. This will encourage them to either declare uselessly high maximum values, or to needlessly crash on OOM when they do end up using more memory than their arbitrary maximum.

I recommend we remove the max value.

This PR also mentions that the initial memory size is rounded up to a page_size boundary.

[lukewagner]

This will encourage them to either declare uselessly high maximum values, or to needlessly crash
on OOM when they do end up using more memory than their arbitrary maximum.

Though initially being in favor of having a max, I find this argument convincing. This week I saw a lot of feedback from Unity's WebGL export (which currently requires the user to pick a fixed size) that picking any absolute memory size is a royal PITA for devs and does result in guess-and-check model which is not a formula for stability. Thus I would expect max to often be set super-conservatively by the tools (destroying its utility to the engine).

[titzer]

Why not just make it optional? It is useful information to the engine if
it's reliable.

On Thu, Sep 24, 2015 at 5:41 PM, Luke Wagner [email protected]
wrote:

This will encourage them to either declare uselessly high maximum values,
or to needlessly crash on
OOM when they do end up using more memory than their arbitrary maximum.

Though initially being in favor of having a max, I find this argument
convincing. I saw a lot of feedback from Unity's WebGL export (which
currently requires the user to pick a fixed size) that picking any absolute
memory size is a royal PITA for devs and does result in crashes. Thus I
would expect max to often be set super-conservatively (removing its utility
to the engine) by the tools.

—
Reply to this email directly or view it on GitHub
#370 (comment).

[sunfishcode]

It is optional already. But if engines optimize better when a max is present, then applications will be encouraged to opt in and declare a max, and this is error-prone.

[titzer]

That's a risk with all performance hints that can be provided by the user.
But an engine doesn't have to blindly follow bogus hints, either.
Concretely, an engine that preallocates address space for the maximum would
probably only do so if that request didn't seem patently absurd. Besides,
an engine will likely have to juggle multiple memories from multiple
modules in the future, so giving it some hints up front can avoid some
potentially severe performance problems or even fatal errors later. I'm
thinking about modules that are small and know they are small, like an
image decoder or encryption library that has a small linear memory inside
used as a buffer. If it has a good estimate of its (small) maximum size,
then it'd be useful to tell the engine so that it's not treated as if it
might suddenly request much more memory later.

On Thu, Sep 24, 2015 at 6:08 PM, Dan Gohman [email protected]
wrote:

It is optional already. But if engines optimize better when a max is
present, then applications will be encouraged to opt in and declare a max,
and this is error-prone.

—
Reply to this email directly or view it on GitHub
#370 (comment).

[sunfishcode]

By the wording in the design document, the max value is hard cap. Applications would OOM if they exceeded their max. This is the main thing I want to fix.

Changing the max to be a pure hint would be another option. However, would this be better than just having known-small applications pre-allocate the amount of space they need and avoiding ever calling resize_memory?

[lukewagner]

If the max-size is only valuable as a pre-allocation hint for smaller heap sizes (b/c we can't trust the large sizes) I'm doubting how often this would help in practice: how often do you need resizing but still know you have a small max size? If you need resizing, it seems likely that you are manipulating some dynamically-chosen amount of data and so you don't have an a priori max. I can see why it's valuable to know that an app never resizes, but that's a separate issue (and can either be observed implicitly by the absence of resize_memory or we could have some module header flag that enables resize_memory).

[ghost]

I ask that this be put on hold while the larger issues are considered, such as #331 I strong disagree with hobbling wasm performance just out of fear that developers will not invest the effort into working within the constraints that performance and a good user experience might demand. I believe there is a strong future for resource planning for wasm applications and that it can be opt-in.

I think it is premature to extrapolate from 'that picking any absolute memory size is a royal PITA for devs and does result in guess-and-check model which is not a formula for stability' feedback. This is also the result of the application making the decisions, rather than the runtime, and there being not negotiation, and perhaps the developers are just not experienced in resource planning and have not designed with this in might - but an enterprise database might plan operations in detail before execution.

For example, for a lot of users with adequate VM the runtime could have made the choice for the user and allocated the maximum. For example, for users on limited devices, the runtime might have classified this into a 'best run all-in' model and launched it in this mode for the user as a first choice allocate the maximum that the runtime chooses for this mode, while giving the user a secondary menu of other options.

Why does the runtime need to OOM if it hits a 'maximum'. Perhaps it needs more context on options, such as re-checking resources and offering the user choices. What about a transition from a wasm32 model to a wasm64 model which it might need a re-launch of the application which would be best done with cooperation from the application.

The issue of what the maximum memory means still seems to need more consideration if there is a 'commit' operation required before using memory.

[sunfishcode]

#331 or other issues can reintroduce a max if it makes sense in combination with a larger proposal. In the meantime, a plain max by itself is undesirable on its own.

[ghost]

@sunfishcode A maximum gives the runtime the option of allocating less memory, which seem technically desirable to me. Your proposal to remove the maximum is equivalent to applications requesting a large maximum. There are issues here to consider but I don't see this as an incremental step forward.

Your proposal also specifies that the minimum is the initial allocation, and there are good technical reasons why the choice of initial size should be up to the implementation. If the implementation has adequate VM, or is running 'all-in', then it should be able to allocate a large initial size and if also fixed then specify this size to the application at compile time so that the application can specialize on a fixed size.

[jfbastien]

I agree with @titzer and would like to keep the optional maximum.

Also:

The initial size of linear memory is the requested size rounded up to the nearest whole multiple of page_size.

That's a bit too much of a guarantee. I'd require that the value be a multiple of page_size, but not guarantee that it's necessarily rounded to the nearest multiple of page_size. I'd drop nearest.

[lukewagner]

I'm fine postponing this discussion of removing max until we get closer to releasing wasm so that we can poll engines and find out if, given the caveats above, anyone is actually optimizing based on max (other than just as a proxy for "is resizing disabled"). Because, if not, "might be useful" isn't sufficient justification given that we can always add later if/when it becomes actually useful.

[jfbastien]

Closing the PR in favor of issue #372.

[lukewagner]

There is still the useful hunk of rounding up initial size to some multiple of page_size. I'd be curious to hear the use case for rounding up to something bigger than page_size given that the whole point of page_size atm is to define the quanta of memory_size. I was actually thinking that we might want different quantas for total memory size vs. the operands to future functions like map_file (due to ARM immediate limitations for the former not present in the latter) and so maybe page_size should be renamed to memory_size_quanta to reserve page_size for later.

[ghost]

@lukewagner For example, on v8 that does not want to emit resizing which destroys performance - the runtime can choose a large initial and final size and achieve better performance. For example, rounding up to a size that suits index masking. The wasm spec should not guarantee a particular initial size so that code is not written to depend on it rather just that the initial size is at least the minimum requested.

[lukewagner]

@titzer Is that (what @JSStats said) something you actually want to do? Preemptively inflating the heap size up to max or some other intermediate large value seems like a significant new source of nondeterminism that will hurt portability.

[jfbastien]

I'd instead allow the allocator to tell the programmer what the actual allocated size is (you asked for X, I have you X+Y!). It's nice to overallocate for a bunch of reasons (hard-coding bounds in the code, nicer bounds checking, potentially using bigger pages sizes to make the page table and TLB happy, ...).

[ghost]

@lukewagner Runtimes that support multi-threaded code might also have a need for at least a fixed VM reserve? If the VM is not already reserved then a new buffer might need to be allocated which would probably require stopping all threads, plus some convention in the code for 'safe' points at which to stop (when there are no pointers into the buffer) and conventions for not holding buffer pointers across calls etc, and this all adds a lot of complexity to a compiler and runtime. If there is a VM reserve then this can be usefully communicated to the application at compile time and might as well be the memory size, and a 'commit' operation added to make more of this reserve available.

[lukewagner]

@JSStats Right, that all would make sense in the impl which can be done w/ or w/o any rounding of the initial, semantically visible, memory_size. We're talking about defining the latter here.