v128.load32_zero and v128.load64_zero instructions

[Maratyszcza]

Introduction

This PR introduce two new variants of load instructions which load a single 32-bit or 64-bit element into the lowest part of 128-bit SIMD vector and zero-extend it to full 128 bits. These instructions natively map to SSE2 and ARM64 NEON instruction, and have two broad use-cases:

1. Non-contiguous loads, when we need to combine elements from disjoint locations in a single SIMD vector. Non-contiguous loads are commonly emulated by doing loads a single elements and then combining the values through shuffles. While is it possible to do through a combination of scalar loads and v128.replace_lane instructions, the resulting code would use be inefficient in using too many general-purpose registers, producing an overly long dependency chain (every v128.replace_lane depends on the previous one), and hitting the long-latency/low-throughput instructions to copy from general-purpose registers to SIMD registers. Non-contiguous loads using the proposed v128.load32_zero and v128.load64_zero instructions avoid all these bottlenecks.
2. Processing fewer than 128 bits of data. Sometimes the algorithm or data structures just don't expose enough data to utilize all 128 bits of a SIMD vector, but would nevertheless benefit from processing fewer elements in parallel (e.g. adding 8 bytes in one SIMD instruction rather than eight scalar instructions).

Applications

• libgav1 (AV1 decoder)
• x264 encoder: sum of transformed differences (processing less than 16 elements)
• x264 encoder: deblocking (non-contiguous load)
• QDPXX/libintrin: matrix-vector multiplication (non-contiguous load)
• QNNPACK: depthwise convolution (processing less than 8 elements)
• cpp Wilson Dslash (non-contiguous load)
• GROMACS: matrix transpose (non-contiguous load)
• OpenBLAS: matrix-matrix multiplication

Mapping to Common Instruction Sets

This section illustrates how the new WebAssembly instructions can be lowered on common instruction sets. However, these patterns are provided only for convenience, compliant WebAssembly implementations do not have to follow the same code generation patterns.

x86/x86-64 processors with AVX instruction set

• v128.load32_zero
  • v = v128.load32_zero(mem) is lowered to VMOVSS xmm_v, [mem]
• v128.load64_zero
  • v = v128.load64_zero(mem) is lowered to VMOVSD xmm_v, [mem]

x86/x86-64 processors with SSE2 instruction set

• v128.load32_zero
  • v = v128.load32_zero(mem) is lowered to MOVSS xmm_v, [mem]
• v128.load64_zero
  • v = v128.load64_zero(mem) is lowered to MOVSD xmm_v, [mem]

ARM64 processors

• v128.load32_zero
  • v = v128.load32_zero(mem) is lowered to LDR Sv, [mem]
• v128.load64_zero
  • v = v128.load64_zero(mem) is lowered to LDR Dv, [mem]

ARMv7 processors with NEON instruction set

• v128.load32_zero
  • v = v128.load32_zero(mem) is lowered to VMOV.I32 Qv, 0 + VLD1.32 {Dv_lo[0]}, [mem]
• v128.load64_zero
  • v = v128.load64_zero(mem) is lowered to VMOV.I32 Dv_hi, 0 + VLD1.32 {Dv_lo}, [mem]

[tlively]

Thanks for the suggestion, @Maratyszcza! Now that we are in phase 3, we have stricter guidelines on adding new instructions. It sounds like these instructions are well supported on multiple architectures, but we need to agree that they are used in multiple important use cases and that they would be expensive to emulate. Can you point to real-world uses of this pattern that we could adapt as benchmarks to determine how much of a benefit these instructions would be?

[Maratyszcza]

@tlively Added examples of applications using these instructions

[Maratyszcza]

XNNPACK has SIMD table-based exp and sigmoid implementations that could be used for evaluation

[jan-wassenberg]

@tlively I agree these would be helpful. Another expensive to emulate use case is when you have an existing data structure of 1-2 floats and we can't be sure 4 floats are accessible. Or the much more common case of remainder handling - using the same code as the main loop, but with 32-bit loads/stores going one element at a time. JPEG XL has several examples of this.

[lemaitre]

Maybe the more general vld1q_lane from Neon might be desirable (https://static.docs.arm.com/den0018/a/DEN0018A_neon_programmers_guide_en.pdf#G15.1154120)

Basically, it can loads a single element into any lane, not just the first one, and leaves the other lanes untouched.

The problem I see is that it would need some kind pattern matching to make this the most efficient on x86 where we only have "load into the first lane and put the rest at zero".
But we can envision that a sequence of load_lane could be converted into shuffles in SSE and even a (masked) gather in AVX2.

If pattern recognition fails (or is disabled), the generated code for a single load_lane would still be faster than scalar load + insert_lane as it would be converted into loadl + shuffle and stay in the SIMD register space.
And the shuffle can be easily eliminated if the WASM runtime detect that the lane index is 0 and the input vector already contains zeros.

---

The store counterpart might also be interesting.
However, the store variant will not be able to handle 8- and 16-bit types efficiently on x86.
We can stay with 32- and 64-bit types, as proposed here, though.

[tlively]

For consistency with the load_splat instructions, these instructions should probably have v32x4 and v64x2 prefixes. More descriptive names might be v32x4.load_lane and v64x2.load_lane.

[Maratyszcza]

IMO it is best to save load_lane for (future) variants which load a single lane while leaving other unchanged (i.e. analogs of vld1q_lane_XX in ARM NEON and _mm_insert_epiXX on x86).

We could rename these instructions to v128.load32_u and v128.load64_u for consistency with v128.load32x2_u and other zero-extending instructions.

[tlively]

Saving load_lane for potential future instructions makes sense to me. How about v32x4.load32 and v64x2.load64? The _u suffix doesn't seem necessary because there is no sign interpretation happening. I still think it makes sense to use the prefixes for hinting at the lane interpretation, but I could probably be convinced otherwise as well.

On a different note, prototypes of these instructions have been merged to both LLVM and Binaryen and will be available in the next version of Emscripten via the builtin functions __builtin_wasm_load32_zero and __builtin_wasm_load64_zero.

[ngzhian]

I think the memory instructions should all start with v128. (Ref: mvp instructions are all of the form <type>.load[<n>_<sx>].)

The shape prefix suggests how the operands are treated, which doesn't apply for loads, since the operands are all memargs. This might be a point of confusion. Making everything start with v128.load_ will help categorize all these variants of load as: "load from memory to get a v128", i.e. these are all the ways you can load something from memory to get a v128. Then the format becomes:

v128.load_<splat/extend/zero/others>_<numberofbytesloaded>_<sign extension>"

For load_splat we might even consider change it to: v128.load_splat8, similar to how we have i32.load8_s.

So maybe load zeroes can be: v128.load_zero32.

I think this has a (imo nice) side effect of making the spec text a bit clearer, because you can now say, all instructions that start with the shape prefix describe how they treat their operands (and you don't have to say "except for memory instructions").

[tlively]

@ngzhian That seems reasonable and consistent to me. We would want to use the v128 prefix for load-extend operations as well. I think it would look more consistent with MVP if we put <numberofbytesloaded> after load, like in v128.load8_splat or v128.load32_zero. WDYT?

[ngzhian]

Yea that looks good to me. It becomes really clear from the name that v128 is the return type, and how many bytes will be loaded. The remaining portion will be to tell us how to get from bytes to v128, of which we can have many different wants.

[tlively]

@Maratyszcza we briefly discussed this in the sync meeting today, and there is general support for these instructions, but we still need benchmarking data to make the case for including them. Would you be able to get performance numbers for these?

[omnisip]

In this case, it would be the same operation with a register instead of a memory argument. There's no undefined behavior.

…

On Mon, Oct 5, 2020, 18:13 Thomas Lively ***@***.***> wrote: @omnisip <https://github.com/omnisip> that sounds like v128.const, but leaving some lanes undefined. Unfortunately we can't have instructions that leave parts of values undefined in WebAssembly because we can't have undefined behavior and we don't want to make the type system so complex that it can track undefined lanes statically. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKQJQF2UM2HZL24ORTJRLSJJOLBANCNFSM4NREEJEQ> .

[tlively]

How is that different from v128.const then?

[omnisip]

The value is calculated at runtime but isn't a pointer in memory.

…

On Mon, Oct 5, 2020, 20:03 Thomas Lively ***@***.***> wrote: How is that different from v128.const then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKQJVRHEU5APTQCBL5DGLSJJ3FLANCNFSM4NREEJEQ> .

[tlively]

Then it sounds like a i64x2.replace_lane 0 I guess that requires a full vector to be materialized first, though.

In general, the answer to "why doesn't the instruction set have X" is some combination of "X is not portable enough," "X is not useful enough," or "no one has suggested X yet". You're totally welcome to suggest new instructions when you identify deficiencies in the instruction set. Useful information to include is how the instruction would be lowered on Intel and ARM ISAs, applications that would benefit from the instruction, and any estimates you have for the performance improvement that instruction can bring. See many of @Maratyszcza's PRs for good examples of new instruction proposals.

[omnisip]

In this case, it would belong with this PR because it would still be the same underlying instructions with a different argument. @Maratyszcza do you want me to make a patch or do you want to add this yourself?

[tlively]

WebAssembly doesn't overload instructions with different kinds of arguments like that, so it will have to be a new instruction proposal.

[omnisip]

There are three identical implementations for this functionality that are useful: 1) from memory like this currently is 2) from 32 or 64 bit register 3) from another v128 vector I'll put the other two in my proposal so they can be evaluated together when nomenclature is discussed.

…

On Mon, Oct 5, 2020, 20:56 Thomas Lively ***@***.***> wrote: WebAssembly doesn't overload instructions with different kinds of arguments like that, so it will have to be a new instruction proposal. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKQJTHVPROKEOKOAS7OQTSJKBOHANCNFSM4NREEJEQ> .

[penzn]

Why doesn't this instruction set have a non-memory load? E.g. to load into the lower part without pulling from memory?

Memory operations either go from memory to a stack slot (load) or from stack slot to memory (store). There are no other possible semantics for memory operations in WebAssembly.

(edit: had virtual registers instead of stack slots, my bad)

[ngzhian]

In the sync today we agreed to add these 2 instructions.
@Maratyszcza can you please update https://github.com/WebAssembly/simd/blob/master/proposals/simd/NewOpcodes.md as well? Put it in the table of memory instructions.

[Maratyszcza]

@ngzhian Done

[ngzhian]

Thanks, LGTM