Fix merge error and compile error and bugs in buildInitValue

[minjang]

My bad. I won't resolve merge conflicts in the github editor directly.

And I fixed/updated some code in buildInitValue. See the inlined comments.

[minjang]

Clang compiler complains this:

Operator '<<' has lower precedence than '-'; '-' will be evaluated first (fix available)clang(-Wshift-op-parentheses)

[ienkovich]

Thanks for the fix. Do you think we also need to change 1UL to 1ULL or use an explicit cast to uint64_t to have all compilers covered?

[minjang]

This would be a sort of INT_MIN. But super interestingly, the dump of initVal remained the same? But what do you think? I think this explicit - is clearer for signed int64_t?

[ienkovich]

Is the point here to always have negative int64_t passed to the attribute? The dump is similar because with your change you modify only those bits that don't matter for the used type.

[minjang]

I think lowest is clearer as well. Let me know this is okay for you.

[ienkovich]

I'm not sure what our semantics should be for corner cases like when all input elements are NaN. But I'd expect -INF to be returned instead of lowest() in case all inputs are -INF.

[minjang]

Yup, looking at more -INF looks good.

[ienkovich]

After thinking more thoroughly about corner cases, I realized we should use different init values for MAXIMUMF and MAXNUMF. To correctly cover all NaNs in input, we should use NaN as init value for MAXIMUMF (any not NaN would replace it then) and -INF for MAXNUMF (any NaN would replace it then). Similar for MINIMUMF and MINNUMF. What do you think?

[minjang]

@ienkovich Alright, we totally missed it. We can simply get the first element of the source (using vector::ExtractOp?). This would delete the entire buildInitValue .

I confirmed that the GPU also does like that: see ReduceOpConversion::accumulate

[ienkovich]

The goal is to choose init value so that the operation to combine it with the reduction result is NOP. I'm not sure the compiler would be able to optimize-out element extraction and its combination with the reduction result.

[minjang]

So, having a constant init value could result in highly optimized code after instruction combines, where the extreme case would be NOP? Do I correctly understand your idea? Right. Element-extraction is an indirection, so compiler is hard to optimize out. In that sense, having a constant init value is good! (extracting the first element would be a fallback then.)

Another question: should we consider NaN as input? How much is this important? I was thinking NaN was deemed an erroneous state in valid tensors? +/-INF are definitely valid. Hm, NaN can be compared.

[ienkovich]

There are unit tests that check kernel results for NaN inputs. Didn't see such a reduction though.

I'm currently improving the reduction lowering code to cover more cases. Will also fix the init value there.