[tvm][any] broadcast with values other than one

[zhiics]

This PR makes relay Any type broadcast with values other than one, i.e.:

x = relay.var('x', shape=(3, 2), dtype="float32")
y = relay.var('y', shape=(relay.Any(), 2), dtype="float32")
relay.add(x, y)

It makes all symbolic vars as auto_broadcast if they haven't been bound to compact buffer before.

@icemelon9 @yzhliu @tqchen @jroesch

[jroesch]

I am a bit confused by the semantics, how does this work? for example imagine that I have two any dimensions which will be (say 20, and 15 at runtime) it doesn't make sense to broad cast against non-1 dimensions.

[zhiics]

@jroesch sorry. I think the example I gave missed something. I updated it.

[zhiics]

close for now. need more consideration about the auto_broadcast binding.

[junrushao]

Hey this is super helpful!

Btw, I notice that in the compile engine it assumes that checked type exists. Does it mean that for now we are only able to deal with the case that dims are fixed and certain dimensions are Any?

[zhiics]

@junrushao1994 Yes, you are right, only the bound is dynamic but rank is fixed.

[kevinthesun]

It will generate runtime error when actual shape is incompatible?

[zhiics]

@kevinthesun Yes, both the interpreter and VM would fail if the real values are incompatible. I can probably add a few tests for them as well.

[icemelon]

It will generate runtime error when actual shape is incompatible?

The shape function will perform the type checking at runtime (see https://github.com/dmlc/tvm/blob/master/python/tvm/relay/op/_tensor.py#L130).
However, I think there's no type checking in the TOPI generated kernels.

[zhiics]

@icemelon9 yeah, that might be a different between Any and tvm symbolic var.

[yzhliu]

Is it possible to have a pass replace attr::buffer_bind_scope ? current impl seems to be a little bit adhoc.

[zhiics]

@yzhliu Yeah, I actually thought about replacing it as well. But I ended up skipping it because it looks there is not much difference. I think we probably want to keep compact memory as much as possible because auto broadcast will bring in the if_then_else guard which may not as performant as the compact version. How do you think?

[zhiics]

@icemelon9 @yzhliu @kevinthesun I think we've probably forgotten this PR.

[junrushao]

Is this ready for review?

[zhiics]

@junrushao1994 yes

[yzhliu]

@yzhliu Yeah, I actually thought about replacing it as well. But I ended up skipping it because it looks there is not much difference. I think we probably want to keep compact memory as much as possible because auto broadcast will bring in the if_then_else guard which may not as performant as the compact version. How do you think?

so it won't work for those ops written in hybrid script, right?

[zhiics]

@yzhliu Thanks. Removed the binding in hybrid script as it will go through lower anyway.

[icemelon]

lgtm