PyTorch frontend: make type inference incremental

[t-vi]

Currently, the PyTorch frontend will use the vanilla TVM type inference pass to get the types.
Combined with the incremental nature of translating the graph, this makes for quadratic (in the graph size) runtime.
This patch runs type inference on a subgraph (starting from the things that has types) instead. It is a bit hacky though because it essentially tries to implement in-place modification where TVM does not foresee it.

For converting Huggingface BERT, this patch gives a ~10x increase in the speed of conversion (from 31 seconds to just below 3), so it is solving a very real and quite bad problem.

[t-vi]

The other question is whether we would want to make the translator a use a class interface so that the type inference can directly access the outputs' types.

[t-vi]

https://discuss.tvm.apache.org/t/incremental-type-propagation/7446/10

[masahi]

The other question is whether we would want to make the translator a use a class interface so that the type inference can directly access the outputs' types.

Yes we can consider this approach. Initially I thought function-only approach would be cleaner (e.g. for recursively converting blocks in If and Loop) but this resulted in passing various constants such as convert_map, prelude, default_dtype etc around each function.

We can introduce a global object (like a class) to hold these constants.

[t-vi]

@masahi @siju-samuel I think this is ready for review now. In BERT conversion, I get a 10x speedup for from_python. In addition to removing the N² problem, it prunes the module before type inference, which seems essential for prelude.

[masahi]

@t-vi Please have a look at the CI issue. It is due to the recent change I made.

[t-vi]

Oh, right, an undetected merge confict. Fixed. Thank you @masahi.

[masahi]

Thanks @t-vi

[t-vi]

Thank you @masahi, for the guidance and discussion, and review!