Support Loop Translation to TensorRT

[narendasan]

In order to fully support RNNs, we want to be able to make recurrent subgraphs to TRT. To do so we must expand the capability of the compiler to recognize these cases and generate a TensorRT Loop.

• Relevant TRT Documentation:

  • https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#define-loops
  • https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_loop.html

• Relevant PyTorch Documentation: https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/OVERVIEW.md#loops

[abhi-iyer]

Some more issues we can address with this issue:

1. Can we expand loop translation when unrolling the loop (see Lowering pass)?

2. Can we backtrace through the graph and freeze and frozen operators?

[narendasan]

To add on, we want to be able to handle types like int inside the loop block at runtime. This will require probably doing things like mapping functions that operate on identified recurrence objects to a custom namespace tensor instead of int instead of standard aten to engage a converter instead of a evaluator and at conversion time freezing external dependencies in the TRT graph.