[Frontend, Tensorflow2] Adding TF2 frontend code with support for control flow ops #8142

rohanmukh · 2021-05-26T19:06:49Z

This PR introduces a new parser tensorflow2.py. To keep the code separate from original parser code and introduce TF2 related changes, much of the existing code is ported into this frontend code. This commit follows the original plan of commits as proposed to support TF2 frontend in relay here: #4102. Exact commit plan here.

Some important points on introducing the TF2 frontend:

All of the existing ops as available in tensorflow.py has been reused.
Support for ops like PartitionedCall, StatelessPartitionedCall invoked as functional nodes.
The control flow logic in TF1 is not used, and in that high level control flow structures like If, While, StatelessWhile has been introduced which are invoked as functional node.

Some points in testing:

All of the unit tests available in tests/python/frontend/tensorflow2/ were used in evaluating the new parser. Previous unit tests cover the support for ops like Add2D, Relu etc. in test_functional.py and some small end-to-end models in test_sequential.py.
Additional tests to test the control flow structures have been introduced in this PR in test_functional.py for ops like If, While and StatelessWhile.

Co-authored-by: David Huang [email protected]
Co-authored-by: Rohan Mukherjee [email protected]
Co-authored-by: Srinidhi Goud [email protected]
Co-authored-by: Xingyu Zhou [email protected]
Co-authored-by: Xiao [email protected]

Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

Co-authored-by: David Huang <[email protected]> Co-authored-by: Rohan Mukherjee <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]>

rohanmukh · 2021-06-02T18:42:37Z

@trevor-m @yongwww @zhiics @weixa @zxy844288792 @dvhg @srkreddy1238

rohanmukh · 2021-06-02T19:40:15Z

tests/python/frontend/tensorflow2/common.py

@@ -23,8 +23,7 @@

 from tvm.runtime.vm import VirtualMachine
 import tvm.contrib.graph_executor as runtime
-from tvm.relay.frontend.tensorflow import from_tensorflow
-
+from tvm.relay.frontend.tensorflow2 import from_tensorflow


Previously the existing parser from relay/frontend/tensorflow.py was being used to test the basic TF2 ops, functions and models. From this PR, all of those test as well as the new ones are being passed through the new frontend parser for TF2, namely relay/frontend/tensorflow2.py

rohanmukh · 2021-06-02T19:45:17Z

python/tvm/relay/frontend/tensorflow2.py

+    else:
+        sym = [_expr.var(node.name, shape=input_shape, dtype=attr["dtype"].name)]
+    return input_shape, sym
+


The utility functions on top are imported from tensorflow.py. Since they were methods inside class they had to extracted. Future refactor PR can address this to allow better code reuse. However all TF1 unit tests and models need to be tested alongside for such a refactor.

rohanmukh · 2021-06-02T19:46:28Z

@srinidhigoud

yongwww · 2021-06-03T07:58:14Z

python/tvm/relay/frontend/tensorflow2.py

+from tensorflow.python.framework import function_def_to_graph
+from tensorflow.python.framework import tensor_util
+from tensorflow.python.framework import dtypes


could be merged

yongwww · 2021-06-03T08:02:21Z

tests/python/frontend/tensorflow2/test_functional_models.py

+            def triple(x):
+                return 3 * x
+
+            cond = True


add case for False

yongwww · 2021-06-03T08:10:44Z

python/tvm/relay/frontend/tensorflow2.py

+        // tensorflow/core/framework/attr_value.proto
+        message AttrValue {
+            oneof value {
+                bytes s = 2;                 // "string"
+                int64 i = 3;                 // "int"
+                float f = 4;                 // "float"
+                bool b = 5;                  // "bool"
+                DataType type = 6;           // "type"
+                TensorShapeProto shape = 7;  // "shape"
+                TensorProto tensor = 8;      // "tensor"
+                ListValue list = 1;          // any "list(...)"            }
+        }


post a link instead of pasting the proto def should be good

yongwww · 2021-06-03T08:13:34Z

python/tvm/relay/frontend/tensorflow2.py

+    """
+    fields = ["s", "i", "f", "b", "type", "shape", "tensor", "func"]
+
+    x = buf


use buf directly?

yongwww · 2021-06-03T08:16:42Z

python/tvm/relay/frontend/tensorflow2.py

+    return attrs
+
+
+def convert_place_holder(shape, node, in_type=None):


Suggested change

def convert_place_holder(shape, node, in_type=None):

def convert_placeholder(shape, node, in_type=None):

yongwww · 2021-06-03T08:17:01Z

python/tvm/relay/frontend/tensorflow2.py

+
+
+def convert_place_holder(shape, node, in_type=None):
+    """convert tf place holder into relay var.


Suggested change

"""convert tf place holder into relay var.

"""convert tf placeholder into relay var.

yongwww · 2021-06-03T08:18:01Z

python/tvm/relay/frontend/tensorflow2.py

+        self.mod = IRModule({})  # relay function and type definitions. defined in tvm/ir/module.py
+        self.params = {}  # for constants (weights) in the entire relay module
+        self.prelude = Prelude(self.mod)  # relay.prelude needed for tensorlist ops


remove the comments?

yongwww · 2021-06-03T08:22:36Z

python/tvm/relay/frontend/tensorflow2.py

+
+        Parameters
+        ----------
+        op_name : str


add definition for graph, node_name as well

yongwww · 2021-06-03T08:24:24Z

python/tvm/relay/frontend/tensorflow2.py

+from ..loops import while_loop as _while_loop
+from .common import infer_type as _infer_type
+
+from .tensorflow import _convert_map as _convert_map_tf1


Suggested change

from .tensorflow import _convert_map as _convert_map_tf1

from .tensorflow import _convert_map as _convert_map_common

yongwww · 2021-06-03T08:26:07Z

python/tvm/relay/frontend/tensorflow2.py

+        None,
+    )
+    loop_inputs = convert_vars(inputs, while_func.signature.input_arg)
+    # in_shapes = nodes[node_name].attr["output_shapes"].list.shape


Suggested change

# in_shapes = nodes[node_name].attr["output_shapes"].list.shape

yongwww · 2021-06-03T08:28:58Z

@kevinthesun

rohanmukh · 2021-06-03T17:33:47Z

Thanks for the comments @yongwww. I addressed the changes in the last commit.

trevor-m

LGTM

zhiics

lgtm

zhiics · 2021-06-08T00:44:35Z

Thanks @yongwww @trevor-m

rohanmukh · 2021-06-08T21:00:21Z

Thanks @zhiics @yongwww @trevor-m

…trol flow ops (apache#8142) * adding tf control flow ops with a different frontend code Co-authored-by: David Huang <[email protected]> Co-authored-by: Rohan Mukherjee <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]> * Some minor fixes * Fixing output order in TF2 outputs Co-authored-by: David Huang <[email protected]> Co-authored-by: Rohan Mukherjee <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]> * Using black * Refactoring Co-authored-by: David Huang <[email protected]> Co-authored-by: Rohan Mukherjee <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]> * resolving a bug with passing output tensors for Functional Graphs * fixing multi output for graph runtime * adding docstring edits * linting + black * linting + black * linting + black * removing unnecessary output propagation across function * addressed comments in PR Co-authored-by: David Huang <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]>

rohanmukh and others added 6 commits May 26, 2021 18:59

Some minor fixes

d2a7fd9

Fixing output order in TF2 outputs

4a75a5e

Co-authored-by: David Huang <[email protected]> Co-authored-by: Rohan Mukherjee <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]>

Using black

ba8517d

Refactoring

d720fe3

Co-authored-by: David Huang <[email protected]> Co-authored-by: Rohan Mukherjee <[email protected]> Co-authored-by: Srinidhi Goud <[email protected]> Co-authored-by: Xingyu Zhou <[email protected]> Co-authored-by: Xiao <[email protected]>

resolving a bug with passing output tensors for Functional Graphs

95ce443

rohanmukh marked this pull request as ready for review June 1, 2021 22:29

fixing multi output for graph runtime

b07a58f

rohanmukh changed the title ~~[WIP] adding tf2 control flow ops with a different frontend code~~ [Frontend][Tensorflow2] adding tf2 control flow ops with a different frontend code Jun 2, 2021

adding docstring edits

897af9f

FrozenGene added the status: need review label Jun 2, 2021

rohanmukh added 4 commits June 2, 2021 01:41

linting + black

54fcdbd

linting + black

d051b46

linting + black

9b26ba2

removing unnecessary output propagation across function

fdb72c5

rohanmukh commented Jun 2, 2021

View reviewed changes

rohanmukh changed the title ~~[Frontend][Tensorflow2] adding tf2 control flow ops with a different frontend code~~ [Frontend, Tensorflow2] adding tf2 control flow ops with a different frontend code Jun 2, 2021

rohanmukh changed the title ~~[Frontend, Tensorflow2] adding tf2 control flow ops with a different frontend code~~ [Frontend, Tensorflow2] Adding TF2 frontend code with support for control flow ops Jun 2, 2021

yongwww reviewed Jun 3, 2021

View reviewed changes

addressed comments in PR

c683c96

trevor-m approved these changes Jun 7, 2021

View reviewed changes

zhiics approved these changes Jun 8, 2021

View reviewed changes

zhiics merged commit 64a8e81 into apache:main Jun 8, 2021

zhiics removed the status: need review label Jun 8, 2021

zhiics added the status: accepted label Jun 8, 2021

junrushao mentioned this pull request Nov 1, 2021

Apache TVM v0.8 Release Note Candidate #9416

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Frontend, Tensorflow2] Adding TF2 frontend code with support for control flow ops #8142

[Frontend, Tensorflow2] Adding TF2 frontend code with support for control flow ops #8142

rohanmukh commented May 26, 2021 •

edited

Loading

rohanmukh commented Jun 2, 2021

rohanmukh Jun 2, 2021

rohanmukh Jun 2, 2021

rohanmukh commented Jun 2, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww Jun 3, 2021

yongwww commented Jun 3, 2021

rohanmukh commented Jun 3, 2021

trevor-m left a comment

zhiics left a comment

zhiics commented Jun 8, 2021

rohanmukh commented Jun 8, 2021

		return attrs


		def convert_place_holder(shape, node, in_type=None):

	def convert_place_holder(shape, node, in_type=None):
	def convert_placeholder(shape, node, in_type=None):



		def convert_place_holder(shape, node, in_type=None):
		"""convert tf place holder into relay var.

	"""convert tf place holder into relay var.
	"""convert tf placeholder into relay var.

	from .tensorflow import _convert_map as _convert_map_tf1
	from .tensorflow import _convert_map as _convert_map_common

[Frontend, Tensorflow2] Adding TF2 frontend code with support for control flow ops #8142

[Frontend, Tensorflow2] Adding TF2 frontend code with support for control flow ops #8142

Conversation

rohanmukh commented May 26, 2021 • edited Loading

rohanmukh commented Jun 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rohanmukh commented Jun 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yongwww commented Jun 3, 2021

rohanmukh commented Jun 3, 2021

trevor-m left a comment

Choose a reason for hiding this comment

zhiics left a comment

Choose a reason for hiding this comment

zhiics commented Jun 8, 2021

rohanmukh commented Jun 8, 2021

rohanmukh commented May 26, 2021 •

edited

Loading