Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update recurrent.py #1106

Merged
merged 8 commits into from
Oct 29, 2020
Merged

Update recurrent.py #1106

merged 8 commits into from
Oct 29, 2020

Conversation

Thinkre
Copy link
Contributor

@Thinkre Thinkre commented Oct 27, 2020

Checklist

  • I've tested that my changes are compatible with the latest version of Tensorflow.
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.

Motivation and Context

Build a dynamic RNN model in static mode for deploying. The dynamic RNN means outputs the last without-padding output.

Description

Although, the official document gives an example of building dynamic RNN model in eager mode, we want the model to be static for serving.

'''
data = tf.convert_to_tensor(data, dtype=tf.float32)
class DynamicRNNExample(tl.models.Model):
def init(self):
super(DynamicRNNExample, self).init()
self.rnnlayer = tl.layers.RNN(
cell=tf.keras.layers.SimpleRNNCell(units=6, dropout=0.1), in_channels=1, return_last_output=True,
return_last_state=True)
def forward(self, x):
z, s = self.rnnlayer(x, sequence_length=tl.layers.retrieve_seq_length_op3(x))
return z, s
model = DynamicRNNExample()
model.eval()
output, state = model(data)
'''
The current RNN layer cannot be built as a dynamic RNN layer for a static model, which is hard to save as trackable model for serving. I test the following code

'''
ni = tl.layers.Input(inputs_shape, name='input_layer')
seq = tl.layers.retrieve_seq_length_op3(ni)
cell = tf.keras.layers.LSTMCell(units=n_hidden, recurrent_dropout=0)
out = RNN(cell=cell,
return_last_output=True,
return_last_state=False,
return_seq_2d=True)(ni,sequence_length=seq)
nn = tl.layers.Dense(n_units=2, act=tf.nn.softmax, name="dense")(out)
model = tl.models.Model(inputs=ni, outputs=nn, name='rnn')
'''
which actually is built as a static RNN.

we force the RNN always to be dynamic to promote the accuracy.

Thinkre and others added 8 commits October 27, 2020 12:12
Motivation and Context

Build a dynamic RNN model in static mode for deploying. The dynamic RNN means outputs the last without-padding output.

Description

Although, the official document gives an example of building dynamic RNN model in eager mode, we want the model to be static for serving.

'''

data = tf.convert_to_tensor(data, dtype=tf.float32)
class DynamicRNNExample(tl.models.Model):
def init(self):
super(DynamicRNNExample, self).init()
self.rnnlayer = tl.layers.RNN(
cell=tf.keras.layers.SimpleRNNCell(units=6, dropout=0.1), in_channels=1, return_last_output=True,
return_last_state=True
)
def forward(self, x):
z, s = self.rnnlayer(x, sequence_length=tl.layers.retrieve_seq_length_op3(x))
return z, s
model = DynamicRNNExample()
model.eval()
output, state = model(data)
'''
The current RNN layer cannot be built as a dynamic RNN layer for a static model, which is hard to save as trackable model for serving. I test the following code

'''
ni = tl.layers.Input(inputs_shape, name='input_layer')
seq = tl.layers.retrieve_seq_length_op3(ni)
cell = tf.keras.layers.LSTMCell(units=n_hidden, recurrent_dropout=0)
out = RNN(cell=cell,
return_last_output=True,
return_last_state=False,
return_seq_2d=True)(ni,sequence_length=seq)
nn = tl.layers.Dense(n_units=2, act=tf.nn.softmax, name="dense")(out)
model = tl.models.Model(inputs=ni, outputs=nn, name='rnn')
'''

which actually is built as a static RNN.

we force the RNN always to be dynamic to promote the accuracy.
rm examples
rm examples
move the modification below sequence length check
@Laicheng0830 Laicheng0830 self-requested a review October 27, 2020 10:01
@Laicheng0830 Laicheng0830 merged commit 04a0801 into tensorlayer:master Oct 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants