Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support nested layer customization and enable better 'in_channels' exception raise #1015

Merged
merged 12 commits into from
Jul 8, 2019

Conversation

ChrisWu1997
Copy link
Member

Checklist

  • I've tested that my changes are compatible with the latest version of Tensorflow.
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.

Motivation and Context

  1. Support nested layer customization.
  2. Enable better 'in_channels' exception raise in dynamic model definition.

Description

I briefly list the design below just for a reminder.

  • Nested layer customization
    • Logic
      Each layer maintain a private attribute self._layers to store possible nested layer. When a Layer is registered as an attribute of another Layer, it is automatically added to the second layer's self._layers. Weights are collected recursively.
      Inner layer set to be unaware of LayerNode concept because it does not need to be involved in the model node graph. Inner layer is only regarded as the outer layer‘s 'private' attribute, which is not known by the model.
    • Example
              class MyLayer(tl.layers.Layer):
      
              def __init__(self, name=None):
                  super(MyLayer, self).__init__(name=name)
                  self.input_layer = tl.layers.Dense(n_units=20)  # it's ok not to pass `in_channels` here
                  self.build(None)
                  self._built = True
      
              def build(self, inputs_shape=None):
                  self.W = self._get_weights('weights', shape=(20, 10))
      
              def forward(self, inputs):
                  inputs = self.input_layer(inputs)
                  output = tf.matmul(inputs, self.W)
                  return output
      MyLayer would contain three weights: W of MyLayer, W and b of the inner Dense.
  • 'in_channels' exception in dynamic model
    Now it is checked when setting Layer as an attribute of dynamic Model. If in_channels not provided, there will be an exception raised, e.g.
    The registered layer `dense_4` should be built in advance. Do you forget to pass the keyword argument 'in_channels'?
    

@ChrisWu1997 ChrisWu1997 merged commit 1477d57 into tensorlayer:master Jul 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants