Keras layer wrappers don't supported

[ragatti]

An current layer of Keras is not supported:
ValueError: Please initialize Prune with a supported layer. Layers should either be a PrunableLayer instance, or should be supported by the PruneRegistry. You passed: <class 'tensorflow.python.keras.layers.wrappers.TimeDistributed'>

[raziel]

Hi. Can you please include the snippet of code you're using to create the model?
Thanks

[ragatti]

I m sorry, the problem is associated with TimeDistributed wrapper around DepthwiseConv2D in Tensorflow >= 1.13 (issue #29438), i.e. tensorflow >= 1.14 is required to run tensorflow-model-optimization. Code example:

from tensorflow.keras import backend as K
from tensorflow_model_optimization.sparsity import keras as sparsity
import tensorflow as tf

K.set_image_data_format("channels_first")
print(K.image_data_format())

pruning_params = {
      'pruning_schedule': sparsity.PolynomialDecay(initial_sparsity=0.50,
                                                   final_sparsity=0.90,
                                                   begin_step=2000,
                                                   end_step=4000,
                                                   frequency=100)
}


l = tf.keras.layers

model = tf.keras.Sequential([
    l.TimeDistributed(sparsity.prune_low_magnitude(
        l.DepthwiseConv2D(depth_multiplier=1,
                        kernel_size=(1, 4), 
                        strides=(1, 1)), **pruning_params), 
    input_shape=(None, 1, 128, 8))
])

However, the TimeDistributed layer is not supported by sparsity.prune_low_magnitude() when the model is changed as shown below:

model = tf.keras.Sequential([
    sparsity.prune_low_magnitude(l.TimeDistributed(
        l.DepthwiseConv2D(depth_multiplier=1,
                        kernel_size=(1, 4), 
                        strides=(1, 1)), 
    input_shape=(None, 1, 128, 8)),**pruning_params)
])

ValueError: Please initialize `Prune` with a supported layer. Layers should either be a `PrunableLayer` instance, or should be supported by the PruneRegistry. You passed: <class 'tensorflow.python.keras.layers.wrappers.TimeDistributed'>

[leochan2009]

same problem here with Timedistribute layer in model optimization. any progress?

[drubinstein]

Similarly, I see this issue for the ClusterRegistry,

"ValueError: Please initialize `Cluster` with a supported layer. Layers should either be a `ClusterableLayer` instance, or should be supported by the ClusteringRegistry. You passed: <class 'tensorflow.python.keras.engine.sequential.Sequential'>

[teijeong]

Hi @nutsiepully, can you update?

[liyunlu0618]

It's intended behavior. The pruning wrapper currently doesn't check recursively for prunable layers. Since you have a work-around of adjusting the order of wrappers, will close this for now. Thanks!

[kareemali56]

I am encountering this error in my machine learning model. Any idea??

ModuleNotFoundError Traceback (most recent call last)
Cell In[1], line 49
47 from keras.callbacks import EarlyStopping
48 import random
---> 49 from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
56 #from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
57 from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor

ModuleNotFoundError: No module named 'tensorflow.keras.wrappers'