SpectralConv2D Implementation

[kelvinkoh0308]

Hi
I think there is something wrong in you SpectralConv2D

`def compute_spectral_norm(self):
for _ in range(self.power_iterations):
k = tf.reshape(self.kernel, (-1, self.filters))

        new_v = l2normalize(tf.matmul(k, self.u))
        new_u = l2normalize(tf.matmul(tf.transpose(k), self.v))
        
    sigma = tf.matmul(tf.matmul(tf.transpose(new_v), k), new_u)
    new_kernel = tf.divide(self.kernel, sigma)

    return new_v, new_u, self.kernel


def call(self, inputs):
    new_v, new_u, new_kernel = self.compute_spectral_norm()
    outputs = self._convolution_op(inputs, new_kernel)

    if self.use_bias:
        if self.data_format == 'channels_first':
                outputs = tf.nn.bias_add(outputs, self.bias, data_format='NCHW')
        else:
            outputs = tf.nn.bias_add(outputs, self.bias, data_format='NHWC')
    if self.activation is not None:
        return self.activation(outputs)

    return outputs + [new_v, new_u, new_kernel]`

Inside the compute_spectral_norm, you did not update anything and return back the old kernel.
Inside the call function, the
if self.activation is not None:
will always be true (even though we pass None to the activation function, tensorflow internally will treat this as a linear function) thus this will just become a normal convolution.