Add a warning for mismatched inputs structure in Functional

[james77777778]

There are several issues related to mismatched inputs structure:

• This works in keras 2.15 with TensorFlow 2.15 but does not 3.5 with TensorFlow 2.17: multiple lookup layer types in inputs #20166
• Keras 3 doesn't map dictionary inputs by "key"  #20136
• A possible bug with tf 2.17.0 #20086
  and more...

As a result, it would be beneficial to add a warning for users.
Ultimately, we might want to raise an error when a mismatch occurs. Otherwise, it could lead to subtle issues if the inputs have the same shape and dtype, as the computation could be incorrect even though the code runs.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.34%. Comparing base (3cc4d44) to head (dddfbfe).

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #20170   +/-   ##
=======================================
  Coverage   79.34%   79.34%           
=======================================
  Files         501      501           
  Lines       47319    47325    +6     
  Branches     8692     8694    +2     
=======================================
+ Hits        37544    37550    +6     
  Misses       8017     8017           
  Partials     1758     1758           

Flag	Coverage Δ
keras	79.19% <100.00%> (+<0.01%)	⬆️
keras-jax	62.47% <100.00%> (+<0.01%)	⬆️
keras-numpy	57.65% <100.00%> (+0.07%)	⬆️
keras-tensorflow	63.85% <100.00%> (+<0.01%)	⬆️
keras-torch	62.50% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fchollet]

Thanks for the improvement -- LGTM

[ThorvaldAagaard]

After upgrading Keras to latest version I am getting the warning, but can't figure out, what is wrong with the input

This is my code

and this is the output:

[<KerasTensor shape=(None, 42), dtype=float16, sparse=False, name=X_input>, <KerasTensor shape=(None, 15), dtype=float16, sparse=False, name=B_input>]                                                                                                                                            
x dtype: float16 shape: (1, 42)                                                                                                                                                                                                                                                                   
b dtype: float16 shape: (1, 15)                                                                                                                                                                                                                                                                   
UserWarning: The structure of `inputs` doesn't match the expected structure: ['X_input', 'B_input']. Received: the structure of inputs=('*', '*')                                             

Hard to see, what is wrong

I have tried with: result = model.predict({'X_input': x, 'B_input': b},verbose=0)
but got the same warning

Code is working fine

[LuigiCerone]

@ThorvaldAagaard Same here. Did you figure it out?

[ThorvaldAagaard]

I did not find a fix, except downgrading Tensorflow to 2.17
I assume it is a bug in the implemenation, but the fix is merged, so perhaps noone else but us are reading this.

There should be a way to disable these warnings.

The change is this

    def _maybe_warn_inputs_struct_mismatch(self, inputs):
        try:
            tree.assert_same_structure(
                inputs, self._inputs_struct, check_types=False
            )
        except:
            model_inputs_struct = tree.map_structure(
                lambda x: x.name, self._inputs_struct
            )
            inputs_struct = tree.map_structure(lambda x: "*", inputs)
            warnings.warn(
                "The structure of `inputs` doesn't match the expected "
                f"structure: {model_inputs_struct}. "
                f"Received: the structure of inputs={inputs_struct}"
            )

so probably
assert_same_structure
is the problem

[ThorvaldAagaard]

I found this, that can suppress all the warnings from Python

import warnings
warnings.filterwarnings("ignore")

It is working

[LuigiCerone]

Simple script to reproduce the error:

keras==3.6.0
tensorflow==2.17.0

import numpy as np
from tensorflow.keras import layers, models

# Generate model
input_tensor_1 = layers.Input(shape=(10,), name='input_1')
input_tensor_2 = layers.Input(shape=(5,), name='input_2')

combined = layers.concatenate([input_tensor_1, input_tensor_2])

output = layers.Dense(1, activation='sigmoid')(combined)

model = models.Model(inputs=[input_tensor_1, input_tensor_2], outputs=output)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Train
num_samples = 1000

X1 = np.random.rand(num_samples, 10)
X2 = np.random.rand(num_samples, 5)
y = np.random.randint(2, size=num_samples)

model.fit([X1, X2], y, epochs=5, batch_size=32, verbose=0)

# Prediction
X1_new = np.random.rand(5, 10)
X2_new = np.random.rand(5, 5)

model.predict([X1_new, X2_new], verbose=0)

which gives the mentioned error:

UserWarning: The structure of `inputs` doesn't match the expected structure: ['input_1', 'input_2']. Received: the structure of inputs=('*', '*')
  warnings.warn(

By debugging, it seems a simple list/tuple mismatch in my case, given by:

            tree.assert_same_structure(
                inputs, self._inputs_struct, check_types=False
            )

Fixed by using tuple as input for model:

model = models.Model(inputs=(input_tensor_1, input_tensor_2), outputs=output)

[james77777778]

Hey @ThorvaldAagaard , @LuigiCerone
Thanks for the report. I have submitted a fix to suppress the warning for mismatched tuples and lists.

[ThorvaldAagaard]

Looking forward to the fix, I did not find a workaround for the predict method