Add TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING#18469
Add TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING#18469ydshieh merged 7 commits intohuggingface:mainfrom
TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING#18469Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
ydshieh
changed the title
TF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING
|
|
||
| self.assertTrue(loss.shape.as_list() == expected_loss_size or loss.shape.as_list() == [1]) | ||
|
|
||
| def check_keras_fit_results(self, val_loss1, val_loss2, atol=1e-2, rtol=1e-3): |
There was a problem hiding this comment.
For TFSegformerForSemanticSegmentation, we need higher tolerances. See the comment for the change in that model file.
Adding check_keras_fit_results here to avoid overwrite test_keras_fit entirely.
| def test_dataset_conversion(self): | ||
| super().test_dataset_conversion() | ||
|
|
||
| def check_keras_fit_results(self, val_loss1, val_loss2, atol=2e-1, rtol=2e-1): |
There was a problem hiding this comment.
Use higher tolerance for TFSegformerForSemanticSegmentation: this model
-has BatchNormalization layer,
- also have several dropout layers,
- as well as a layer
TFSegformerDropPathwhich has random operation during training.
These factors together cause the statistic of moving_average and moving_variance different, and we have larger validation loss.
There was a problem hiding this comment.
I have found that we need larger tolerances for dense prediction tasks like semantic segmentation. It was the case for TFData2VecVisionForSemanticSegmentation as well.
| def call(self, hidden_states: tf.Tensor, training: bool = False) -> tf.Tensor: | ||
| hidden_states = self.dense(hidden_states) | ||
| hidden_states = self.dropout(hidden_states) | ||
| hidden_states = self.dropout(hidden_states, training=training) |
There was a problem hiding this comment.
We still pass this argument, right?
There was a problem hiding this comment.
We can avoid it since these are supposed to be set automatically during training by the Keras engine. But I find that explicitly specifying it gives me mental peace.
There was a problem hiding this comment.
Thank you for the information!
|
|
||
| hidden_states = self.linear_fuse(tf.concat(all_hidden_states[::-1], axis=-1)) | ||
| hidden_states = self.batch_norm(hidden_states) | ||
| hidden_states = self.batch_norm(hidden_states, training=training) |
|
Test failures are |
|
Thank you, @ydshieh for this. I appreciate the help. |
sgugger
left a comment
sgugger
left a comment
There was a problem hiding this comment.
Thanks a lot for fixing this!
amyeroberts
left a comment
amyeroberts
left a comment
There was a problem hiding this comment.
Looks good! Thanks for the fix ❤️
6 participants
What does this PR do?
The original goal is to fix
TFSegformerModelTest.test_keras_fit, but it ends up the followingTF_MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPINGto some__init__files.trainingarguments in a few layers forTFSegformerModel_prepare_for_classto deal with 2 more image tasksTFData2VecVisionForSemanticSegmentationloss: we need batch dimension (without this,test_dataset_conversionfailed - this was previously skipped due to the lack of labels)