Add support of segmentation mask in RandomCutout

[cosmo3769]

What does this PR do?

Related: #1992

Add support of segmentation mask in Augmix layer. Here is the colab link for demo.

Output generated by demo:

Who can review?

@ianstenbit @jbischof

[ianstenbit]

Thanks for the PR!

I think we need to think a bit about how to handle segmentation masks in this case.
For example, in GridMask, we actually keep the full segmentation mask unedited as the ground-truth label. It may be that this is also the best approach here.

My concern with zeroing-out the mask is that we're effectively telling the model that this is neither foreground class nor background class. Maybe that's okay, but I don't have a great intuition for that.

It could be ideal to instead set the masked-out area to have the background class, but that introduces some API awkwardness as the caller needs to specify what the background class is.

My intuition is that the best approach is probably to leave the seg mask unmodified, but it's hard to say for sure.

I'd propose that we test each option with my (WIP) KerasCV segmentation guide and see which one yields the best results

[cosmo3769]

Hey @ianstenbit,

Thank you for this intuition and your colab notebook. I will test out all the possibilities and see which one yields the best results.
Also, I have been creating one colab where I was testing all these layers. I was thinking of making it backend agnostic, testing all the layers and publishing a report in keras.io. 😄

[cosmo3769]

Hey @ianstenbit, I have tested it and I find out that it works best when we don't apply random cutout to mask. Here is the colab link.

• With RandomCutout applied to mask - 80.75% test accuracy.
• With RandomCutout not applied to mask - 83.89% test accuracy.

I have updated the code to remove randomcutout from mask.

[ianstenbit]

Hey @ianstenbit, I have tested it and I find out that it works best when we don't apply random cutout to mask. Here is the colab link.

• With RandomCutout applied to mask - 80.75% test accuracy.
• With RandomCutout not applied to mask - 83.89% test accuracy.

I have updated the code to remove randomcutout from mask.

Awesome thank you for doing this test -- that's awesome!

[ianstenbit]

/gcbrun

[ianstenbit]

Thanks for the PR!

[ianstenbit]

/gcbrun