Add UperNet

[NielsRogge]

What does this PR do?

This PR adds the classic UperNet framework to Transformers.

Many papers that introduce a new vision backbone, such as BEiT, ConvNeXt, Swin,... benchmark their model on downstream tasks such as semantic segmentation and object detection. All of these papers use the UperNet framework (introduced in 2018) when evaluating their backbone on semantic segmentation.

Hence, this PR implements this framework, making use of the new AutoBackbone API to make the following possible:

from transformers import SwinConfig, UperNetConfig, UperNetForSemanticSegmentation

backbone_config = SwinConfig(out_features=["stage1", "stage2", "stage3", "stage4"])

config = UperNetConfig(backbone_config=backbone_config)
model = UperNetForSemanticSegmentation(config)

In the code above, we're instantiating the UperNet framework with Swin Transformer as backbone. The code looks equivalent for another backbone, like ConvNeXt:

from transformers import ConvNextBackbone, UperNetConfig, UperNetForSemanticSegmentation

backbone_config = ConvNextBackbone(out_features=["stage1", "stage2", "stage3", "stage4"])

config = UperNetConfig(backbone_config=backbone_config)
model = UperNetForSemanticSegmentation(config)

To do:

• looking into supporting from_pretrained of backbones => will be done in a follow-up PR
• make sure UperNetImageProcessor does exact same preprocessing
• make UperNetImageProcessor also take segmentation_maps as optional input
• add image processor tests
• convert all checkpoints + update organization
• fix integration tests

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

Thanks for working on this new model. Left a couple of comments.

[sgugger]

Maybe raise an error if then the type is not a PretrainedConfig?

[alaradirik]

Thanks for adding this!

Left a few minor comments but everything looks good apart from the issues/comments related to configuration and model parameter initialisation (+ organization name update) and works as expected.

[sgugger]

LGTM on the model initialization. One last comment on the image processor: it needs to be added in the auto mapping, and since it seems to be a full copy of the SegformerImageProcessor, you should re-use it in the auto-mapping and not introduce a new image processor here.

[sgugger]

Thanks for all your work on this model!

[NielsRogge]

Thanks for the review, I'm waiting for the authors to respond regarding the creation of an organization on the hub.