Skip to content

Conversation

@NielsRogge
Copy link
Contributor

@NielsRogge NielsRogge commented Dec 14, 2022

What does this PR do?

This PR adds the AutoModelForUniversalSegmentation class and corresponding mapping. Models that can be added to this mapping include DETR, MaskFormer, Mask2Former and OneFormer.

To do:

  • update pipeline

@sgugger for some reason make fixup complains:

Traceback (most recent call last):
  File "/Users/nielsrogge/Documents/python_projecten/transformers/utils/check_repo.py", line 827, in <module>
    check_repo_quality()
  File "/Users/nielsrogge/Documents/python_projecten/transformers/utils/check_repo.py", line 816, in check_repo_quality
    check_models_are_in_init()
  File "/Users/nielsrogge/Documents/python_projecten/transformers/utils/check_repo.py", line 369, in check_models_are_in_init
    raise Exception(f"The following models should be in the main init: {','.join(models_not_in_init)}.")
Exception: The following models should be in the main init: MaskFormerForUniversalSegmentation.

@sgugger
Copy link
Collaborator

sgugger commented Dec 14, 2022

I think the error message is pretty clear on what to do. The model would also need to be added to the doc page of maskformer if we go through with this.

I'm not convinced we should however. While adding a new auto-model API could make sense (I'd wait to have more than one model though), renaming the model class a year after the model has been released is not something we should do (the same way we keep GPTLMHeadModel for instance).

Copy link
Contributor

@alaradirik alaradirik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this! Looks good to me but some files still need editing, which cause some tests to fail.

Could you update the references to the deprecated model and output classes in maskformer.mdx, configuration_maskformer.py, image_processing_maskformer.py, test_feature_extraction_maskformer.py, test_modeling_maskformer.py, and update check_repo.py?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Dec 14, 2022

The documentation is not available anymore as the PR was closed or merged.

@sgugger
Copy link
Collaborator

sgugger commented Dec 14, 2022

I'm unsure why you keep changing more of the MaskFormerForInstanceSegmentation. I think I may have been unclear in my previous comments. I am against changing that name in a model that has been around for 10 months now, for the same reason we did not change the name of GPTLMHeadModel or BertLMHeadModel even if those are not ideal.

Comment on lines +445 to 446
("detr", "DetrForSegmentation"),
("maskformer", "MaskFormerForInstanceSegmentation"),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally all models in this mapping should be called ForUniversalSegmentation, but we can keep the name if you prefer so

@NielsRogge
Copy link
Contributor Author

@sgugger I'll keep the old names, but add them to the same (new) mapping.

@NielsRogge NielsRogge mentioned this pull request Dec 16, 2022
5 tasks
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating!

@NielsRogge NielsRogge merged commit 7f99861 into huggingface:main Dec 16, 2022
gsarti added a commit to gsarti/transformers that referenced this pull request Dec 16, 2022
… add_get_encoder_decoder_fsmt

* 'main' of ssh://github.com/huggingface/transformers: (1433 commits)
  Add Universal Segmentation class + mapping (huggingface#20766)
  Stop calling expand_1d on newer TF versions (huggingface#20786)
  Fix object detection2 (huggingface#20798)
  [Pipeline] skip feature extraction test if in `IMAGE_PROCESSOR_MAPPING` (huggingface#20790)
  Recompile `apex` in `DeepSpeed` CI image (huggingface#20788)
  Move convert_to_rgb to image_transforms module (huggingface#20784)
  Generate: use `GenerationConfig` as the basis for `.generate()` parametrization (huggingface#20388)
  Install video dependency for pipeline CI (huggingface#20777)
  Fixing object detection with `layoutlm` (huggingface#20776)
  [Pipeline] fix failing bloom `pipeline` test (huggingface#20778)
  Patch for FlanT5-XXL 8bit support (huggingface#20760)
  Install vision for TF pipeline tests (huggingface#20771)
  Even more validation. (huggingface#20762)
  Add Swin backbone (huggingface#20769)
  Install `torch-tensorrt 1.3.0` for DeepSpeed CI (huggingface#20764)
  Replaces xxx_required with requires_backends (huggingface#20715)
  [CI-Test] Fixes but also skips the mT5 tests (huggingface#20755)
  Fix attribute error problem  (huggingface#20765)
  [Tests] Improve test_attention_outputs (huggingface#20701)
  Fix missing `()` in some usage of `is_flaky` (huggingface#20749)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants