Skip to content

Conversation

@younesbelkada
Copy link
Contributor

@younesbelkada younesbelkada commented Jan 26, 2023

What does this PR do?

This PR fixes a failing doctest for PerceiverModel. Link to failing job: https://github.com/huggingface/transformers/actions/runs/4002271138/jobs/6869333719

With #21225 being merged, the snippet here:

...
model = PerceiverModel(config, input_preprocessor=preprocessor, decoder=decoder)
# you can then do a forward pass as follows:
tokenizer = PerceiverTokenizer()

has been modified by:

...
model = PerceiverModel(config, input_preprocessor=preprocessor, decoder=decoder)
# you can then do a forward pass as follows:
tokenizer = AutoTokenizer()

As there is no canonical way to automatically load a tokenizer from something which is different than a path, or model id, one should load any default tokenizer by instantiating it using the child class and not using AutoTokenizer.

This PR reverts this change and fixes the doctest

cc @ydshieh 💯

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jan 26, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Collaborator

@ydshieh ydshieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look more than reasonable to me! Thanks. Let Sylvain to see if he has any other thought though, as he is the master of auto!

@younesbelkada younesbelkada requested a review from sgugger January 26, 2023 13:21
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

@younesbelkada younesbelkada merged commit b225ee6 into huggingface:main Jan 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants