[Hubert] Fix Hubert processing auto

[younesbelkada]

What does this PR do?

Currently on the main branch, the scripts provided on the docstring of Hubert fails:

from transformers import AutoProcessor, HubertModel
from datasets import load_dataset
import soundfile as sf

processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft")
model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft")

def map_to_array(batch):
  speech, _ = sf.read(batch["file"])
  batch["speech"] = speech
  return batch

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
        
ds = ds.map(map_to_array)

input_values = processor(ds["speech"][0], return_tensors="pt").input_values  # Batch size 1
hidden_states = model(input_values).last_hidden_statehidden_states = model(input_values).last_hidden_state

With the PR #21225 all custom xxxProcessor have been removed in favor of AutoProcessor in docstrings. Since Hubert was not included inside the processor automapping the script above lead into a bug, since if the model is not present in the auto mapping dictionary, the script will try to load a tokenizer:

transformers/src/transformers/models/auto/processing_auto.py

Line 275 in 255257f

return AutoTokenizer.from_pretrained(

/ Hence, Wav2vec2CTCTokenizer was loaded instead of Wav2vec2Processor that is supposed to be the target object to be loaded.

This PR fixes this by adding hubert inside the automapping class for AutoProcessor

This PR also fixes 2 failing doctests for HubertModel, link to failing job: https://github.com/huggingface/transformers/actions/runs/4002271138/jobs/6869333719

cc @sgugger @ydshieh

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

Thanks for the fix!

[ydshieh]

Thanks!