add initial design for uniform processors + align model

[molbap]

What does this PR do?

Adds a uniform signature for processors. This PR adds the initial design + one model for the larger #30511.

Usage

As before, kwargs that are passed to processors at __call__ time take priority. However, per-modality processors can be instantiated with their own kwargs, and if they are not overriden at call time, they will serve as defaults.

Type hinting of kwargs is preserved if they are passed as structured dictionary entries

It also works with kwargs passed without nesting:

Merging of kwargs and handling priority order is done in processing_utils through a dedicated method.
The order of operations is as follows:

1. kwargs passed as before have highest priority to preserve BC.

high_priority_kwargs = {"crop_size" = (224, 224), "padding" = "max_length"}
processor(..., **high_priority_kwargs)

2. kwargs passed as modality-specific kwargs have second priority. This is the recommended API.

processor(..., text_kwargs={"padding": "max_length"}, images_kwargs={"crop_size": (224, 224)}})

3. kwargs passed during instantiation of a modality processor have fourth priority.

tokenizer = tokenizer_class(..., {"padding": "max_length"})
image_processor = image_processor_class(...)
processor(tokenizer, image_processor) # will pass max_length unless overriden by kwargs at call

4. defaults kwargs specified at processor level have lowest priority.

class MyProcessingKwargs(ProcessingKwargs, CommonKwargs, TextKwargs, ImagesKwargs, total=False):
    _defaults = {
        "text_kwargs": {
            "padding": "max_length",
            "max_length": 64,
        },
    }

Missing:

• Move tests of kwargs checking to a common util

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[molbap]

cc @amyeroberts for a review with a narrower scope than the parent PR 😁

[molbap]

cc @amyeroberts, sorry to divide my pings 😅 PR is big and I wanted to split it up, this one should be merge-able and be the basis for the rest, I'll rebase afterwards (and @qubvel welcome if you want to take a look!)

It includes the kwargs merging just mentioned in the other PR, moved them to processing common!

[amyeroberts]

Thanks!

Same comment as for the other PR - it would be good to move out the kwarg prep logic to out of the config and tests to make sure we can properly control tokenizer kwargs with tokenizer.init_kwargs and the input kwargs.

cc @qubvel

[amyeroberts]

@molbap OK, sounds good! Let's wait for @qubvel's eagle eye. Once tests are added, ping me again for a final review 🤗

[qubvel]

Nice to see _merge_kwargs as a separate method, this is exactly what came to my mind while reviewing the first PR 🙂

[ArthurZucker]

a lot cleaner!

[ArthurZucker]

Suggested change

	_defaults = {
	padding: "max_length"
	max_lenght: 64

should work no? Or does it not update the default for type-hints?

[molbap]

yes it works for sure, this was to have a structured dict for defaults. Can change :)

[molbap]

ah, now I remember, it actually can't work like that since Typed Dicts don't support default values, they are made to hold the layout. They can have any attributes however, but it won't pass a value as default -like a dataclass would, but in this case we'd lose typing-, hence the manual operation

[ArthurZucker]

ok got it thanks! Let's maybe comment about this!

[amyeroberts]

Do we have a comment for future code inspectors? I'm assuming here isn't the best place (we don't want it for all models) but didn't find a corresponding one elsewhere on a quick skim

[molbap]

On that: there's doc in processing_utils.ProcessingKwargs, I added a comment nudging users to check there for documentation!

[molbap]

I have updated the PR description to be more self-contained: by using typing_extensions the imports for Unpack work, but type hinting doesn't for versions of python < 3.11. Functionality, though, is preserved for older versions I tested.

[amyeroberts]

Looks great!

Just a few small comments

[amyeroberts]

Do we have a comment for future code inspectors? I'm assuming here isn't the best place (we don't want it for all models) but didn't find a corresponding one elsewhere on a quick skim

[molbap]

@amyeroberts FYI Kept digging for the kwargs merging logic, and found an edge case that was giving unreliable results in the tokenizer. Refactored and doubled number of tests to avoid further trickery (including an edge case found earlier by @qubvel ), logic should be easier to read now. Nothing else changed, and tests should pass reliably.