Generate: correct default model input creation for decoder-only models

[gante]

What does this PR do?

Fixes #21578 and addresses concerns in #21575

Context

Support for .generate() from inputs_embeds with selected decoder-only models was added recently (#21405). This feature enables a .generate(inputs_embeds=inputs_embeds) call, i.e. without a input_ids.

This specific call strategy, under the hood, implies that .generate() is in charge of creating a) an input_ids for later use (to concatenate the generated tokens) and b) the corresponding attention_mask. This automated creation was not working properly for batch_size>1.

Changes

The changes in the PR respect the following desiderata (which required moving a few things around):

1. The attention_mask can be automatically inferred (with all ones) regardless of the shape of inputs_embeds;
2. When inputs_embeds is passed and input_ids is not, the automated input_ids has a sequence length of 1. This is particularly relevant for BLIP, as we don't want input_ids to start with the embeddings' sequence length.

This PR also adds/enhances tests, to ensure we don't regress on this capability.

⚠️ if approved, I will make the corresponding TF changes before merging.

[gante]

cc @dimitry12 this fixes the error you reported in #21575 (GPT2 was throwing the same error as GPTJ, and gets fixed here)

cc @NielsRogge please don't forget to add batched tests on models with generation capabilities, generate+batching is surprisingly tricky🙏

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

Thanks for your work on this!

[sgugger]

But it does seem like it breaks a lot of tests 😅

[gante]

(Merging -- the failing test is a known failing test)

[gante]

@dimitry12 lmk if you see the error when using GPT-J :)

[dimitry12]

@dimitry12 lmk if you see the error when using GPT-J :)

@gante GPT-J generation without dummy input_ids using only inputs_embeds works without errors now. Thank you!