Fix number of minimal calls to the Hub with peft integration#25715

sgugger merged 5 commits intomainhuggingface/transformers:mainfrom

min_callshuggingface/transformers:min_callsCopy head branch name to clipboard