udpate ip-adapter slow tests#6760

yiyixuxu merged 3 commits intomainhuggingface/diffusers:mainfrom

fix-ipadapter-testshuggingface/diffusers:fix-ipadapter-testsCopy head branch name to clipboard