How to Use Multi-ID

[GavinZhao19]

Hello,

I recently came across the Multi-ID feature mentioned in the documentation, but I'm having trouble finding clear examples of how to use it. Specifically, I'm interested in combining images of different individuals, such as uploading two separate photos and generating a composite image.

For instance, I'd like to upload photos of two celebrities, Yang Mi and Taylor Swift, and create a merged image of them. However, I haven't found any examples in the code or on the Hugging Face platform that demonstrate how to achieve this.

Could you please provide some guidance or instructions on how to use the Multi-ID feature for this purpose? Any example code or detailed steps would be greatly appreciated.

Thank you for your assistance.

[ResearcherXman]

We don't have a specific plan to release this part of codes by now, but we do encourage the community to implement and PR.

[zewolf5]

Well. Since the answer to having multiple input images is suggested here and works:
#56 (comment)

Not sure if its already there, but multiple reference images? I could not find a way to select more than one image in the sample code provided. Topic #33 mentioned use of multiple reference images for better results.

I'm no expert but this should work image = pipe( ... image_embeds=(face_emb1 + face_emb2 + face_emb3 + face_emb4)/4, ...)

It seems to also work for adding different pictures into the same calculation. Did some quick testing and it seems to be the way to go.

Load the different pictures, either many from only one or multiple people and get a combined "face".
If you have gotten the sample code up and running, you got something like

face_emb = face_info['embedding']
change it to
face_emb = (face_info1['embedding'] + face_info2['embedding']) / 2
or
face_emb = (face_info1['embedding'] + face_info2['embedding'] + face_info3['embedding']) / 3
etc.

I made a function for that taking a folder of face images:
face_embeds = helpers.getAverageFaceEmb(face_folder)

So get 1 face_info object for each image. Add and divide. I myself have not quite read and understood the concepts, but seems to be some math vectors or such that defines what a face looks like. So pure math to add the definition of the faces.

[platform-kit]

We don't have a specific plan to release this part of codes by now, but we do encourage the community to implement and PR.

Is there a reason you are being vague about it? What do you gain by not releasing the code that you demonstrated in your paper?

[wangqixun]

We don't have a specific plan to release this part of codes by now, but we do encourage the community to implement and PR.

Is there a reason you are being vague about it? What do you gain by not releasing the code that you demonstrated in your paper?

maybe its name is not OpenAID

[usamaehsan]

Well. Since the answer to having multiple input images is suggested here and works: #56 (comment)

Not sure if its already there, but multiple reference images? I could not find a way to select more than one image in the sample code provided. Topic #33 mentioned use of multiple reference images for better results.

I'm no expert but this should work image = pipe( ... image_embeds=(face_emb1 + face_emb2 + face_emb3 + face_emb4)/4, ...)

It seems to also work for adding different pictures into the same calculation. Did some quick testing and it seems to be the way to go.

Load the different pictures, either many from only one or multiple people and get a combined "face". If you have gotten the sample code up and running, you got something like

face_emb = face_info['embedding'] change it to face_emb = (face_info1['embedding'] + face_info2['embedding']) / 2 or face_emb = (face_info1['embedding'] + face_info2['embedding'] + face_info3['embedding']) / 3 etc.

I made a function for that taking a folder of face images: face_embeds = helpers.getAverageFaceEmb(face_folder)

So get 1 face_info object for each image. Add and divide. I myself have not quite read and understood the concepts, but seems to be some math vectors or such that defines what a face looks like. So pure math to add the definition of the faces.

@zewolf5 can we use different weight to each image?

[G-force78]

We don't have a specific plan to release this part of codes by now, but we do encourage the community to implement and PR.

In your original demo on hf you had it implemented. Why was it removed?

[cubiq]

It's possible with attention masking

(this is done in comfyui)

https://github.com/cubiq/ComfyUI_InstantID

[Enzzer]

It's possible with attention masking

(this is done in comfyui)

https://github.com/cubiq/ComfyUI_InstantID

Can you please share the workflow you used to do this?

[cubiq]

@Enzzer the workflow is in the repository

[Enzzer]

@cubiq Thanks for sharing the workflow!
I tried using it as it is (other than using my face and pose images) but I'm not getting the same results as you.
I'm getting both faces as the last input face image, seems like the first face is lost in the conditioning chain.
Any idea why?

[cubiq]

you probably just need to update the extension

[Enzzer]

@cubiq I tried updating both ComfyUI and your extension but I'm still getting the same face twice.
Can you share your input images so I can test with them?

[toonverbeek]

FWIW, also getting the same result as @Enzzer where both images are merged into one face. @Enzzer did you find a solution by any chance?

[cubiq]

the latest version, with the provided workflow works very well.

[toonverbeek]

Yes confirmed, I used the reference pose image from the repository with two custom portrait photo's and results are now a lot better. Thanks for the update 👍

[xpluscal]

hey @toonverbeek can you share a workflow on how you combined pose with multi ID? How did you achieve this?

[toonverbeek]

hey @toonverbeek can you share a workflow on how you combined pose with multi ID? How did you achieve this?

I used essentially the default multi-id workflow included in this repository. I kept the pose reference image but replaced the two other inputs with my own images. That worked well enough for my usecase.