Suggestion. Add a mask for face swap that is not the same as for the face detection.

[zewolf5]

For a better end result, using any of the ComfyUI tools to get a mask with only the face will be better than a square box for the face swap part.
So I tested this and works like charm. Just allow for a second mask into the "Faceswap generate" to be used for the actual faceswap. That face swap mask is too small to be used for face detection and almost always fail.

Or add a Dilation field to increase the area around the one mask being used.

[zewolf5]

Ignore this. Padding = Dilation it seems.

So just close this.

[nosiu]

this might be a good idea for an attention mask for control net, adding the second mask