Misplaced inpaint mask in ultrawide images

[djdookie]

If I inpaint in an image with 960x384 pixels, the mask is misplaced somehow.

In the result, the marked area is inpainted. Not the face. I tried a lot of different parameters, but can't influence it.

[evilstiefel]

I've noticed similar results. It works fine for e.g. 768x512, but certain resolutions lead to coordinate system missmatches between what's masked and what's used when inpainting.

[Xyem]

Just found this happens to me with source images that are 768x512, so it being ultra-wide is not required. It seems to resize the mask, because if I draw a vertical line 1/5 and 4/5 across the image, the rightmost one will be just to the left of the centre of the image.

EDIT: I just realised that the "canvas" for drawing the inpaint area extends quite a way to the right of the input image. I think the issue is that if your browser is not wide enough, the input image is scaled down but the mask canvas is not.

EDIT #2: Zooming out fixes the issue, so the above seems to be the case.

[djdookie]

Good hint. Would be nice ti get a fix for that, so that the mask is also scaled with the window.

[djdookie]

I think I can confirm this assumtion.
The mask layer is not scaled down to the image size.
I can even paint the mask on the right output window.

You can see the two masked areas and the result in the small output window in the same picture. Filled the mask with latent noise in 1 sampling step made it clearly visible.
The output windows is also way to small.

[scalemaildev]

We could get around this just by uploading a mask instead of drawing one.

Unfortunately, I don't see any information on how to actually create a mask image for uploading.

[Xyem]

Unfortunately, I don't see any information on how to actually create a mask image for uploading.

It's just a black and white image (no grey or alpha, if I understand correctly), created in the image editing software of your choice, where white is the part you want to change.

EDIT: Just noticed you can make part of the image not opaque and that will be regarded as part of the mask when you upload it:

erase a part of picture in external editor and upload a transparent picture. Any even slightly transparent areas will become part of the mask

Though I'm not sure how that works with "original" fill pattern (does it still provide the RGB, for example).

[evilstiefel]

Using external masks works, but the workflow is a lot slower than just using the unbuilt feature.

[scalemaildev]

In the meantime I'm just cutting out lower res squares of the area I want to inpaint in Photoshop and then pasting them back into the image.

[djdookie]

I tried to figure out what the problem is and noticed two things:

1. If you set this in style.css, the display doesn't overflow into the right area anymore.

#img2maskimg .h-60{ height: 30rem; overflow: hidden !important; }

2. If you first load an image of 512x512px, and then a wider one, the yellow marked values don't get updated.
   

If you correct them by hand in the devconsole, it's better.

Two things that help improve the situation, but the mask is still off in the final picture. I am still searching for a reason.
Tell me if anyone has an idea please. ;)

[48design]

I'm in the process of figuring this out. Here's a quick-fix you can do to be able to draw accurate masks again:

Create a user.css file in the root directory with the following content:

[data-testid="image"] .touch-none canvas {
    width: 100% !important;
    object-fit: contain;
}

then restart the webui.

There's an offset between the mouse and the "brush" on the canvas, but ignoring one's actual mouse cursor, it's possible to draw the mask.

[48design]

@AUTOMATIC1111 does this functionality come completely out of gradio? Because I couldn't find any reference to the masks/canvasess in the code of this repo.

[48design]

It seems like they are aware over at gradio that something's not right with the inpainting functionality and are working on an overhaul: gradio-app/gradio#1721
It would still be nice to have a temporary fix until it's fixed in gradio itself, and I'll see if I can come up with something.

[djdookie]

@48design Thank you so much for investigating and finally fixing this!

Unfortunately if you place an image in the inpainting tab, you always get an error if you switch the tab and then switch back to inpainting.

[Xyem]

Not only that, inpainting now can't be done on tall images (512x768) because you can't inpaint on the bottom part of the image because of the pointer/brush misalignment. When the pointer leaves the bottom of the image, you can no longer draw. This is as far as I can draw down on such an image.