Question Regarding Image Editing/Inpainting Support in the Appendix of Your Paper

[ikumi-0036]

Hi, thank you very much for sharing your excellent research and code.

I’m currently trying to use the UniCon model for inpainting tasks, specifically editing only user-specified masked regions while preserving the rest of the image.

From the paper (especially Figure 10) and the prepare_mask_and_masked_image function in pipeline_unicon.py, it seems that masked editing is supported.
However, I noticed that this function is not called at all during inference, which means that even if I pass a mask_image, it has no effect, and inpainting does not actually work in practice.

Structurally, mask_image and its associated logic seem to be present, so I believe it’s intended to be supported, but I couldn’t find any example or documentation on how to actually use it during inference.

I would greatly appreciate your guidance on the following points:

How can I correctly perform inpainting using mask_image in the UniCon pipeline?

Is there an official or recommended way to apply prepare_mask_and_masked_image during inference?

If you have any working example or demo code, I’d be very grateful if you could share it.

Thank you very much in advance!

[lixirui142]

Hi, thanks for your detailed question

Yes, the UniCon pipeline does support inpainting. You can use the mask_image parameter in the pipeline forward function to specify the inpainting area, white indicating the area to inpaint and black indicating the area to preserve.

After the code reconstruction for release, we find that our guidance method (Eq. 7) doesn't behave as expected. So we tentatively do not include the inpainting examples. We plan to add some inpainting and editing examples from Figure 10 to the notebook in the next few days. Please stay tuned for updates!

[lixirui142]

The inpainting examples have been updated. We also fix bugs in the pipeline to make inpainting work correctly. Please note to update your code before trying inpainting tasks.