Resolution-robust Large Mask Inpainting with Fourier Convolutions
Authors:
Roman Suvorov,
Elizaveta Logacheva,
Anton Mashikhin,
Anastasia Remizova,
Arsenii Ashukha,
Aleksei Silvestrov,
Nae** Kong,
Harshith Goka,
Kiwoong Park,
Victor Lempitsky
Abstract:
Modern image inpainting systems, despite the significant progress, often struggle with large missing areas, complex geometric structures, and high-resolution images. We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function. To alleviate this issue, we propose a new method called large mask inpainting (LaMa). LaMa…
▽ More
Modern image inpainting systems, despite the significant progress, often struggle with large missing areas, complex geometric structures, and high-resolution images. We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function. To alleviate this issue, we propose a new method called large mask inpainting (LaMa). LaMa is based on i) a new inpainting network architecture that uses fast Fourier convolutions (FFCs), which have the image-wide receptive field; ii) a high receptive field perceptual loss; iii) large training masks, which unlocks the potential of the first two components. Our inpainting network improves the state-of-the-art across a range of datasets and achieves excellent performance even in challenging scenarios, e.g. completion of periodic structures. Our model generalizes surprisingly well to resolutions that are higher than those seen at train time, and achieves this at lower parameter&time costs than the competitive baselines. The code is available at \url{https://github.com/saic-mdal/lama}.
△ Less
Submitted 10 November, 2021; v1 submitted 15 September, 2021;
originally announced September 2021.
High-Resolution Daytime Translation Without Domain Labels
Authors:
Ivan Anokhin,
Pavel Solovev,
Denis Korzhenkov,
Alexey Kharlamov,
Taras Khakhulin,
Alexey Silvestrov,
Sergey Nikolenko,
Victor Lempitsky,
Gleb Sterkin
Abstract:
Modeling daytime changes in high resolution photographs, e.g., re-rendering the same scene under different illuminations typical for day, night, or dawn, is a challenging image manipulation task. We present the high-resolution daytime translation (HiDT) model for this task. HiDT combines a generative image-to-image model and a new upsampling scheme that allows to apply image translation at high re…
▽ More
Modeling daytime changes in high resolution photographs, e.g., re-rendering the same scene under different illuminations typical for day, night, or dawn, is a challenging image manipulation task. We present the high-resolution daytime translation (HiDT) model for this task. HiDT combines a generative image-to-image model and a new upsampling scheme that allows to apply image translation at high resolution. The model demonstrates competitive results in terms of both commonly used GAN metrics and human evaluation. Importantly, this good performance comes as a result of training on a dataset of still landscape images with no daytime labels available. Our results are available at https://saic-mdal.github.io/HiDT/.
△ Less
Submitted 23 March, 2020; v1 submitted 19 March, 2020;
originally announced March 2020.
