Does Diffusion Beat GAN in Image Super Resolution?
Authors:
Denis Kuznedelev,
Valerii Startsev,
Daniil Shlenskii,
Sergey Kastryulin
Abstract:
There is a prevalent opinion in the recent literature that Diffusion-based models outperform GAN-based counterparts on the Image Super Resolution (ISR) problem. However, in most studies, Diffusion-based ISR models were trained longer and utilized larger networks than the GAN baselines. This raises the question of whether the superiority of Diffusion models is due to the Diffusion paradigm being be…
▽ More
There is a prevalent opinion in the recent literature that Diffusion-based models outperform GAN-based counterparts on the Image Super Resolution (ISR) problem. However, in most studies, Diffusion-based ISR models were trained longer and utilized larger networks than the GAN baselines. This raises the question of whether the superiority of Diffusion models is due to the Diffusion paradigm being better suited for the ISR task or if it is a consequence of the increased scale and computational resources used in contemporary studies. In our work, we compare Diffusion-based and GAN-based Super Resolution under controlled settings, where both approaches are matched in terms of architecture, model and dataset size, and computational budget. We show that a GAN-based model can achieve results comparable to a Diffusion-based model. Additionally, we explore the impact of design choices such as text conditioning and augmentation on the performance of ISR models, showcasing their effect on several downstream tasks. We will release the inference code and weights of our scaled GAN.
△ Less
Submitted 27 May, 2024;
originally announced May 2024.
YaART: Yet Another ART Rendering Technology
Authors:
Sergey Kastryulin,
Artem Konev,
Alexander Shishenya,
Eugene Lyapustin,
Artem Khurshudov,
Alexander Tselousov,
Nikita Vinokurov,
Denis Kuznedelev,
Alexander Markovich,
Grigoriy Livshits,
Alexey Kirillov,
Anastasiia Tabisheva,
Liubov Chubarova,
Marina Kaminskaia,
Alexander Ustyuzhanin,
Artemii Shvetsov,
Daniil Shlenskii,
Valerii Startsev,
Dmitrii Kornilov,
Mikhail Romanov,
Artem Babenko,
Sergei Ovcharenko,
Valentin Khrulkov
Abstract:
In the rapidly progressing field of generative models, the development of efficient and high-fidelity text-to-image diffusion systems represents a significant frontier. This study introduces YaART, a novel production-grade text-to-image cascaded diffusion model aligned to human preferences using Reinforcement Learning from Human Feedback (RLHF). During the development of YaART, we especially focus…
▽ More
In the rapidly progressing field of generative models, the development of efficient and high-fidelity text-to-image diffusion systems represents a significant frontier. This study introduces YaART, a novel production-grade text-to-image cascaded diffusion model aligned to human preferences using Reinforcement Learning from Human Feedback (RLHF). During the development of YaART, we especially focus on the choices of the model and training dataset sizes, the aspects that were not systematically investigated for text-to-image cascaded diffusion models before. In particular, we comprehensively analyze how these choices affect both the efficiency of the training process and the quality of the generated images, which are highly important in practice. Furthermore, we demonstrate that models trained on smaller datasets of higher-quality images can successfully compete with those trained on larger datasets, establishing a more efficient scenario of diffusion models training. From the quality perspective, YaART is consistently preferred by users over many existing state-of-the-art models.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.