Computer Science > Computer Vision and Pattern Recognition
[Submitted on 7 Jun 2023 (v1), last revised 17 Feb 2024 (this version, v8)]
Title:GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation
View PDF HTML (experimental)Abstract:Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification. However, the usage of diffusion models to generate the high-quality object detection data remains an underexplored area, where not only image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode the semantic layouts. In this paper, we propose the GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only the bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.
Submission history
From: Kai Chen [view email][v1] Wed, 7 Jun 2023 17:17:58 UTC (8,986 KB)
[v2] Thu, 8 Jun 2023 08:21:55 UTC (8,986 KB)
[v3] Fri, 9 Jun 2023 09:51:36 UTC (8,978 KB)
[v4] Wed, 28 Jun 2023 15:52:13 UTC (8,978 KB)
[v5] Tue, 17 Oct 2023 07:51:55 UTC (9,001 KB)
[v6] Thu, 28 Dec 2023 08:26:55 UTC (17,342 KB)
[v7] Sun, 4 Feb 2024 13:05:49 UTC (17,343 KB)
[v8] Sat, 17 Feb 2024 01:43:45 UTC (17,343 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.