-
FACET: Fairness in Computer Vision Evaluation Benchmark
Authors:
Laura Gustafson,
Chloe Rolland,
Nikhila Ravi,
Quentin Duval,
Aaron Adcock,
Cheng-Yang Fu,
Melissa Hall,
Candace Ross
Abstract:
Computer vision models have known performance disparities across attributes such as gender and skin tone. This means during tasks such as classification and detection, model performance differs for certain classes based on the demographics of the people in the image. These disparities have been shown to exist, but until now there has not been a unified approach to measure these differences for com…
▽ More
Computer vision models have known performance disparities across attributes such as gender and skin tone. This means during tasks such as classification and detection, model performance differs for certain classes based on the demographics of the people in the image. These disparities have been shown to exist, but until now there has not been a unified approach to measure these differences for common use-cases of computer vision models. We present a new benchmark named FACET (FAirness in Computer Vision EvaluaTion), a large, publicly available evaluation set of 32k images for some of the most common vision tasks - image classification, object detection and segmentation. For every image in FACET, we hired expert reviewers to manually annotate person-related attributes such as perceived skin tone and hair type, manually draw bounding boxes and label fine-grained person-related classes such as disk jockey or guitarist. In addition, we use FACET to benchmark state-of-the-art vision models and present a deeper understanding of potential performance disparities and challenges across sensitive demographic attributes. With the exhaustive annotations collected, we probe models using single demographics attributes as well as multiple attributes using an intersectional approach (e.g. hair color and perceived skin tone). Our results show that classification, detection, segmentation, and visual grounding models exhibit performance disparities across demographic attributes and intersections of attributes. These harms suggest that not all people represented in datasets receive fair and equitable treatment in these vision tasks. We hope current and future results using our benchmark will contribute to fairer, more robust vision models. FACET is available publicly at https://facet.metademolab.com/
△ Less
Submitted 31 August, 2023;
originally announced September 2023.
-
Segment Anything
Authors:
Alexander Kirillov,
Eric Mintun,
Nikhila Ravi,
Hanzi Mao,
Chloe Rolland,
Laura Gustafson,
Tete Xiao,
Spencer Whitehead,
Alexander C. Berg,
Wan-Yen Lo,
Piotr Dollár,
Ross Girshick
Abstract:
We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date (by far), with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and…
▽ More
We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date (by far), with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive -- often competitive with or even superior to prior fully supervised results. We are releasing the Segment Anything Model (SAM) and corresponding dataset (SA-1B) of 1B masks and 11M images at https://segment-anything.com to foster research into foundation models for computer vision.
△ Less
Submitted 5 April, 2023;
originally announced April 2023.
-
Enhancing the Guidance of the Intentional Model "MAP": Graph Theory Application
Authors:
Rebecca Deneckere,
Elena Kornyshova,
Colette Rolland
Abstract:
The MAP model was introduced in information system engineering in order to model processes on a flexible way. The intentional level of this model helps an engineer to execute a process with a strong relationship to the situation of the project at hand. In the literature, attempts for having a practical use of maps are not numerous. Our aim is to enhance the guidance mechanisms of the process exe…
▽ More
The MAP model was introduced in information system engineering in order to model processes on a flexible way. The intentional level of this model helps an engineer to execute a process with a strong relationship to the situation of the project at hand. In the literature, attempts for having a practical use of maps are not numerous. Our aim is to enhance the guidance mechanisms of the process execution by reusing graph algorithms. After clarifying the existing relationship between graphs and maps, we improve the MAP model by adding qualitative criteria. We then offer a way to express maps with graphs and propose to use Graph theory algorithms to offer an automatic guidance of the map. We illustrate our proposal by an example and discuss its limitations.
△ Less
Submitted 2 November, 2009;
originally announced November 2009.
