-
Gemini: A Family of Highly Capable Multimodal Models
Authors:
Gemini Team,
Rohan Anil,
Sebastian Borgeaud,
Jean-Baptiste Alayrac,
Jiahui Yu,
Radu Soricut,
Johan Schalkwyk,
Andrew M. Dai,
Anja Hauth,
Katie Millican,
David Silver,
Melvin Johnson,
Ioannis Antonoglou,
Julian Schrittwieser,
Amelia Glaese,
Jilin Chen,
Emily Pitler,
Timothy Lillicrap,
Angeliki Lazaridou,
Orhan Firat,
James Molloy,
Michael Isard,
Paul R. Barham,
Tom Hennigan,
Benjamin Lee
, et al. (1325 additional authors not shown)
Abstract:
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr…
▽ More
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.
△ Less
Submitted 17 June, 2024; v1 submitted 18 December, 2023;
originally announced December 2023.
-
A Deep Learning Approach for Characterizing Major Galaxy Mergers
Authors:
Skanda Koppula,
Victor Bapst,
Marc Huertas-Company,
Sam Blackwell,
Agnieszka Grabska-Barwinska,
Sander Dieleman,
Andrea Huber,
Natasha Antropova,
Mikolaj Binkowski,
Hannah Openshaw,
Adria Recasens,
Fernando Caro,
Avishai Deke,
Yohan Dubois,
Jesus Vega Ferrero,
David C. Koo,
Joel R. Primack,
Trevor Back
Abstract:
Fine-grained estimation of galaxy merger stages from observations is a key problem useful for validation of our current theoretical understanding of galaxy formation. To this end, we demonstrate a CNN-based regression model that is able to predict, for the first time, using a single image, the merger stage relative to the first perigee passage with a median error of 38.3 million years (Myrs) over…
▽ More
Fine-grained estimation of galaxy merger stages from observations is a key problem useful for validation of our current theoretical understanding of galaxy formation. To this end, we demonstrate a CNN-based regression model that is able to predict, for the first time, using a single image, the merger stage relative to the first perigee passage with a median error of 38.3 million years (Myrs) over a period of 400 Myrs. This model uses no specific dynamical modeling and learns only from simulated merger events. We show that our model provides reasonable estimates on real observations, approximately matching prior estimates provided by detailed dynamical modeling. We provide a preliminary interpretability analysis of our models, and demonstrate first steps toward calibrated uncertainty estimation.
△ Less
Submitted 9 February, 2021;
originally announced February 2021.
-
Machine Learning for Health (ML4H) Workshop at NeurIPS 2018
Authors:
Natalia Antropova,
Andrew L. Beam,
Brett K. Beaulieu-Jones,
Irene Chen,
Corey Chivers,
Adrian Dalca,
Sam Finlayson,
Madalina Fiterau,
Jason Alan Fries,
Marzyeh Ghassemi,
Mike Hughes,
Bruno Jedynak,
Jasvinder S. Kandola,
Matthew McDermott,
Tristan Naumann,
Peter Schulam,
Farah Shamout,
Alexandre Yahi
Abstract:
This volume represents the accepted submissions from the Machine Learning for Health (ML4H) workshop at the conference on Neural Information Processing Systems (NeurIPS) 2018, held on December 8, 2018 in Montreal, Canada.
This volume represents the accepted submissions from the Machine Learning for Health (ML4H) workshop at the conference on Neural Information Processing Systems (NeurIPS) 2018, held on December 8, 2018 in Montreal, Canada.
△ Less
Submitted 24 November, 2018; v1 submitted 17 November, 2018;
originally announced November 2018.
-
Multi-task Learning in the Computerized Diagnosis of Breast Cancer on DCE-MRIs
Authors:
Natalia Antropova,
Benjamin Huynh,
Maryellen Giger
Abstract:
Hand-crafted features extracted from dynamic contrast-enhanced magnetic resonance images (DCE-MRIs) have shown strong predictive abilities in characterization of breast lesions. However, heterogeneity across medical image datasets hinders the generalizability of these features. One of the sources of the heterogeneity is the variation of MR scanner magnet strength, which has a strong influence on i…
▽ More
Hand-crafted features extracted from dynamic contrast-enhanced magnetic resonance images (DCE-MRIs) have shown strong predictive abilities in characterization of breast lesions. However, heterogeneity across medical image datasets hinders the generalizability of these features. One of the sources of the heterogeneity is the variation of MR scanner magnet strength, which has a strong influence on image quality, leading to variations in the extracted image features. Thus, statistical decision algorithms need to account for such data heterogeneity. Despite the variations, we hypothesize that there exist underlying relationships between the features extracted from the datasets acquired with different magnet strength MR scanners. We compared the use of a multi-task learning (MTL) method that incorporates those relationships during the classifier training to support vector machines run on a merged dataset that includes cases with various MRI strength images. As a result, higher predictive power is achieved with the MTL method.
△ Less
Submitted 14 January, 2017;
originally announced January 2017.
