-
Gemini: A Family of Highly Capable Multimodal Models
Authors:
Gemini Team,
Rohan Anil,
Sebastian Borgeaud,
Jean-Baptiste Alayrac,
Jiahui Yu,
Radu Soricut,
Johan Schalkwyk,
Andrew M. Dai,
Anja Hauth,
Katie Millican,
David Silver,
Melvin Johnson,
Ioannis Antonoglou,
Julian Schrittwieser,
Amelia Glaese,
Jilin Chen,
Emily Pitler,
Timothy Lillicrap,
Angeliki Lazaridou,
Orhan Firat,
James Molloy,
Michael Isard,
Paul R. Barham,
Tom Hennigan,
Benjamin Lee
, et al. (1325 additional authors not shown)
Abstract:
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr…
▽ More
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.
△ Less
Submitted 17 June, 2024; v1 submitted 18 December, 2023;
originally announced December 2023.
-
Fluorescent Neuronal Cells v2: Multi-Task, Multi-Format Annotations for Deep Learning in Microscopy
Authors:
Luca Clissa,
Antonio Macaluso,
Roberto Morelli,
Alessandra Occhinegro,
Emiliana Piscitiello,
Ludovico Taddei,
Marco Luppi,
Roberto Amici,
Matteo Cerri,
Timna Hitrec,
Lorenzo Rinaldi,
Antonio Zoccoli
Abstract:
Fluorescent Neuronal Cells v2 is a collection of fluorescence microscopy images and the corresponding ground-truth annotations, designed to foster innovative research in the domains of Life Sciences and Deep Learning. This dataset encompasses three image collections in which rodent neuronal cells' nuclei and cytoplasm are stained with diverse markers to highlight their anatomical or functional cha…
▽ More
Fluorescent Neuronal Cells v2 is a collection of fluorescence microscopy images and the corresponding ground-truth annotations, designed to foster innovative research in the domains of Life Sciences and Deep Learning. This dataset encompasses three image collections in which rodent neuronal cells' nuclei and cytoplasm are stained with diverse markers to highlight their anatomical or functional characteristics. Alongside the images, we provide ground-truth annotations for several learning tasks, including semantic segmentation, object detection, and counting. The contribution is two-fold. First, given the variety of annotations and their accessible formats, we envision our work facilitating methodological advancements in computer vision approaches for segmentation, detection, feature learning, unsupervised and self-supervised learning, transfer learning, and related areas. Second, by enabling extensive exploration and benchmarking, we hope Fluorescent Neuronal Cells v2 will catalyze breakthroughs in fluorescence microscopy analysis and promote cutting-edge discoveries in life sciences. The data are available at: https://amsacta.unibo.it/id/eprint/7347
△ Less
Submitted 26 July, 2023;
originally announced July 2023.
-
Application of the nnU-Net for automatic segmentation of lung lesion on CT images, and implication on radiomic models
Authors:
Matteo Ferrante,
Lisa Rinaldi,
Francesca Botta,
Xiaobin Hu,
Andreas Dolp,
Marta Minotti,
Francesca De Piano,
Gianluigi Funicelli,
Stefania Volpe,
Federica Bellerba,
Paolo De Marco,
Sara Raimondi,
Stefania Rizzo,
Kuangyu Shi,
Marta Cremonesi,
Barbara A. Jereczek-Fossa,
Lorenzo Spaggiari,
Filippo De Marinis,
Roberto Orecchia,
Daniela Origgi
Abstract:
Lesion segmentation is a crucial step of the radiomic workflow. Manual segmentation requires long execution time and is prone to variability, impairing the realisation of radiomic studies and their robustness. In this study, a deep-learning automatic segmentation method was applied on computed tomography images of non-small-cell lung cancer patients. The use of manual vs automatic segmentation in…
▽ More
Lesion segmentation is a crucial step of the radiomic workflow. Manual segmentation requires long execution time and is prone to variability, impairing the realisation of radiomic studies and their robustness. In this study, a deep-learning automatic segmentation method was applied on computed tomography images of non-small-cell lung cancer patients. The use of manual vs automatic segmentation in the performance of survival radiomic models was assessed, as well. METHODS A total of 899 NSCLC patients were included (2 proprietary: A and B, 1 public datasets: C). Automatic segmentation of lung lesions was performed by training a previously developed architecture, the nnU-Net, including 2D, 3D and cascade approaches. The quality of automatic segmentation was evaluated with DICE coefficient, considering manual contours as reference. The impact of automatic segmentation on the performance of a radiomic model for patient survival was explored by extracting radiomic hand-crafted and deep-learning features from manual and automatic contours of dataset A, and feeding different machine learning algorithms to classify survival above/below median. Models' accuracies were assessed and compared. RESULTS The best agreement between automatic and manual contours with DICE=0.78 +(0.12) was achieved by averaging predictions from 2D and 3D models, and applying a post-processing technique to extract the maximum connected component. No statistical differences were observed in the performances of survival models when using manual or automatic contours, hand-crafted, or deep features. The best classifier showed an accuracy between 0.65 and 0.78. CONCLUSION The promising role of nnU-Net for automatic segmentation of lung lesions was confirmed, dramatically reducing the time-consuming physicians' workload without impairing the accuracy of survival predictive models based on radiomics.
△ Less
Submitted 24 September, 2022;
originally announced September 2022.
-
Improving Order with Queues
Authors:
Andreas Karrenbauer,
Leonie Wennmann,
Kurt Mehlhorn,
Pranabendu Misra,
Paolo Luigi Rinaldi,
Anna Twelsiek,
Siavash Rahimi Shateranloo,
Alireza Haqi
Abstract:
Patience Sort sorts a sequence of numbers with a minimal number of queues that work according to the First-In-First-Out (FIFO) principle. More precisely, if the length of the longest descreasing subsequence of the input sequence is $L$, then Patience Sort uses $L$ queues. We ask how much one can improve order with $k$ queues, where $k < L$? We address this question for two measures of sortedness:…
▽ More
Patience Sort sorts a sequence of numbers with a minimal number of queues that work according to the First-In-First-Out (FIFO) principle. More precisely, if the length of the longest descreasing subsequence of the input sequence is $L$, then Patience Sort uses $L$ queues. We ask how much one can improve order with $k$ queues, where $k < L$? We address this question for two measures of sortedness: number of down-steps and length of the longest descreasing subsequence. For the first measure, we give an optimal algorithm. For the second measure, we give an algorithm that reduces the LDS from $L$ to $L - k + 1$, and we provide a sequence with LDS $L$ that can't be reduced to an LDS lower than $L - k + 1$ with $k$ queues. Moreover, we study the mergeability of two sequences of numbers, providing an optimal linear algorithm for two queues with LDS $\leq 2$. The research was inspired by a problem arising in car manufacturing.
△ Less
Submitted 30 June, 2023; v1 submitted 6 July, 2022;
originally announced July 2022.
-
Quantification of pulmonary involvement in COVID-19 pneumonia by means of a cascade oftwo U-nets: training and assessment on multipledatasets using different annotation criteria
Authors:
Francesca Lizzi,
Abramo Agosti,
Francesca Brero,
Raffaella Fiamma Cabini,
Maria Evelina Fantacci,
Silvia Figini,
Alessandro Lascialfari,
Francesco Laruina,
Piernicola Oliva,
Stefano Piffer,
Ian Postuma,
Lisa Rinaldi,
Cinzia Talamonti,
Alessandra Retico
Abstract:
The automatic assignment of a severity score to the CT scans of patients affected by COVID-19 pneumonia could reduce the workload in radiology departments. This study aims at exploiting Artificial intelligence (AI) for the identification, segmentation and quantification of COVID-19 pulmonary lesions. We investigated the effects of using multiple datasets, heterogeneously populated and annotated ac…
▽ More
The automatic assignment of a severity score to the CT scans of patients affected by COVID-19 pneumonia could reduce the workload in radiology departments. This study aims at exploiting Artificial intelligence (AI) for the identification, segmentation and quantification of COVID-19 pulmonary lesions. We investigated the effects of using multiple datasets, heterogeneously populated and annotated according to different criteria. We developed an automated analysis pipeline, the LungQuant system, based on a cascade of two U-nets. The first one (U-net_1) is devoted to the identification of the lung parenchyma, the second one (U-net_2) acts on a bounding box enclosing the segmented lungs to identify the areas affected by COVID-19 lesions. Different public datasets were used to train the U-nets and to evaluate their segmentation performances, which have been quantified in terms of the Dice index. The accuracy in predicting the CT-Severity Score (CT-SS) of the LungQuant system has been also evaluated. Both Dice and accuracy showed a dependency on the quality of annotations of the available data samples. On an independent and publicly available benchmark dataset, the Dice values measured between the masks predicted by LungQuant system and the reference ones were 0.95$\pm$0.01 and 0.66$\pm$0.13 for the segmentation of lungs and COVID-19 lesions, respectively. The accuracy of 90% in the identification of the CT-SS on this benchmark dataset was achieved. We analysed the impact of using data samples with different annotation criteria in training an AI-based quantification system for pulmonary involvement in COVID-19 pneumonia. In terms of the Dice index, the U-net segmentation quality strongly depends on the quality of the lesion annotations. Nevertheless, the CT-SS can be accurately predicted on independent validation sets, demonstrating the satisfactory generalization ability of the LungQuant.
△ Less
Submitted 6 May, 2021;
originally announced May 2021.
-
Automatic Cell Counting in Flourescent Microscopy Using Deep Learning
Authors:
R. Morelli,
L. Clissa,
M. Dalla,
M. Luppi,
L. Rinaldi,
A. Zoccoli
Abstract:
Counting cells in fluorescent microscopy is a tedious, time-consuming task that researchers have to accomplish to assess the effects of different experimental conditions on biological structures of interest. Although such objects are generally easy to identify, the process of manually annotating cells is sometimes subject to arbitrariness due to the operator's interpretation of the borderline case…
▽ More
Counting cells in fluorescent microscopy is a tedious, time-consuming task that researchers have to accomplish to assess the effects of different experimental conditions on biological structures of interest. Although such objects are generally easy to identify, the process of manually annotating cells is sometimes subject to arbitrariness due to the operator's interpretation of the borderline cases.
We propose a Machine Learning approach that exploits a fully-convolutional network in a binary segmentation fashion to localize the objects of interest. Counts are then retrieved as the number of detected items.
Specifically, we adopt a UNet-like architecture leveraging residual units and an extended bottleneck for enlarging the field-of-view. In addition, we make use of weighted maps that penalize the errors on cells boundaries increasingly with overcrowding. These changes provide more context and force the model to focus on relevant features during pixel-wise classification. As a result, the model performance is enhanced, especially in presence of clum** cells, artifacts and confounding biological structures. Posterior assessment of the results with domain experts confirms that the model detects cells of interest correctly. The model demonstrates a human-level ability inasmuch even erroneous predictions seem to fall within the limits of operator interpretation. This qualitative assessment is also corroborated by quantitative metrics as an ${F_1}$ score of 0.87.
Despite some difficulties in interpretation, results are also satisfactory with respect to the counting task, as testified by mean and median absolute error of, respectively, 0.8 and 1.
△ Less
Submitted 24 February, 2021;
originally announced March 2021.
-
Using HEP experiment workflows for the benchmarking and accounting of WLCG computing resources
Authors:
Andrea Valassi,
Manfred Alef,
Jean-Michel Barbet,
Olga Datskova,
Riccardo De Maria,
Miguel Fontes Medeiros,
Domenico Giordano,
Costin Grigoras,
Christopher Hollowell,
Martina Javurkova,
Viktor Khristenko,
David Lange,
Michele Michelotto,
Lorenzo Rinaldi,
Andrea SciabĂ ,
Cas Van Der Laan
Abstract:
Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (HS06) suite for over a decade. It has recently become clear that HS06, which is based on real applications from non-HEP domains, no longer describes typical HEP workloads. The aim of the HEP-Benchmarks project is to develop a new benchmark suite for WLCG compute resources, based on real applications from the LHC experiments. B…
▽ More
Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (HS06) suite for over a decade. It has recently become clear that HS06, which is based on real applications from non-HEP domains, no longer describes typical HEP workloads. The aim of the HEP-Benchmarks project is to develop a new benchmark suite for WLCG compute resources, based on real applications from the LHC experiments. By construction, these new benchmarks are thus guaranteed to have a score highly correlated to the throughputs of HEP applications, and a CPU usage pattern similar to theirs. Linux containers and the CernVM-FS filesystem are the two main technologies enabling this approach, which had been considered impossible in the past. In this paper, we review the motivation, implementation and outlook of the new benchmark suite.
△ Less
Submitted 25 June, 2020; v1 submitted 3 April, 2020;
originally announced April 2020.
-
Automating ATLAS Computing Operations using the Site Status Board
Authors:
Julia Andreeva,
Carlos Borrego Iglesias,
Simone Campana,
Alessandro Di Girolamo,
Ivan Dzhunov,
Xavier Espinal Curull,
Stavro Gayazov,
Erekle Magradze,
Michal Maciej Nowotka,
Lorenzo Rinaldi,
Pablo Saiz,
Jaroslava Schovancova,
Graeme Andrew Stewart,
Michael Wright
Abstract:
The automation of operations is essential to reduce manpower costs and improve the reliability of the system. The Site Status Board (SSB) is a framework which allows Virtual Organizations to monitor their computing activities at distributed sites and to evaluate site performance. The ATLAS experiment intensively uses the SSB for the distributed computing shifts, for estimating data processing and…
▽ More
The automation of operations is essential to reduce manpower costs and improve the reliability of the system. The Site Status Board (SSB) is a framework which allows Virtual Organizations to monitor their computing activities at distributed sites and to evaluate site performance. The ATLAS experiment intensively uses the SSB for the distributed computing shifts, for estimating data processing and data transfer efficiencies at a particular site, and for implementing automatic exclusion of sites from computing activities, in case of potential problems. The ATLAS SSB provides a real-time aggregated monitoring view and keeps the history of the monitoring metrics. Based on this history, usability of a site from the perspective of ATLAS is calculated. The paper will describe how the SSB is integrated in the ATLAS operations and computing infrastructure and will cover implementation details of the ATLAS SSB sensors and alarm system, based on the information in the SSB. It will demonstrate the positive impact of the use of the SSB on the overall performance of ATLAS computing activities and will overview future plans.
△ Less
Submitted 28 January, 2013; v1 submitted 1 January, 2013;
originally announced January 2013.