Showing 1–11 of 11 results for author: Tavakkol, S

Gemini: A Family of Highly Capable Multimodal Models

Authors: Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee , et al. (1325 additional authors not shown)

Abstract: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr… ▽ More This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI. △ Less

Submitted 17 June, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

Two-Step Active Learning for Instance Segmentation with Uncertainty and Diversity Sampling

Authors: Ke Yu, Stephen Albro, Giulia DeSalvo, Suraj Kothawade, Abdullah Rashwan, Sasan Tavakkol, Kayhan Batmanghelich, Xiaoqi Yin

Abstract: Training high-quality instance segmentation models requires an abundance of labeled images with instance masks and classifications, which is often expensive to procure. Active learning addresses this challenge by striving for optimum performance with minimal labeling cost by selecting the most informative and representative images for labeling. Despite its potential, active learning has been less… ▽ More Training high-quality instance segmentation models requires an abundance of labeled images with instance masks and classifications, which is often expensive to procure. Active learning addresses this challenge by striving for optimum performance with minimal labeling cost by selecting the most informative and representative images for labeling. Despite its potential, active learning has been less explored in instance segmentation compared to other tasks like image classification, which require less labeling. In this study, we propose a post-hoc active learning algorithm that integrates uncertainty-based sampling with diversity-based sampling. Our proposed algorithm is not only simple and easy to implement, but it also delivers superior performance on various datasets. Its practical application is demonstrated on a real-world overhead imagery dataset, where it increases the labeling efficiency fivefold. △ Less

Submitted 27 September, 2023; originally announced September 2023.

Comments: UNCV ICCV 2023

Agile Modeling: From Concept to Classifier in Minutes

Authors: Otilia Stretcu, Edward Vendrow, Kenji Hata, Krishnamurthy Viswanathan, Vittorio Ferrari, Sasan Tavakkol, Wenlei Zhou, Aditya Avinash, Enming Luo, Neil Gordon Alldrin, MohammadHossein Bateni, Gabriel Berger, Andrew Bunner, Chun-Ta Lu, Javier A Rey, Giulia DeSalvo, Ranjay Krishna, Ariel Fuxman

Abstract: The application of computer vision to nuanced subjective use cases is growing. While crowdsourcing has served the vision community well for most objective tasks (such as labeling a "zebra"), it now falters on tasks where there is substantial subjectivity in the concept (such as identifying "gourmet tuna"). However, empowering any user to develop a classifier for their concept is technically diffic… ▽ More The application of computer vision to nuanced subjective use cases is growing. While crowdsourcing has served the vision community well for most objective tasks (such as labeling a "zebra"), it now falters on tasks where there is substantial subjectivity in the concept (such as identifying "gourmet tuna"). However, empowering any user to develop a classifier for their concept is technically difficult: users are neither machine learning experts, nor have the patience to label thousands of examples. In reaction, we introduce the problem of Agile Modeling: the process of turning any subjective visual concept into a computer vision model through a real-time user-in-the-loop interactions. We instantiate an Agile Modeling prototype for image classification and show through a user study (N=14) that users can create classifiers with minimal effort under 30 minutes. We compare this user driven process with the traditional crowdsourcing paradigm and find that the crowd's notion often differs from that of the user's, especially as the concepts become more subjective. Finally, we scale our experiments with simulations of users training classifiers for ImageNet21k categories to further demonstrate the efficacy. △ Less

Submitted 12 May, 2023; v1 submitted 24 February, 2023; originally announced February 2023.

A First-Order Numerical Algorithm without Matrix Operations

Authors: Muhammad Adil, Ramtin Madani, Sasan Tavakkol, Ali Davoudi

Abstract: This paper offers a matrix-free first-order numerical method to solve large-scale conic optimization problems. Solving systems of linear equations pose the most computationally challenging part in both first-order and second-order numerical algorithms. Existing direct and indirect methods are either computationally expensive or compromise on solution accuracy. Alternatively, we propose an easy-to-… ▽ More This paper offers a matrix-free first-order numerical method to solve large-scale conic optimization problems. Solving systems of linear equations pose the most computationally challenging part in both first-order and second-order numerical algorithms. Existing direct and indirect methods are either computationally expensive or compromise on solution accuracy. Alternatively, we propose an easy-to-compute decomposition method to solve sparse linear systems that arise in conic optimization problems. Its iterations are tractable, highly parallelizable, with closed-form solutions. This algorithm can be easily implemented on distributed platforms, such as graphics processing units, with orders-of-magnitude time improvement. The performance of the proposed solver is demonstrated on large-scale conic optimization problems and is compared with the state-of-the-art first-order solvers. △ Less

Submitted 9 March, 2022; originally announced March 2022.

An Automatic Approach for Generating Rich, Linked Geo-Metadata from Historical Map Images

Authors: Zekun Li, Yao-Yi Chiang, Sasan Tavakkol, Basel Shbita, Johannes H. Uhl, Stefan Leyk, Craig A. Knoblock

Abstract: Historical maps contain detailed geographic information difficult to find elsewhere covering long-periods of time (e.g., 125 years for the historical topographic maps in the US). However, these maps typically exist as scanned images without searchable metadata. Existing approaches making historical maps searchable rely on tedious manual work (including crowd-sourcing) to generate the metadata (e.g… ▽ More Historical maps contain detailed geographic information difficult to find elsewhere covering long-periods of time (e.g., 125 years for the historical topographic maps in the US). However, these maps typically exist as scanned images without searchable metadata. Existing approaches making historical maps searchable rely on tedious manual work (including crowd-sourcing) to generate the metadata (e.g., geolocations and keywords). Optical character recognition (OCR) software could alleviate the required manual work, but the recognition results are individual words instead of location phrases (e.g., "Black" and "Mountain" vs. "Black Mountain"). This paper presents an end-to-end approach to address the real-world problem of finding and indexing historical map images. This approach automatically processes historical map images to extract their text content and generates a set of metadata that is linked to large external geospatial knowledge bases. The linked metadata in the RDF (Resource Description Framework) format support complex queries for finding and indexing historical maps, such as retrieving all historical maps covering mountain peaks higher than 1,000 meters in California. We have implemented the approach in a system called mapKurator. We have evaluated mapKurator using historical maps from several sources with various map styles, scales, and coverage. Our results show significant improvement over the state-of-the-art methods. The code has been made publicly available as modules of the Kartta Labs project at https://github.com/kartta-labs/Project. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: 10.1145/3394486.3403381

Rapid Convergence of First-Order Numerical Algorithms via Adaptive Conditioning

Authors: Muhammad Adil, Sasan Tavakkol, Ramtin Madani

Abstract: This paper is an attempt to remedy the problem of slow convergence for first-order numerical algorithms by proposing an adaptive conditioning heuristic. First, we propose a parallelizable numerical algorithm that is capable of solving large-scale conic optimization problems on distributed platforms such as {graphics processing unit} with orders-of-magnitude time improvement. Proof of global conver… ▽ More This paper is an attempt to remedy the problem of slow convergence for first-order numerical algorithms by proposing an adaptive conditioning heuristic. First, we propose a parallelizable numerical algorithm that is capable of solving large-scale conic optimization problems on distributed platforms such as {graphics processing unit} with orders-of-magnitude time improvement. Proof of global convergence is provided for the proposed algorithm. We argue that on the contrary to common belief, the condition number of the data matrix is not a reliable predictor of convergence speed. In light of this observation, an adaptive conditioning heuristic is proposed which enables higher accuracy compared to other first-order numerical algorithms. Numerical experiments on a wide range of large-scale linear programming and second-order cone programming problems demonstrate the scalability and computational advantages of the proposed algorithm compared to commercial and open-source state-of-the-art solvers. △ Less

Submitted 28 February, 2021; originally announced March 2021.

Kartta Labs: Collaborative Time Travel

Authors: Sasan Tavakkol, Feng Han, Brandon Mayer, Mark Phillips, Cyrus Shahabi, Yao-Yi Chiang, Raimondas Kiveris

Abstract: We introduce the modular and scalable design of Kartta Labs, an open source, open data, and scalable system for virtually reconstructing cities from historical maps and photos. Kartta Labs relies on crowdsourcing and artificial intelligence consisting of two major modules: Maps and 3D models. Each module, in turn, consists of sub-modules that enable the system to reconstruct a city from historical… ▽ More We introduce the modular and scalable design of Kartta Labs, an open source, open data, and scalable system for virtually reconstructing cities from historical maps and photos. Kartta Labs relies on crowdsourcing and artificial intelligence consisting of two major modules: Maps and 3D models. Each module, in turn, consists of sub-modules that enable the system to reconstruct a city from historical maps and photos. The result is a spatiotemporal reference that can be used to integrate various collected data (curated, sensed, or crowdsourced) for research, education, and entertainment purposes. The system empowers the users to experience collaborative time travel such that they work together to reconstruct the past and experience it on an open source and open data platform. △ Less

Submitted 6 October, 2020; originally announced October 2020.

Adaptive Third Order Adams-Bashforth Time Step** for Extended Boussinesq Equations

Authors: Sasan Tavakkol, Sangyoung Son, Patrick Lynett

Abstract: We develop the third-order adaptive Adams-Bashforth time step** and the second-order finite difference equation for variable time steps. We incorporate these schemes in the Celeris Advent software to discretize and solve the 2D extended Boussinesq equations. This software uses a hybrid finite volume - finite difference scheme and leverages the GPU to solve the equations faster than real-time whi… ▽ More We develop the third-order adaptive Adams-Bashforth time step** and the second-order finite difference equation for variable time steps. We incorporate these schemes in the Celeris Advent software to discretize and solve the 2D extended Boussinesq equations. This software uses a hybrid finite volume - finite difference scheme and leverages the GPU to solve the equations faster than real-time while concurrently visualizing them. We simulate several benchmarks using the adaptive time step** scheme of Celeris Advent and demonstrate the capability of the software in modeling wave-breaking, wave runup, irregular waves, and rip currents. The adaptive scheme significantly improves the robustness of the model while providing faster computational performance. △ Less

Submitted 10 November, 2020; v1 submitted 5 September, 2019; originally announced September 2019.

A Review on the Applications of Crowdsourcing in Human Pathology

Authors: Roshanak Alialy, Sasan Tavakkol, Elham Tavakkol, Amir Ghorbani-Aghbologhi, Alireza Ghaffarieh, Seon Ho Kim, Cyrus Shahabi

Abstract: The advent of the digital pathology has introduced new avenues of diagnostic medicine. Among them, crowdsourcing has attracted researchers' attention in the recent years, allowing them to engage thousands of untrained individuals in research and diagnosis. While there exist several articles in this regard, prior works have not collectively documented them. We, therefore, aim to review the applicat… ▽ More The advent of the digital pathology has introduced new avenues of diagnostic medicine. Among them, crowdsourcing has attracted researchers' attention in the recent years, allowing them to engage thousands of untrained individuals in research and diagnosis. While there exist several articles in this regard, prior works have not collectively documented them. We, therefore, aim to review the applications of crowdsourcing in human pathology in a semi-systematic manner. We firstly, introduce a novel method to do a systematic search of the literature. Utilizing this method, we, then, collect hundreds of articles and screen them against a pre-defined set of criteria. Furthermore, we crowdsource part of the screening process, to examine another potential application of crowdsourcing. Finally, we review the selected articles and characterize the prior uses of crowdsourcing in pathology. △ Less

Submitted 20 November, 2017; v1 submitted 9 October, 2017; originally announced October 2017.

Celeris: A GPU-accelerated open source software with a Boussinesq-type wave solver for real-time interactive simulation and visualization

Authors: Sasan Tavakkol, Patrick Lynett

Abstract: In this paper, we introduce an interactive coastal wave simulation and visualization software, called Celeris. Celeris is an open source software which needs minimum preparation to run on a Windows machine. The software solves the extended Boussinesq equations using a hybrid finite volume - finite difference method and supports moving shoreline boundaries. The simulation and visualization are perf… ▽ More In this paper, we introduce an interactive coastal wave simulation and visualization software, called Celeris. Celeris is an open source software which needs minimum preparation to run on a Windows machine. The software solves the extended Boussinesq equations using a hybrid finite volume - finite difference method and supports moving shoreline boundaries. The simulation and visualization are performed on the GPU using Direct3D libraries, which enables the software to run faster than real-time. Celeris provides a first-of-its-kind interactive modeling platform for coastal wave applications and it supports simultaneous visualization with both photorealistic and colormapped rendering capabilities. We validate our software through comparison with three standard benchmarks for non-breaking and breaking waves. △ Less

Submitted 20 September, 2017; v1 submitted 18 November, 2016; originally announced November 2016.

Journal ref: Computer Physics Communications 217 (2017): 117-127

Wall and Bed Shear Force in Rectangular Open Channels

Authors: Sasan Tavakkol, Amir Reza Zarrati

Abstract: This paper studies the effects of velocity gradients and secondary currents on the distribution of the shear force between the walls and bed of rectangular open channels. We show that neglecting the effect of secondary currents and assuming zero-shear division lines does not yield acceptable results. We, accordingly, introduce a method to determine the percentage of the total shear force acting on… ▽ More This paper studies the effects of velocity gradients and secondary currents on the distribution of the shear force between the walls and bed of rectangular open channels. We show that neglecting the effect of secondary currents and assuming zero-shear division lines does not yield acceptable results. We, accordingly, introduce a method to determine the percentage of the total shear force acting on the walls and bed of rectangular open channels, which takes both the velocity gradients and secondary currents into account. Using the channel bisectors, along which there is no secondary flows effect, and orthogonal trajectories to isovels, along which there is no shear stress, we divide the channel cross section into three major subsections, namely bed area, walls area, and shared area. The geometry of each subsection is derived given the location of the maximum velocity. The share of the bed and wall shear forces from the shared area, are calculated afterward. The results for bed and walls shear forces agree well with the experimental data with an average relative error less than 5% for both regular flows and flows carrying suspended sediment. This method also provides a physics-driven range for the wall and bed shear forces which nicely covers the experimental data. △ Less

Submitted 20 September, 2017; v1 submitted 5 November, 2016; originally announced November 2016.