-
Geometric Constraints in Deep Learning Frameworks: A Survey
Authors:
Vibhas K Vats,
David J Crandall
Abstract:
Stereophotogrammetry is an emerging technique of scene understanding. Its origins go back to at least the 1800s when people first started to investigate using photographs to measure the physical properties of the world. Since then, thousands of approaches have been explored. The classic geometric techniques of Shape from Stereo is built on using geometry to define constraints on scene and camera g…
▽ More
Stereophotogrammetry is an emerging technique of scene understanding. Its origins go back to at least the 1800s when people first started to investigate using photographs to measure the physical properties of the world. Since then, thousands of approaches have been explored. The classic geometric techniques of Shape from Stereo is built on using geometry to define constraints on scene and camera geometry and then solving the non-linear systems of equations. More recent work has taken an entirely different approach, using end-to-end deep learning without any attempt to explicitly model the geometry. In this survey, we explore the overlap for geometric-based and deep learning-based frameworks. We compare and contrast geometry enforcing constraints integrated into a deep learning framework for depth estimation or other closely related problems. We present a new taxonomy for prevalent geometry enforcing constraints used in modern deep learning frameworks. We also present insightful observations and potential future research directions.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
A Survey on Human-AI Teaming with Large Pre-Trained Models
Authors:
Vanshika Vats,
Marzia Binta Nizam,
Minghao Liu,
Ziyuan Wang,
Richard Ho,
Mohnish Sai Prasad,
Vincent Titterton,
Sai Venkat Malreddy,
Riya Aggarwal,
Yanwen Xu,
Lei Ding,
Jay Mehta,
Nathan Grinnell,
Li Liu,
Sijia Zhong,
Devanathan Nallur Gandamani,
Xinyi Tang,
Rohan Ghosalkar,
Celeste Shen,
Rachel Shen,
Nafisa Hussain,
Kesav Ravichandran,
James Davis
Abstract:
In the rapidly evolving landscape of artificial intelligence (AI), the collaboration between human intelligence and AI systems, known as Human-AI (HAI) Teaming, has emerged as a cornerstone for advancing problem-solving and decision-making processes. The advent of Large Pre-trained Models (LPtM) has significantly transformed this landscape, offering unprecedented capabilities by leveraging vast am…
▽ More
In the rapidly evolving landscape of artificial intelligence (AI), the collaboration between human intelligence and AI systems, known as Human-AI (HAI) Teaming, has emerged as a cornerstone for advancing problem-solving and decision-making processes. The advent of Large Pre-trained Models (LPtM) has significantly transformed this landscape, offering unprecedented capabilities by leveraging vast amounts of data to understand and predict complex patterns. This paper surveys the pivotal integration of LPtMs with HAI, emphasizing how these models enhance collaborative intelligence beyond traditional approaches. It examines the potential of LPtMs in augmenting human capabilities, discussing this collaboration for AI model improvements, effective teaming, ethical considerations, and their broad applied implications in various sectors. Through this exploration, the study sheds light on the transformative impact of LPtM-enhanced HAI Teaming, providing insights for future research, policy development, and strategic implementations aimed at harnessing the full potential of this collaboration for research and societal benefit.
△ Less
Submitted 26 June, 2024; v1 submitted 7 March, 2024;
originally announced March 2024.
-
Assessing the Impact of Prompting Methods on ChatGPT's Mathematical Capabilities
Authors:
Yuhao Chen,
Chloe Wong,
Hanwen Yang,
Juan Aguenza,
Sai Bhujangari,
Benthan Vu,
Xun Lei,
Amisha Prasad,
Manny Fluss,
Eric Phuong,
Minghao Liu,
Raja Kumar,
Vanshika Vats,
James Davis
Abstract:
This study critically evaluates the efficacy of prompting methods in enhancing the mathematical reasoning capability of large language models (LLMs). The investigation uses three prescriptive prompting methods - simple, persona, and conversational prompting - known for their effectiveness in enhancing the linguistic tasks of LLMs. We conduct this analysis on OpenAI's LLM chatbot, ChatGPT-3.5, on e…
▽ More
This study critically evaluates the efficacy of prompting methods in enhancing the mathematical reasoning capability of large language models (LLMs). The investigation uses three prescriptive prompting methods - simple, persona, and conversational prompting - known for their effectiveness in enhancing the linguistic tasks of LLMs. We conduct this analysis on OpenAI's LLM chatbot, ChatGPT-3.5, on extensive problem sets from the MATH, GSM8K, and MMLU datasets, encompassing a broad spectrum of mathematical challenges. A grading script adapted to each dataset is used to determine the effectiveness of these prompting interventions in enhancing the model's mathematical analysis power. Contrary to expectations, our empirical analysis reveals that none of the investigated methods consistently improves over ChatGPT-3.5's baseline performance, with some causing significant degradation. Our findings suggest that prompting strategies do not necessarily generalize to new domains, in this study failing to enhance mathematical performance.
△ Less
Submitted 20 February, 2024; v1 submitted 22 December, 2023;
originally announced December 2023.
-
GC-MVSNet: Multi-View, Multi-Scale, Geometrically-Consistent Multi-View Stereo
Authors:
Vibhas K. Vats,
Sripad Joshi,
David J. Crandall,
Md. Alimoor Reza,
Soon-heung Jung
Abstract:
Traditional multi-view stereo (MVS) methods rely heavily on photometric and geometric consistency constraints, but newer machine learning-based MVS methods check geometric consistency across multiple source views only as a post-processing step. In this paper, we present a novel approach that explicitly encourages geometric consistency of reference view depth maps across multiple source views at di…
▽ More
Traditional multi-view stereo (MVS) methods rely heavily on photometric and geometric consistency constraints, but newer machine learning-based MVS methods check geometric consistency across multiple source views only as a post-processing step. In this paper, we present a novel approach that explicitly encourages geometric consistency of reference view depth maps across multiple source views at different scales during learning (see Fig. 1). We find that adding this geometric consistency loss significantly accelerates learning by explicitly penalizing geometrically inconsistent pixels, reducing the training iteration requirements to nearly half that of other MVS methods. Our extensive experiments show that our approach achieves a new state-of-the-art on the DTU and BlendedMVS datasets, and competitive results on the Tanks and Temples benchmark. To the best of our knowledge, GC-MVSNet is the first attempt to enforce multi-view, multi-scale geometric consistency during learning.
△ Less
Submitted 21 December, 2023; v1 submitted 30 October, 2023;
originally announced October 2023.
-
Controlling the Quality of Distillation in Response-Based Network Compression
Authors:
Vibhas Vats,
David Crandall
Abstract:
The performance of a distillation-based compressed network is governed by the quality of distillation. The reason for the suboptimal distillation of a large network (teacher) to a smaller network (student) is largely attributed to the gap in the learning capacities of given teacher-student pair. While it is hard to distill all the knowledge of a teacher, the quality of distillation can be controll…
▽ More
The performance of a distillation-based compressed network is governed by the quality of distillation. The reason for the suboptimal distillation of a large network (teacher) to a smaller network (student) is largely attributed to the gap in the learning capacities of given teacher-student pair. While it is hard to distill all the knowledge of a teacher, the quality of distillation can be controlled to a large extent to achieve better performance. Our experiments show that the quality of distillation is largely governed by the quality of teacher's response, which in turn is heavily affected by the presence of similarity information in its response. A well-trained large capacity teacher loses similarity information between classes in the process of learning fine-grained discriminative properties for classification. The absence of similarity information causes the distillation process to be reduced from one example-many class learning to one example-one class learning, thereby throttling the flow of diverse knowledge from the teacher. With the implicit assumption that only the instilled knowledge can be distilled, instead of focusing only on the knowledge distilling process, we scrutinize the knowledge inculcation process. We argue that for a given teacher-student pair, the quality of distillation can be improved by finding the sweet spot between batch size and number of epochs while training the teacher. We discuss the steps to find this sweet spot for better distillation. We also propose the distillation hypothesis to differentiate the behavior of the distillation process between knowledge distillation and regularization effect. We conduct all our experiments on three different datasets.
△ Less
Submitted 18 December, 2021;
originally announced December 2021.
-
SURF-SVM Based Identification and Classification of Gastrointestinal Diseases in Wireless Capsule Endoscopy
Authors:
Vanshika Vats,
Pooja Goel,
Amodini Agarwal,
Nidhi Goel
Abstract:
Endoscopy provides a major contribution to the diagnosis of the Gastrointestinal Tract (GIT) diseases. With Colon Endoscopy having its certain limitations, Wireless Capsule Endoscopy is gradually taking over it in the terms of ease and efficiency. WCE is performed with a miniature optical endoscope which is swallowed by the patient and transmits colour images wirelessly during its journey through…
▽ More
Endoscopy provides a major contribution to the diagnosis of the Gastrointestinal Tract (GIT) diseases. With Colon Endoscopy having its certain limitations, Wireless Capsule Endoscopy is gradually taking over it in the terms of ease and efficiency. WCE is performed with a miniature optical endoscope which is swallowed by the patient and transmits colour images wirelessly during its journey through the GIT, inside the body of the patient. These images are used to implement an effective and computationally efficient approach which aims to detect the abnormal and normal tissues in the GIT automatically, and thus helps in reducing the manual work of the reviewers. The algorithm further aims to classify the diseased tissues into various GIT diseases that are commonly known to be affecting the tract. In this manuscript, the descriptor used for the detection of the interest points is Speeded Up Robust Features (SURF), which uses the colour information contained in the images which is converted to CIELAB space colours for better identification. The features extracted at the interest points are then used to train and test a Support Vector Machine (SVM), so that it automatically classifies the images into normal or abnormal and further detects the specific abnormalities. SVM, along with a few parameters, gives a very high accuracy of 94.58% while classifying normal and abnormal images and an accuracy of 82.91% while classifying into multi-class. The present work is an improvement on the previously reported analyses which were only limited to the bi-class classification using this approach.
△ Less
Submitted 2 September, 2020;
originally announced September 2020.
