-
Dissecting Query-Key Interaction in Vision Transformers
Authors:
Xu Pan,
Aaron Philip,
Ziqian Xie,
Odelia Schwartz
Abstract:
Self-attention in vision transformers is often thought to perform perceptual grou** where tokens attend to other tokens with similar embeddings, which could correspond to semantically similar features of an object. However, attending to dissimilar tokens can be beneficial by providing contextual information. We propose to use the Singular Value Decomposition to dissect the query-key interaction…
▽ More
Self-attention in vision transformers is often thought to perform perceptual grou** where tokens attend to other tokens with similar embeddings, which could correspond to semantically similar features of an object. However, attending to dissimilar tokens can be beneficial by providing contextual information. We propose to use the Singular Value Decomposition to dissect the query-key interaction (i.e. ${\textbf{W}_q}^\top\textbf{W}_k$). We find that early layers attend more to similar tokens, while late layers show increased attention to dissimilar tokens, providing evidence corresponding to perceptual grou** and contextualization, respectively. Many of these interactions between features represented by singular vectors are interpretable and semantic, such as attention between relevant objects, between parts of an object, or between the foreground and background. This offers a novel perspective on interpreting the attention mechanism, which contributes to understanding how transformer models utilize context and salient features when processing images.
△ Less
Submitted 26 May, 2024; v1 submitted 4 April, 2024;
originally announced May 2024.
-
A Mixed-Integer Conic Program for the Moving-Target Traveling Salesman Problem based on a Graph of Convex Sets
Authors:
Allen George Philip,
Zhongqiang Ren,
Sivakumar Rathinam,
Howie Choset
Abstract:
This paper introduces a new formulation that finds the optimum for the Moving-Target Traveling Salesman Problem (MT-TSP), which seeks to find a shortest path for an agent, that starts at a depot, visits a set of moving targets exactly once within their assigned time-windows, and returns to the depot. The formulation relies on the key idea that when the targets move along lines, their trajectories…
▽ More
This paper introduces a new formulation that finds the optimum for the Moving-Target Traveling Salesman Problem (MT-TSP), which seeks to find a shortest path for an agent, that starts at a depot, visits a set of moving targets exactly once within their assigned time-windows, and returns to the depot. The formulation relies on the key idea that when the targets move along lines, their trajectories become convex sets within the space-time coordinate system. The problem then reduces to finding the shortest path within a graph of convex sets, subject to some speed constraints. We compare our formulation with the current state-of-the-art Mixed Integer Conic Program (MICP) solver for the MT-TSP. The experimental results show that our formulation outperforms the MICP for instances with up to 20 targets, with up to two orders of magnitude reduction in runtime, and up to a 60\% tighter optimality gap. We also show that the solution cost from the convex relaxation of our formulation provides significantly tighter lower bounds for the MT-TSP than the ones from the MICP.
△ Less
Submitted 10 March, 2024; v1 submitted 7 March, 2024;
originally announced March 2024.
-
Leveraging cough sounds to optimize chest x-ray usage in low-resource settings
Authors:
Alexander Philip,
Sanya Chawla,
Lola Jover,
George P. Kafentzis,
Joe Brew,
Vishakh Saraf,
Shibu Vijayan,
Peter Small,
Carlos Chaccour
Abstract:
Chest X-ray is a commonly used tool during triage, diagnosis and management of respiratory diseases. In resource-constricted settings, optimizing this resource can lead to valuable cost savings for the health care system and the patients as well as to and improvement in consult time. We used prospectively-collected data from 137 patients referred for chest X-ray at the Christian Medical Center and…
▽ More
Chest X-ray is a commonly used tool during triage, diagnosis and management of respiratory diseases. In resource-constricted settings, optimizing this resource can lead to valuable cost savings for the health care system and the patients as well as to and improvement in consult time. We used prospectively-collected data from 137 patients referred for chest X-ray at the Christian Medical Center and Hospital (CMCH) in Purnia, Bihar, India. Each patient provided at least five coughs while awaiting radiography. Collected cough sounds were analyzed using acoustic AI methods. Cross-validation was done on temporal and spectral features on the cough sounds of each patient. Features were summarized using standard statistical approaches. Three models were developed, tested and compared in their capacity to predict an abnormal result in the chest X-ray. All three methods yielded models that could discriminate to some extent between normal and abnormal with the logistic regression performing best with an area under the receiver operating characteristic curves ranging from 0.7 to 0.78. Despite limitations and its relatively small sample size, this study shows that AI-enabled algorithms can use cough sounds to predict which individuals presenting for chest radiographic examination will have a normal or abnormal results. These results call for expanding this research given the potential optimization of limited health care resources in low- and middle-income countries.
△ Less
Submitted 13 February, 2024;
originally announced February 2024.
-
C*: A New Bounding Approach for the Moving-Target Traveling Salesman Problem
Authors:
Allen George Philip,
Zhongqiang Ren,
Sivakumar Rathinam,
Howie Choset
Abstract:
We introduce a new bounding approach called Continuity* (C*) that provides optimality guarantees to the Moving-Target Traveling Salesman Problem (MT-TSP). Our approach relies on relaxing the continuity constraints on the agent's tour. This is done by partitioning the targets' trajectories into small sub-segments and allowing the agent to arrive at any point in one of the sub-segments and depart fr…
▽ More
We introduce a new bounding approach called Continuity* (C*) that provides optimality guarantees to the Moving-Target Traveling Salesman Problem (MT-TSP). Our approach relies on relaxing the continuity constraints on the agent's tour. This is done by partitioning the targets' trajectories into small sub-segments and allowing the agent to arrive at any point in one of the sub-segments and depart from any point in the same sub-segment when visiting each target. This lets us pose the bounding problem as a Generalized Traveling Salesman Problem (GTSP) in a graph where the cost of traveling an edge requires us to solve a new problem called the Shortest Feasible Travel (SFT). We also introduce C*-lite, which follows the same approach as C*, but uses simple and easy to compute lower-bounds to the SFT. We first prove that the proposed algorithms provide lower bounds to the MT-TSP. We also provide computational results to corroborate the performance of C* and C*-lite for instances with up to 15 targets. For the special case where targets travel along lines, we compare our C* variants with the SOCP based method, which is the current state-of-the-art solver for MT-TSP. While the SOCP based method performs well for instances with 5 and 10 targets, C* outperforms the SOCP based method for instances with 15 targets. For the general case, on average, our approaches find feasible solutions within ~4% of the lower bounds for the tested instances.
△ Less
Submitted 9 December, 2023;
originally announced December 2023.
-
Schrödinger as a Quantum Programmer: Estimating Entanglement via Steering
Authors:
Aby Philip,
Soorya Rethinasamy,
Vincent Russo,
Mark M. Wilde
Abstract:
Quantifying entanglement is an important task by which the resourcefulness of a quantum state can be measured. Here, we develop a quantum algorithm that tests for and quantifies the separability of a general bipartite state by using the quantum steering effect, the latter initially discovered by Schrödinger. Our separability test consists of a distributed quantum computation involving two parties:…
▽ More
Quantifying entanglement is an important task by which the resourcefulness of a quantum state can be measured. Here, we develop a quantum algorithm that tests for and quantifies the separability of a general bipartite state by using the quantum steering effect, the latter initially discovered by Schrödinger. Our separability test consists of a distributed quantum computation involving two parties: a computationally limited client, who prepares a purification of the state of interest, and a computationally unbounded server, who tries to steer the reduced systems to a probabilistic ensemble of pure product states. To design a practical algorithm, we replace the role of the server with a combination of parameterized unitary circuits and classical optimization techniques to perform the necessary computation. The result is a variational quantum steering algorithm (VQSA), a modified separability test that is implementable on quantum computers that are available today. We then simulate our VQSA on noisy quantum simulators and find favorable convergence properties on the examples tested. We also develop semidefinite programs, executable on classical computers, that benchmark the results obtained from our VQSA. Thus, our findings provide a meaningful connection between steering, entanglement, quantum algorithms, and quantum computational complexity theory. They also demonstrate the value of a parameterized mid-circuit measurement in a VQSA.
△ Less
Submitted 1 June, 2024; v1 submitted 14 March, 2023;
originally announced March 2023.
-
Prediction of ICD Codes with Clinical BERT Embeddings and Text Augmentation with Label Balancing using MIMIC-III
Authors:
Brent Biseda,
Gaurav Desai,
Haifeng Lin,
Anish Philip
Abstract:
This paper achieves state of the art results for the ICD code prediction task using the MIMIC-III dataset. This was achieved through the use of Clinical BERT (Alsentzer et al., 2019). embeddings and text augmentation and label balancing to improve F1 scores for both ICD Chapter as well as ICD disease codes. We attribute the improved performance mainly to the use of novel text augmentation to shuff…
▽ More
This paper achieves state of the art results for the ICD code prediction task using the MIMIC-III dataset. This was achieved through the use of Clinical BERT (Alsentzer et al., 2019). embeddings and text augmentation and label balancing to improve F1 scores for both ICD Chapter as well as ICD disease codes. We attribute the improved performance mainly to the use of novel text augmentation to shuffle the order of sentences during training. In comparison to the Top-32 ICD code prediction (Keyang Xu, et. al.) with an F1 score of 0.76, we achieve a final F1 score of 0.75 but on a total of the top 50 ICD codes.
△ Less
Submitted 24 August, 2020;
originally announced August 2020.
-
Dr. Tux: A Question Answering System for Ubuntu users
Authors:
Bijil Abraham Philip,
Manas Jog,
Apurv Milind Upasani
Abstract:
Various forums and question answering (Q&A) sites are available online that allow Ubuntu users to find results similar to their queries. However, searching for a result is often time consuming as it requires the user to find a specific problem instance relevant to his/her query from a large set of questions. In this paper, we present an automated question answering system for Ubuntu users called D…
▽ More
Various forums and question answering (Q&A) sites are available online that allow Ubuntu users to find results similar to their queries. However, searching for a result is often time consuming as it requires the user to find a specific problem instance relevant to his/her query from a large set of questions. In this paper, we present an automated question answering system for Ubuntu users called Dr. Tux that is designed to answer user's queries by selecting the most similar question from an online database. The prototype was implemented in Python and uses NLTK and CoreNLP tools for Natural Language Processing. The data for the prototype was taken from the AskUbuntu website which contains about 150k questions. The results obtained from the manual evaluation of the prototype were promising while also presenting some interesting opportunities for improvement.
△ Less
Submitted 25 August, 2018;
originally announced August 2018.
-
Development of a Window Based Security System for Electronic Data Interchange
Authors:
Achimugu Philip,
Oluwagbemi Oluwatolani,
Abah Joshua
Abstract:
The Electronic Data Interchange (EDI) is the exchange of standardized documents between computer systems for business use. The objective of this study is to make Electronic Data Interchange secure to use and to eliminate human intervention in the transfer of data between business partners so that productivity and efficiency can be improved and also promote its usage between two or more trading org…
▽ More
The Electronic Data Interchange (EDI) is the exchange of standardized documents between computer systems for business use. The objective of this study is to make Electronic Data Interchange secure to use and to eliminate human intervention in the transfer of data between business partners so that productivity and efficiency can be improved and also promote its usage between two or more trading organizations. This paper provides an overview of EDI by describing the traditional problems of exchanging information in business environments and how the EDI solves those problems and gives benefits to the company that makes use of EDI. This paper also introduces the common EDI Standards and explains how it works, how it is used over the internet and the security measures implemented. The system was executed on both local area network and wide area network after a critical study of the existing EDI methods and also implemented using VB.Net programming language. Finally, an interactive program was developed that handles the transfer of files, with special attention to the security of the items that are being transferred from one computer workstation to another.
△ Less
Submitted 4 August, 2011;
originally announced August 2011.
-
The Impact of Information Technology in Nigeria's Banking Industry
Authors:
Oluwagbemi Oluwatolani,
Abah Joshua,
Achimugu Philip
Abstract:
Today, information technology (IT) has become a key element in economic development and a backbone of knowledge-based economies in terms of operations, quality delivery of services and productivity of services. Therefore, taking advantage of information technologies (IT) is an increasing challenge for develo** countries. There is now growing evidence that Knowledge-driven innovation is a decisiv…
▽ More
Today, information technology (IT) has become a key element in economic development and a backbone of knowledge-based economies in terms of operations, quality delivery of services and productivity of services. Therefore, taking advantage of information technologies (IT) is an increasing challenge for develo** countries. There is now growing evidence that Knowledge-driven innovation is a decisive factor in the competitiveness of nations, industries, organizations and firms. Organizations like the banking sector have benefited substantially from e-banking, which is one among the IT applications for strengthening the competitiveness. This paper presents the current trend in the application of IT in the banking industries in Nigeria and gives an insight into how quality banking has been enhanced via IT. The paper further reveals that the deployment of IT facilities in the Nigerian Banking industry has brought about fundamental changes in the content and quality of banking business in the country. This analysis and clarification of how Nigerian Banks have used IT to reengineer their operations is detailed through literature review and observation. Three categories of variables that relate to the use and implementation of information technology devices were considered in this paper. These include the nature and degree of adoption of innovative technologies; degree of utilization of the identified technologies; and the impact of the adoption of IT devices on the bank operations.
△ Less
Submitted 4 August, 2011;
originally announced August 2011.
-
Development and Deployment of Fixed Wireless Access in South West Nigeria: Performance and Evaluation
Authors:
Oluwaranti Adeniran,
Achimugu Philip
Abstract:
Fixed Wireless Access (FWA) involves the use of wireless technology to replace copper to connect subscribers to the telephone network. It is a variant of wireless broadband which provides an alternative in the so-called 'last mile' connectivity between the subscriber and the fixed telecommunications network. FWA could either be narrowband or broadband and it is predominantly deployed using the Cod…
▽ More
Fixed Wireless Access (FWA) involves the use of wireless technology to replace copper to connect subscribers to the telephone network. It is a variant of wireless broadband which provides an alternative in the so-called 'last mile' connectivity between the subscriber and the fixed telecommunications network. FWA could either be narrowband or broadband and it is predominantly deployed using the Code Division Multiple Access (CDMA) technology. In assessing the extent of development and deployment of FWA, the perspective of the operators and users was elicited primarily through the use of questionnaires. Issues like setup cost, tax, Government incentive, availability of infrastructure and manpower applied to the operators while on the users' part factors like quality of service, signal strength as well as call rate were considered. The South western zone of Nigeria is regarded as one of the most urbanized regions in the south of Sahara, this is not out of place considering the fact that Lagos which is the nation's commercial nerve centre and a mega-city is located within this region. The scope of this research covered this very lively part of the country. The relationship between the parameters or variables considered was established using an appropriate statistical method: The Regression analysis. In terms of users' preference, Global System of Mobile communication (GSM) was compared with FWA. Results were interpreted and suitable conclusions were drawn to wrap up a quite revealing work.
△ Less
Submitted 4 August, 2011;
originally announced August 2011.
