-
Improving Noise Robustness through Abstractions and its Impact on Machine Learning
Authors:
Alfredo Ibias,
Karol Capala,
Varun Ravi Varma,
Anna Drozdz,
Jose Sousa
Abstract:
Noise is a fundamental problem in learning theory with huge effects in the application of Machine Learning (ML) methods, due to real world data tendency to be noisy. Additionally, introduction of malicious noise can make ML methods fail critically, as is the case with adversarial attacks. Thus, finding and develo** alternatives to improve robustness to noise is a fundamental problem in ML. In th…
▽ More
Noise is a fundamental problem in learning theory with huge effects in the application of Machine Learning (ML) methods, due to real world data tendency to be noisy. Additionally, introduction of malicious noise can make ML methods fail critically, as is the case with adversarial attacks. Thus, finding and develo** alternatives to improve robustness to noise is a fundamental problem in ML. In this paper, we propose a method to deal with noise: mitigating its effect through the use of data abstractions. The goal is to reduce the effect of noise over the model's performance through the loss of information produced by the abstraction. However, this information loss comes with a cost: it can result in an accuracy reduction due to the missing information. First, we explored multiple methodologies to create abstractions, using the training dataset, for the specific case of numerical data and binary classification tasks. We also tested how these abstractions can affect robustness to noise with several experiments that explore the robustness of an Artificial Neural Network to noise when trained using raw data \emph{vs} when trained using abstracted data. The results clearly show that using abstractions is a viable approach for develo** noise robust ML methods.
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
A Hybrid Deep Learning Classification of Perimetric Glaucoma Using Peripapillary Nerve Fiber Layer Reflectance and Other OCT Parameters from Three Anatomy Regions
Authors:
Ou Tan,
David S. Greenfield,
Brian A. Francis,
Rohit Varma,
Joel S. Schuman,
David Huang,
Dongseok Choi
Abstract:
Precis: A hybrid deep-learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. Objective: To investigate if a deep learning model could be used to combine nerve fiber layer (NFL) reflectance and other OCT parameters for glaucoma diagnosis. Patients and Methods: This is a prospective observational study where of 106 normal subjects and 164 perimetric glaucoma…
▽ More
Precis: A hybrid deep-learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. Objective: To investigate if a deep learning model could be used to combine nerve fiber layer (NFL) reflectance and other OCT parameters for glaucoma diagnosis. Patients and Methods: This is a prospective observational study where of 106 normal subjects and 164 perimetric glaucoma (PG) patients. Peripapillary NFL reflectance map, NFL thickness map, optic head analysis of disc, and macular ganglion cell complex thickness were obtained using spectral domain OCT. A hybrid deep learning model combined a fully connected network (FCN) and a convolution neural network (CNN) to develop and combine those OCT maps and parameters to distinguish normal and PG eyes. Two deep learning models were compared based on whether the NFL reflectance map was used as part of the input or not. Results: The hybrid deep learning model with reflectance achieved 0.909 sensitivity at 99% specificity and 0.926 at 95%. The overall accuracy was 0.948 with 0.893 sensitivity and 1.000 specificity, and the AROC was 0.979, which is significantly better than the logistic regression models (p < 0.001). The second best model is the hybrid deep learning model w/o reflectance, which also had significantly higher AROC than logistic regression models (p < 0.001). Logistic regression with reflectance model had slightly higher AROC or sensitivity than the other logistic regression model without reflectance (p = 0.024). Conclusions: Hybrid deep learning model significantly improved the diagnostic accuracy, without or without NFL reflectance. Hybrid deep learning model, combining reflectance/NFL thickness/GCC thickness/ONH parameter, may be a practical model for glaucoma screen purposes.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses
Authors:
Sahil Girhepuje,
Siva Sankar Sajeev,
Purvam Jain,
Arya Sikder,
Adithya Rama Varma,
Ryan George,
Akshay Govind Srinivasan,
Mahendra Kurup,
Ashmit Sinha,
Sudip Mondal
Abstract:
Large Language Models (LLMs) currently struggle with tool invocation and chaining, as they often hallucinate or miss essential steps in a sequence. We propose RE-GAINS and EnChAnT, two novel frameworks that empower LLMs to tackle complex user queries by making API calls to external tools based on tool descriptions and argument lists. Tools are chained based on the expected output, without receivin…
▽ More
Large Language Models (LLMs) currently struggle with tool invocation and chaining, as they often hallucinate or miss essential steps in a sequence. We propose RE-GAINS and EnChAnT, two novel frameworks that empower LLMs to tackle complex user queries by making API calls to external tools based on tool descriptions and argument lists. Tools are chained based on the expected output, without receiving the actual results from each individual call. EnChAnT, an open-source solution, leverages an LLM format enforcer, OpenChat 3.5 (an LLM), and ToolBench's API Retriever. RE-GAINS utilizes OpenAI models and embeddings with a specialized prompt based on the $\underline{R}$easoning vi$\underline{a}$ $\underline{P}$lanning $(RAP)$ framework. Both frameworks are low cost (0.01\$ per query). Our key contribution is enabling LLMs for tool invocation and chaining using modifiable, externally described tools.
△ Less
Submitted 20 June, 2024; v1 submitted 28 January, 2024;
originally announced January 2024.
-
CACTUS: a Comprehensive Abstraction and Classification Tool for Uncovering Structures
Authors:
Luca Gherardini,
Varun Ravi Varma,
Karol Capala,
Roger Woods,
Jose Sousa
Abstract:
The availability of large data sets is providing an impetus for driving current artificial intelligent developments. There are, however, challenges for develo** solutions with small data sets due to practical and cost-effective deployment and the opacity of deep learning models. The Comprehensive Abstraction and Classification Tool for Uncovering Structures called CACTUS is presented for improve…
▽ More
The availability of large data sets is providing an impetus for driving current artificial intelligent developments. There are, however, challenges for develo** solutions with small data sets due to practical and cost-effective deployment and the opacity of deep learning models. The Comprehensive Abstraction and Classification Tool for Uncovering Structures called CACTUS is presented for improved secure analytics by effectively employing explainable artificial intelligence. It provides additional support for categorical attributes, preserving their original meaning, optimising memory usage, and speeding up the computation through parallelisation. It shows to the user the frequency of the attributes in each class and ranks them by their discriminative power. Its performance is assessed by application to the Wisconsin diagnostic breast cancer and Thyroid0387 data sets.
△ Less
Submitted 23 August, 2023;
originally announced August 2023.
-
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Authors:
Yanli Zhao,
Andrew Gu,
Rohan Varma,
Liang Luo,
Chien-Chin Huang,
Min Xu,
Less Wright,
Hamid Shojanazeri,
Myle Ott,
Sam Shleifer,
Alban Desmaison,
Can Balioglu,
Pritam Damania,
Bernard Nguyen,
Geeta Chauhan,
Yuchen Hao,
Ajit Mathews,
Shen Li
Abstract:
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains. Despite the remarkable progress made in the field of machine learning systems research, which has enabled the development and exploration of large models, such abilities remain confined to a small group of advanced users and industry leaders, resulting in an implicit tech…
▽ More
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains. Despite the remarkable progress made in the field of machine learning systems research, which has enabled the development and exploration of large models, such abilities remain confined to a small group of advanced users and industry leaders, resulting in an implicit technical barrier for the wider community to access and leverage these technologies. In this paper, we introduce PyTorch Fully Sharded Data Parallel (FSDP) as an industry-grade solution for large model training. FSDP has been closely co-designed with several key PyTorch core components including Tensor implementation, dispatcher system, and CUDA memory caching allocator, to provide non-intrusive user experiences and high training efficiency. Additionally, FSDP natively incorporates a range of techniques and settings to optimize resource utilization across a variety of hardware configurations. The experimental results demonstrate that FSDP is capable of achieving comparable performance to Distributed Data Parallel while providing support for significantly larger models with near-linear scalability in terms of TFLOPS.
△ Less
Submitted 12 September, 2023; v1 submitted 21 April, 2023;
originally announced April 2023.
-
The Third DIHARD Diarization Challenge
Authors:
Neville Ryant,
Prachi Singh,
Venkat Krishnamohan,
Rajat Varma,
Kenneth Church,
Christopher Cieri,
Jun Du,
Sriram Ganapathy,
Mark Liberman
Abstract:
DIHARD III was the third in a series of speaker diarization challenges intended to improve the robustness of diarization systems to variability in recording equipment, noise conditions, and conversational domain. Speaker diarization was evaluated under two speech activity conditions (diarization from a reference speech activity vs. diarization from scratch) and 11 diverse domains. The domains span…
▽ More
DIHARD III was the third in a series of speaker diarization challenges intended to improve the robustness of diarization systems to variability in recording equipment, noise conditions, and conversational domain. Speaker diarization was evaluated under two speech activity conditions (diarization from a reference speech activity vs. diarization from scratch) and 11 diverse domains. The domains span a range of recording conditions and interaction types, including read audio-books, meeting speech, clinical interviews, web videos, and, for the first time, conversational telephone speech. A total of 30 organizations (forming 21teams) from industry and academia submitted 499 valid system outputs. The evaluation results indicate that speaker diarization has improved markedly since DIHARD I, particularly for two-party interactions, but that for many domains (e.g., web video) the problem remains far from solved.
△ Less
Submitted 5 April, 2021; v1 submitted 2 December, 2020;
originally announced December 2020.
-
Post Quantum Secure Command and Control of Mobile Agents : Inserting quantum-resistant encryption schemes in the Secure Robot Operating System
Authors:
Richa Varma,
Chris Melville,
Claudio Pinello,
Tuhin Sahai
Abstract:
The secure command and control (C&C) of mobile agents arises in various settings including unmanned aerial vehicles, single pilot operations in commercial settings, and mobile robots to name a few. As more and more of these applications get integrated into aerospace and defense use cases, the security of the communication channel between the ground station and the mobile agent is of increasing imp…
▽ More
The secure command and control (C&C) of mobile agents arises in various settings including unmanned aerial vehicles, single pilot operations in commercial settings, and mobile robots to name a few. As more and more of these applications get integrated into aerospace and defense use cases, the security of the communication channel between the ground station and the mobile agent is of increasing importance. The development of quantum computing devices poses a unique threat to secure communications due to the vulnerability of asymmetric ciphers to Shor's algorithm. Given the active development of new quantum resistant encryption techniques, we report the first integration of post-quantum secure encryption schemes with the robot operating system (ROS) and C&C of mobile agents, in general. We integrate these schemes in the application and network layers, and study the performance of these methods by comparing them to present day security schemes such as the widely used RSA algorithm.
△ Less
Submitted 16 September, 2020;
originally announced September 2020.
-
PyTorch Distributed: Experiences on Accelerating Data Parallel Training
Authors:
Shen Li,
Yanli Zhao,
Rohan Varma,
Omkar Salpekar,
Pieter Noordhuis,
Teng Li,
Adam Paszke,
Jeff Smith,
Brian Vaughan,
Pritam Damania,
Soumith Chintala
Abstract:
This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module. PyTorch is a widely-adopted scientific computing package used in deep learning research and applications. Recent advances in deep learning argue for the value of large datasets and large models, which necessitates the ability to scale out model training to more computational resources. D…
▽ More
This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module. PyTorch is a widely-adopted scientific computing package used in deep learning research and applications. Recent advances in deep learning argue for the value of large datasets and large models, which necessitates the ability to scale out model training to more computational resources. Data parallelism has emerged as a popular solution for distributed training thanks to its straightforward principle and broad applicability. In general, the technique of distributed data parallelism replicates the model on every computational resource to generate gradients independently and then communicates those gradients at each iteration to keep model replicas consistent. Despite the conceptual simplicity of the technique, the subtle dependencies between computation and communication make it non-trivial to optimize the distributed training efficiency. As of v1.5, PyTorch natively provides several techniques to accelerate distributed data parallel, including bucketing gradients, overlap** computation with communication, and skip** gradient synchronization. Evaluations show that, when configured appropriately, the PyTorch distributed data parallel module attains near-linear scalability using 256 GPUs.
△ Less
Submitted 28 June, 2020;
originally announced June 2020.
-
Vector-Valued Graph Trend Filtering with Non-Convex Penalties
Authors:
Rohan Varma,
Harlin Lee,
Jelena Kovačević,
Yuejie Chi
Abstract:
This work studies the denoising of piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness over a graph, where the value at each node can be vector-valued. We extend the graph trend filtering framework to denoising vector-valued graph signals with a family of non-convex regularizers, which exhibit superior recovery performance over existing convex regularizers. Using an orac…
▽ More
This work studies the denoising of piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness over a graph, where the value at each node can be vector-valued. We extend the graph trend filtering framework to denoising vector-valued graph signals with a family of non-convex regularizers, which exhibit superior recovery performance over existing convex regularizers. Using an oracle inequality, we establish the statistical error rates of first-order stationary points of the proposed non-convex method for generic graphs. Furthermore, we present an ADMM-based algorithm to solve the proposed method and establish its convergence. Numerical experiments are conducted on both synthetic and real-world data for denoising, support recovery, event detection, and semi-supervised classification.
△ Less
Submitted 18 November, 2019; v1 submitted 29 May, 2019;
originally announced May 2019.
-
Water Distribution System Design Using Multi-Objective Genetic Algorithm with External Archive and Local Search
Authors:
Mahesh Patil,
M. Naveen Naidu,
A. Vasan,
Murari R. R. Varma
Abstract:
Hybridisation of the multi-objective optimisation algorithm NSGA-II and local search is proposed for water distribution system design. Results obtained with the proposed algorithm are presented for four medium-size water networks taken from the literature. Local search is found to be beneficial for one of the networks in terms of finding new solutions not reported earlier. It is also shown that si…
▽ More
Hybridisation of the multi-objective optimisation algorithm NSGA-II and local search is proposed for water distribution system design. Results obtained with the proposed algorithm are presented for four medium-size water networks taken from the literature. Local search is found to be beneficial for one of the networks in terms of finding new solutions not reported earlier. It is also shown that simply using an external archive to save all non-dominated solutions visited by the population, even without local search, leads to substantial improvement in the non-dominated set produced by the algorithm.
△ Less
Submitted 20 May, 2019;
originally announced May 2019.
-
Water Distribution System Design Using Multi-Objective Particle Swarm Optimisation
Authors:
Mahesh B. Patil,
M. Naveen Naidu,
A. Vasan,
Murari R. R. Varma
Abstract:
Application of the multi-objective particle swarm optimisation (MOPSO) algorithm to design of water distribution systems is described. An earlier MOPSO algorithm is augmented with (a) local search, (b) a modified strategy for assigning the leader, and (c) a modified mutation scheme. For one of the benchmark problems described in the literature, the effect of each of the above features on the algor…
▽ More
Application of the multi-objective particle swarm optimisation (MOPSO) algorithm to design of water distribution systems is described. An earlier MOPSO algorithm is augmented with (a) local search, (b) a modified strategy for assigning the leader, and (c) a modified mutation scheme. For one of the benchmark problems described in the literature, the effect of each of the above features on the algorithm performance is demonstrated. The augmented MOPSO algorithm (called MOPSO+) is applied to five benchmark problems, and in each case, it finds non-dominated solutions not reported earlier. In addition, for the purpose of comparing Pareto fronts (sets of non-dominated solutions) obtained by different algorithms, a new criterion is suggested, and its usefulness is pointed out with an example. Finally, some suggestions regarding future research directions are made.
△ Less
Submitted 14 March, 2019;
originally announced March 2019.
-
Sampling Theory for Graph Signals on Product Graphs
Authors:
Rohan Varma,
Jelena Kovačević
Abstract:
In this paper, we extend the sampling theory on graphs by constructing a framework that exploits the structure in product graphs for efficient sampling and recovery of bandlimited graph signals that lie on them. Product graphs are graphs that are composed from smaller graph atoms; we motivate how this model is a flexible and useful way to model richer classes of data that can be multi-modal in nat…
▽ More
In this paper, we extend the sampling theory on graphs by constructing a framework that exploits the structure in product graphs for efficient sampling and recovery of bandlimited graph signals that lie on them. Product graphs are graphs that are composed from smaller graph atoms; we motivate how this model is a flexible and useful way to model richer classes of data that can be multi-modal in nature. Previous works have established a sampling theory on graphs for bandlimited signals. Importantly, the framework achieves significant savings in both sample complexity and computational complexity
△ Less
Submitted 26 September, 2018;
originally announced September 2018.
-
Automatic Assessment of Artistic Quality of Photos
Authors:
Ashish Verma,
Kranthi Koukuntla,
Rohit Varma,
Snehasis Mukherjee
Abstract:
This paper proposes a technique to assess the aesthetic quality of photographs. The goal of the study is to predict whether a given photograph is captured by professional photographers, or by common people, based on a measurement of artistic quality of the photograph. We propose a Multi-Layer-Perceptron based system to analyze some low, mid and high level image features and find their effectivenes…
▽ More
This paper proposes a technique to assess the aesthetic quality of photographs. The goal of the study is to predict whether a given photograph is captured by professional photographers, or by common people, based on a measurement of artistic quality of the photograph. We propose a Multi-Layer-Perceptron based system to analyze some low, mid and high level image features and find their effectiveness to measure artistic quality of the image and produce a measurement of the artistic quality of the image on a scale of 10. We validate the proposed system on a large dataset, containing images downloaded from the internet. The dataset contains some images captured by professional photographers and the rest of the images captured by common people. The proposed measurement of artistic quality of images provides higher value of photo quality for the images captured by professional photographers, compared to the values provided for the other images.
△ Less
Submitted 17 April, 2018;
originally announced April 2018.
-
Signal Representations on Graphs: Tools and Applications
Authors:
Siheng Chen,
Rohan Varma,
Aarti Singh,
Jelena Kovačević
Abstract:
We present a framework for representing and modeling data on graphs. Based on this framework, we study three typical classes of graph signals: smooth graph signals, piecewise-constant graph signals, and piecewise-smooth graph signals. For each class, we provide an explicit definition of the graph signals and construct a corresponding graph dictionary with desirable properties. We then study how su…
▽ More
We present a framework for representing and modeling data on graphs. Based on this framework, we study three typical classes of graph signals: smooth graph signals, piecewise-constant graph signals, and piecewise-smooth graph signals. For each class, we provide an explicit definition of the graph signals and construct a corresponding graph dictionary with desirable properties. We then study how such graph dictionary works in two standard tasks: approximation and sampling followed with recovery, both from theoretical as well as algorithmic perspectives. Finally, for each class, we present a case study of a real-world problem by using the proposed methodology.
△ Less
Submitted 16 December, 2015;
originally announced December 2015.
-
Signal Recovery on Graphs: Fundamental Limits of Sampling Strategies
Authors:
Siheng Chen,
Rohan Varma,
Aarti Singh,
Jelena Kovačević
Abstract:
This paper builds theoretical foundations for the recovery of a newly proposed class of smooth graph signals, approximately bandlimited graph signals, under three sampling strategies: uniform sampling, experimentally designed sampling and active sampling. We then state minimax lower bounds on the maximum risk for the approximately bandlimited class under these three sampling strategies and show th…
▽ More
This paper builds theoretical foundations for the recovery of a newly proposed class of smooth graph signals, approximately bandlimited graph signals, under three sampling strategies: uniform sampling, experimentally designed sampling and active sampling. We then state minimax lower bounds on the maximum risk for the approximately bandlimited class under these three sampling strategies and show that active sampling cannot fundamentally outperform experimentally designed sampling. We propose a recovery strategy to compare uniform sampling with experimentally designed sampling. As the proposed recovery strategy lends itself well to statistical analysis, we derive the exact mean square error for each sampling strategy. To study convergence rates, we introduce two types of graphs and find that (1) the proposed recovery strategy achieves the optimal rates; and (2) the experimentally designed sampling fundamentally outperforms uniform sampling for Type-2 class of graphs. To validate our proposed recovery strategy, we test it on five specific graphs: a ring graph with $k$ nearest neighbors, an Erdős-Rényi graph, a random geometric graph, a small-world graph and a power-law graph and find that experimental results match the proposed theory well. This work also presents a comprehensive explanation for when and why sampling for semi-supervised learning with graphs works.
△ Less
Submitted 19 February, 2017; v1 submitted 16 December, 2015;
originally announced December 2015.
-
Signal Recovery on Graphs: Random versus Experimentally Designed Sampling
Authors:
Siheng Chen,
Rohan Varma,
Aarti Singh,
Jelena Kovačević
Abstract:
We study signal recovery on graphs based on two sampling strategies: random sampling and experimentally designed sampling. We propose a new class of smooth graph signals, called approximately bandlimited, which generalizes the bandlimited class and is similar to the globally smooth class. We then propose two recovery strategies based on random sampling and experimentally designed sampling. The pro…
▽ More
We study signal recovery on graphs based on two sampling strategies: random sampling and experimentally designed sampling. We propose a new class of smooth graph signals, called approximately bandlimited, which generalizes the bandlimited class and is similar to the globally smooth class. We then propose two recovery strategies based on random sampling and experimentally designed sampling. The proposed recovery strategy based on experimentally designed sampling is similar to the leverage scores used in the matrix approximation. We show that while both strategies are unbiased estimators for the low-frequency components, the convergence rate of experimentally designed sampling is much faster than that of random sampling when a graph is irregular. We validate the proposed recovery strategies on three specific graphs: a ring graph, an Erdős-Rényi graph, and a star graph. The simulation results support the theoretical analysis.
△ Less
Submitted 29 May, 2015; v1 submitted 21 April, 2015;
originally announced April 2015.
-
Discrete Signal Processing on Graphs: Sampling Theory
Authors:
Siheng Chen,
Rohan Varma,
Aliaksei Sandryhaila,
Jelena Kovačević
Abstract:
We propose a sampling theory for signals that are supported on either directed or undirected graphs. The theory follows the same paradigm as classical sampling theory. We show that perfect recovery is possible for graph signals bandlimited under the graph Fourier transform. The sampled signal coefficients form a new graph signal, whose corresponding graph structure preserves the first-order differ…
▽ More
We propose a sampling theory for signals that are supported on either directed or undirected graphs. The theory follows the same paradigm as classical sampling theory. We show that perfect recovery is possible for graph signals bandlimited under the graph Fourier transform. The sampled signal coefficients form a new graph signal, whose corresponding graph structure preserves the first-order difference of the original graph signal. For general graphs, an optimal sampling operator based on experimentally designed sampling is proposed to guarantee perfect recovery and robustness to noise; for graphs whose graph Fourier transforms are frames with maximal robustness to erasures as well as for Erdős-Rényi graphs, random sampling leads to perfect recovery with high probability. We further establish the connection to the sampling theory of finite discrete-time signal processing and previous work on signal recovery on graphs. To handle full-band graph signals, we propose a graph filter bank based on sampling theory on graphs. Finally, we apply the proposed sampling theory to semi-supervised classification on online blogs and digit images, where we achieve similar or better performance with fewer labeled samples compared to previous work.
△ Less
Submitted 8 August, 2015; v1 submitted 3 March, 2015;
originally announced March 2015.
