-
Sum-of-norms regularized Nonnegative Matrix Factorization
Authors:
Andersen Ang,
Waqas Bin Hamed,
Hans De Sterck
Abstract:
When applying nonnegative matrix factorization (NMF), generally the rank parameter is unknown. Such rank in NMF, called the nonnegative rank, is usually estimated heuristically since computing the exact value of it is NP-hard. In this work, we propose an approximation method to estimate such rank while solving NMF on-the-fly. We use sum-of-norm (SON), a group-lasso structure that encourages pairwi…
▽ More
When applying nonnegative matrix factorization (NMF), generally the rank parameter is unknown. Such rank in NMF, called the nonnegative rank, is usually estimated heuristically since computing the exact value of it is NP-hard. In this work, we propose an approximation method to estimate such rank while solving NMF on-the-fly. We use sum-of-norm (SON), a group-lasso structure that encourages pairwise similarity, to reduce the rank of a factor matrix where the rank is overestimated at the beginning. On various datasets, SON-NMF is able to reveal the correct nonnegative rank of the data without any prior knowledge nor tuning.
SON-NMF is a nonconvx nonsmmoth non-separable non-proximable problem, solving it is nontrivial. First, as rank estimation in NMF is NP-hard, the proposed approach does not enjoy a lower computational complexity. Using a graph-theoretic argument, we prove that the complexity of the SON-NMF is almost irreducible. Second, the per-iteration cost of any algorithm solving SON-NMF is possibly high, which motivated us to propose a first-order BCD algorithm to approximately solve SON-NMF with a low per-iteration cost, in which we do so by the proximal average operator. Lastly, we propose a simple greedy method for post-processing.
SON-NMF exhibits favourable features for applications. Beside the ability to automatically estimate the rank from data, SON-NMF can deal with rank-deficient data matrix, can detect weak component with small energy. Furthermore, on the application of hyperspectral imaging, SON-NMF handle the issue of spectral variability naturally.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
SleepPPG-Net2: Deep learning generalization for sleep staging from photoplethysmography
Authors:
Shirel Attia,
Revital Shani Hershkovich,
Alissa Tabakhov,
Angeleene Ang,
Sharon Haimov,
Riva Tauman,
Joachim A. Behar
Abstract:
Background: Sleep staging is a fundamental component in the diagnosis of sleep disorders and the management of sleep health. Traditionally, this analysis is conducted in clinical settings and involves a time-consuming scoring procedure. Recent data-driven algorithms for sleep staging, using the photoplethysmogram (PPG) time series, have shown high performance on local test sets but lower performan…
▽ More
Background: Sleep staging is a fundamental component in the diagnosis of sleep disorders and the management of sleep health. Traditionally, this analysis is conducted in clinical settings and involves a time-consuming scoring procedure. Recent data-driven algorithms for sleep staging, using the photoplethysmogram (PPG) time series, have shown high performance on local test sets but lower performance on external datasets due to data drift. Methods: This study aimed to develop a generalizable deep learning model for the task of four class (wake, light, deep, and rapid eye movement (REM)) sleep staging from raw PPG physiological time-series. Six sleep datasets, totaling 2,574 patients recordings, were used. In order to create a more generalizable representation, we developed and evaluated a deep learning model called SleepPPG-Net2, which employs a multi-source domain training approach.SleepPPG-Net2 was benchmarked against two state-of-the-art models. Results: SleepPPG-Net2 showed consistently higher performance over benchmark approaches, with generalization performance (Cohen's kappa) improving by up to 19%. Performance disparities were observed in relation to age, sex, and sleep apnea severity. Conclusion: SleepPPG-Net2 sets a new standard for staging sleep from raw PPG time-series.
△ Less
Submitted 10 April, 2024;
originally announced April 2024.
-
You Only Scan Once: A Dynamic Scene Reconstruction Pipeline for 6-DoF Robotic Gras** of Novel Objects
Authors:
Lei Zhou,
Haozhe Wang,
Zhengshen Zhang,
Zhiyang Liu,
Francis EH Tay,
adn Marcelo H. Ang. Jr
Abstract:
In the realm of robotic gras**, achieving accurate and reliable interactions with the environment is a pivotal challenge. Traditional methods of grasp planning methods utilizing partial point clouds derived from depth image often suffer from reduced scene understanding due to occlusion, ultimately impeding their gras** accuracy. Furthermore, scene reconstruction methods have primarily relied u…
▽ More
In the realm of robotic gras**, achieving accurate and reliable interactions with the environment is a pivotal challenge. Traditional methods of grasp planning methods utilizing partial point clouds derived from depth image often suffer from reduced scene understanding due to occlusion, ultimately impeding their gras** accuracy. Furthermore, scene reconstruction methods have primarily relied upon static techniques, which are susceptible to environment change during manipulation process limits their efficacy in real-time gras** tasks. To address these limitations, this paper introduces a novel two-stage pipeline for dynamic scene reconstruction. In the first stage, our approach takes scene scanning as input to register each target object with mesh reconstruction and novel object pose tracking. In the second stage, pose tracking is still performed to provide object poses in real-time, enabling our approach to transform the reconstructed object point clouds back into the scene. Unlike conventional methodologies, which rely on static scene snapshots, our method continuously captures the evolving scene geometry, resulting in a comprehensive and up-to-date point cloud representation. By circumventing the constraints posed by occlusion, our method enhances the overall grasp planning process and empowers state-of-the-art 6-DoF robotic gras** algorithms to exhibit markedly improved accuracy.
△ Less
Submitted 4 April, 2024;
originally announced April 2024.
-
Inhomogeneous graph trend filtering via a l2,0 cardinality penalty
Authors:
Xiaoqing Huang,
Andersen Ang,
Kun Huang,
Jie Zhang,
Yijie Wang
Abstract:
We study estimation of piecewise smooth signals over a graph. We propose a $\ell_{2,0}$-norm penalized Graph Trend Filtering (GTF) model to estimate piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness across the nodes. We prove that the proposed GTF model is simultaneously a k-means clustering on the signal over the nodes and a minimum graph cut on the edges of the graph…
▽ More
We study estimation of piecewise smooth signals over a graph. We propose a $\ell_{2,0}$-norm penalized Graph Trend Filtering (GTF) model to estimate piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness across the nodes. We prove that the proposed GTF model is simultaneously a k-means clustering on the signal over the nodes and a minimum graph cut on the edges of the graph, where the clustering and the cut share the same assignment matrix. We propose two methods to solve the proposed GTF model: a spectral decomposition method and a method based on simulated annealing. In the experiment on synthetic and real-world datasets, we show that the proposed GTF model has a better performances compared with existing approaches on the tasks of denoising, support recovery and semi-supervised classification. We also show that the proposed GTF model can be solved more efficiently than existing models for the dataset with a large edge set.
△ Less
Submitted 4 June, 2024; v1 submitted 11 April, 2023;
originally announced April 2023.
-
Perspectives on AI Architectures and Co-design for Earth System Predictability
Authors:
Maruti K. Mudunuru,
James A. Ang,
Mahantesh Halappanavar,
Simon D. Hammond,
Maya B. Gokhale,
James C. Hoe,
Tushar Krishna,
Sarat S. Sreepathi,
Matthew R. Norman,
Ivy B. Peng,
Philip W. Jones
Abstract:
Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new par…
▽ More
Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, lab, modeling, and analysis activities, called ModEx. The BER's `Model-Experimentation', ModEx, is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process-based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the `AI Architectures and Co-design' session and associated outcomes. The AI Architectures and Co-design session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including: (1) DOE HPC Systems, (2) Cloud HPC Systems, and (3) Edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this co-design area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as: (1) reimagining co-design, (2) data acquisition to distribution, (3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with earth system modeling and simulation, and (4) AI-enabled sensor integration into earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper.
△ Less
Submitted 7 April, 2023;
originally announced April 2023.
-
Fast Projection onto the Capped Simplex with Applications to Sparse Regression in Bioinformatics
Authors:
Andersen Ang,
Jianzhu Ma,
Nianjun Liu,
Kun Huang,
Yijie Wang
Abstract:
We consider the problem of projecting a vector onto the so-called k-capped simplex, which is a hyper-cube cut by a hyperplane. For an n-dimensional input vector with bounded elements, we found that a simple algorithm based on Newton's method is able to solve the projection problem to high precision with a complexity roughly about O(n), which has a much lower computational cost compared with the ex…
▽ More
We consider the problem of projecting a vector onto the so-called k-capped simplex, which is a hyper-cube cut by a hyperplane. For an n-dimensional input vector with bounded elements, we found that a simple algorithm based on Newton's method is able to solve the projection problem to high precision with a complexity roughly about O(n), which has a much lower computational cost compared with the existing sorting-based methods proposed in the literature. We provide a theory for partial explanation and justification of the method.
We demonstrate that the proposed algorithm can produce a solution of the projection problem with high precision on large scale datasets, and the algorithm is able to significantly outperform the state-of-the-art methods in terms of runtime (about 6-8 times faster than a commercial software with respect to CPU time for input vector with 1 million variables or more).
We further illustrate the effectiveness of the proposed algorithm on solving sparse regression in a bioinformatics problem. Empirical results on the GWAS dataset (with 1,500,000 single-nucleotide polymorphisms) show that, when using the proposed method to accelerate the Projected Quasi-Newton (PQN) method, the accelerated PQN algorithm is able to handle huge-scale regression problem and it is more efficient (about 3-6 times faster) than the current state-of-the-art methods.
△ Less
Submitted 25 October, 2021; v1 submitted 16 October, 2021;
originally announced October 2021.
-
An NLG pipeline for a legal expert system: a work in progress
Authors:
Inari Listenmaa,
Jason Morris,
Alfred Ang,
Maryam Hanafiah,
Regina Cheong
Abstract:
We present the NLG component for L4, a prototype domain-specific language (DSL) for drafting laws and contracts. As a concrete use case, we describe a pipeline for a legal expert system created from L4 code. The NLG component is used in two steps. The first step is to create an interview, whose answers are processed into a query for an automated reasoner. The second step is to render the answers o…
▽ More
We present the NLG component for L4, a prototype domain-specific language (DSL) for drafting laws and contracts. As a concrete use case, we describe a pipeline for a legal expert system created from L4 code. The NLG component is used in two steps. The first step is to create an interview, whose answers are processed into a query for an automated reasoner. The second step is to render the answers of the reasoner in natural language.
△ Less
Submitted 6 July, 2021;
originally announced July 2021.
-
Characterizing an Analogical Concept Memory for Architectures Implementing the Common Model of Cognition
Authors:
Shiwali Mohan,
Matt Klenk,
Matthew Shreve,
Kent Evans,
Aaron Ang,
John Maxwell
Abstract:
Architectures that implement the Common Model of Cognition - Soar, ACT-R, and Sigma - have a prominent place in research on cognitive modeling as well as on designing complex intelligent agents. In this paper, we explore how computational models of analogical processing can be brought into these architectures to enable concept acquisition from examples obtained interactively. We propose a new anal…
▽ More
Architectures that implement the Common Model of Cognition - Soar, ACT-R, and Sigma - have a prominent place in research on cognitive modeling as well as on designing complex intelligent agents. In this paper, we explore how computational models of analogical processing can be brought into these architectures to enable concept acquisition from examples obtained interactively. We propose a new analogical concept memory for Soar that augments its current system of declarative long-term memories. We frame the problem of concept learning as embedded within the larger context of interactive task learning (ITL) and embodied language processing (ELP). We demonstrate that the analogical learning methods implemented in the proposed memory can quickly learn a diverse types of novel concepts that are useful not only in recognition of a concept in the environment but also in action selection. Our approach has been instantiated in an implemented cognitive system \textsc{Aileen} and evaluated on a simulated robotic domain.
△ Less
Submitted 29 July, 2020; v1 submitted 2 June, 2020;
originally announced June 2020.
-
Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization
Authors:
Andersen Man Shun Ang,
Jeremy E. Cohen,
Nicolas Gillis,
Le Thi Khanh Hien
Abstract:
This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block-coordinate algorithms for…
▽ More
This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block-coordinate algorithms for dense NTF, in particular for challenging computational scenarios, while requiring a negligible additional computational budget.
△ Less
Submitted 20 November, 2020; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Deep Learning Algorithms to Isolate and Quantify the Structures of the Anterior Segment in Optical Coherence Tomography Images
Authors:
Tan Hung Pham,
Sripad Krishna Devalla,
Aloysius Ang,
Soh Zhi Da,
Alexandre H. Thiery,
Craig Boote,
Ching-Yu Cheng,
Victor Koh,
Michael J. A. Girard
Abstract:
Accurate isolation and quantification of intraocular dimensions in the anterior segment (AS) of the eye using optical coherence tomography (OCT) images is important in the diagnosis and treatment of many eye diseases, especially angle closure glaucoma. In this study, we developed a deep convolutional neural network (DCNN) for the localization of the scleral spur, and the segmentation of anterior s…
▽ More
Accurate isolation and quantification of intraocular dimensions in the anterior segment (AS) of the eye using optical coherence tomography (OCT) images is important in the diagnosis and treatment of many eye diseases, especially angle closure glaucoma. In this study, we developed a deep convolutional neural network (DCNN) for the localization of the scleral spur, and the segmentation of anterior segment structures (iris, corneo-sclera shell, anterior chamber). With limited training data, the DCNN was able to detect the scleral spur on unseen ASOCT images as accurately as an experienced ophthalmologist; and simultaneously isolated the anterior segment structures with a Dice coefficient of 95.7%. We then automatically extracted eight clinically relevant ASOCT parameters and proposed an automated quality check process that asserts the reliability of these parameters. When combined with an OCT machine capable of imaging multiple radial sections, the algorithms can provide a more complete objective assessment. This is an essential step toward providing a robust automated framework for reliable quantification of ASOCT scans, for applications in the diagnosis and management of angle closure glaucoma.
△ Less
Submitted 1 September, 2019;
originally announced September 2019.
-
Deep ensemble learning for Alzheimers disease classification
Authors:
Ning An,
Huitong Ding,
Jiaoyun Yang,
Rhoda Au,
Ting Fang Alvin Ang
Abstract:
Ensemble learning use multiple algorithms to obtain better predictive performance than any single one of its constituent algorithms could. With growing popularity of deep learning, researchers have started to ensemble them for various purposes. Few if any, however, has used the deep learning approach as a means to ensemble algorithms. This paper presents a deep ensemble learning framework which ai…
▽ More
Ensemble learning use multiple algorithms to obtain better predictive performance than any single one of its constituent algorithms could. With growing popularity of deep learning, researchers have started to ensemble them for various purposes. Few if any, however, has used the deep learning approach as a means to ensemble algorithms. This paper presents a deep ensemble learning framework which aims to harness deep learning algorithms to integrate multisource data and tap the wisdom of experts. At the voting layer, a sparse autoencoder is trained for feature learning to reduce the correlation of attributes and diversify the base classifiers ultimately. At the stacking layer, a nonlinear feature-weighted method based on deep belief networks is proposed to rank the base classifiers which may violate the conditional independence. Neural network is used as meta classifier. At the optimizing layer, under-sampling and threshold-moving are used to cope with cost-sensitive problem. Optimized predictions are obtained based on ensemble of probabilistic predictions by similarity calculation. The proposed deep ensemble learning framework is used for Alzheimers disease classification. Experiments with the clinical dataset from national Alzheimers coordinating center demonstrate that the classification accuracy of our proposed framework is 4% better than 6 well-known ensemble approaches as well as the standard stacking algorithm. Adequate coverage of more accurate diagnostic services can be provided by utilizing the wisdom of averaged physicians. This paper points out a new way to boost the primary care of Alzheimers disease from the view of machine learning.
△ Less
Submitted 29 May, 2019;
originally announced May 2019.
