Showing 1–11 of 11 results for author: Kong, N

f-GAN: A frequency-domain-constrained generative adversarial network for PPG to ECG synthesis

Authors: Nathan C. L. Kong, Dae Lee, Huyen Do, Dae Hoon Park, Cong Xu, Hongda Mao, Jonathan Chung

Abstract: Electrocardiograms (ECGs) and photoplethysmograms (PPGs) are generally used to monitor an individual's cardiovascular health. In clinical settings, ECGs and fingertip PPGs are the main signals used for assessing cardiovascular health, but the equipment necessary for their collection precludes their use in daily monitoring. Although PPGs obtained from wrist-worn devices are susceptible to noise due… ▽ More Electrocardiograms (ECGs) and photoplethysmograms (PPGs) are generally used to monitor an individual's cardiovascular health. In clinical settings, ECGs and fingertip PPGs are the main signals used for assessing cardiovascular health, but the equipment necessary for their collection precludes their use in daily monitoring. Although PPGs obtained from wrist-worn devices are susceptible to noise due to motion, they have been widely used to continuously monitor cardiovascular health because of their convenience. Therefore, we would like to combine the ease with which PPGs can be collected with the information that ECGs provide about cardiovascular health by develo** models to synthesize ECG signals from paired PPG signals. We tackled this problem using generative adversarial networks (GANs) and found that models trained using the original GAN formulations can be successfully used to synthesize ECG signals from which heart rate can be extracted using standard signal processing pipelines. Incorporating a frequency-domain constraint to model training improved the stability of model performance and also the performance on heart rate estimation. △ Less

Submitted 15 May, 2024; originally announced June 2024.

Saltation Matrices: The Essential Tool for Linearizing Hybrid Dynamical Systems

Authors: Nathan J. Kong, J. Joe Payne, James Zhu, Aaron M. Johnson

Abstract: Hybrid dynamical systems, i.e. systems that have both continuous and discrete states, are ubiquitous in engineering, but are difficult to work with due to their discontinuous transitions. For example, a robot leg is able to exert very little control effort while it is in the air compared to when it is on the ground. When the leg hits the ground, the penetrating velocity instantaneously collapses t… ▽ More Hybrid dynamical systems, i.e. systems that have both continuous and discrete states, are ubiquitous in engineering, but are difficult to work with due to their discontinuous transitions. For example, a robot leg is able to exert very little control effort while it is in the air compared to when it is on the ground. When the leg hits the ground, the penetrating velocity instantaneously collapses to zero. These instantaneous changes in dynamics and discontinuities (or jumps) in state make standard smooth tools for planning, estimation, control, and learning difficult for hybrid systems. One of the key tools for accounting for these jumps is called the saltation matrix. The saltation matrix is the sensitivity update when a hybrid jump occurs and has been used in a variety of fields including robotics, power circuits, and computational neuroscience. This paper presents an intuitive derivation of the saltation matrix and discusses what it captures, where it has been used in the past, how it is used for linear and quadratic forms, how it is computed for rigid body systems with unilateral constraints, and some of the structural properties of the saltation matrix in these cases. △ Less

Submitted 20 June, 2024; v1 submitted 12 June, 2023; originally announced June 2023.

Hybrid iLQR Model Predictive Control for Contact Implicit Stabilization on Legged Robots

Authors: Nathan J. Kong, Chuanzheng Li, Aaron M. Johnson

Abstract: Model Predictive Control (MPC) is a popular strategy for controlling robots but is difficult for systems with contact due to the complex nature of hybrid dynamics. To implement MPC for systems with contact, dynamic models are often simplified or contact sequences fixed in time in order to plan trajectories efficiently. In this work, we extend Hybrid iterative Linear Quadratic Regulator to work in… ▽ More Model Predictive Control (MPC) is a popular strategy for controlling robots but is difficult for systems with contact due to the complex nature of hybrid dynamics. To implement MPC for systems with contact, dynamic models are often simplified or contact sequences fixed in time in order to plan trajectories efficiently. In this work, we extend Hybrid iterative Linear Quadratic Regulator to work in a MPC fashion (HiLQR MPC) by 1) modifying how the cost function is computed when contact modes do not align, 2) utilizing parallelizations when simulating rigid body dynamics, and 3) using efficient analytical derivative computations of the rigid body dynamics. The result is a system that can modify the contact sequence of the reference behavior and plan whole body motions cohesively -- which is crucial when dealing with large perturbations. HiLQR MPC is tested on two systems: first, the hybrid cost modification is validated on a simple actuated bouncing ball hybrid system. Then HiLQR MPC is compared against methods that utilize centroidal dynamic assumptions on a quadruped robot (Unitree A1). HiLQR MPC outperforms the centroidal methods in both simulation and hardware tests. △ Less

Submitted 6 November, 2023; v1 submitted 10 July, 2022; originally announced July 2022.

Comments: in IEEE Transactions on Robotics, 2023. arXiv admin note: substantial text overlap with arXiv:2103.14584

The Uncertainty Aware Salted Kalman Filter: State Estimation for Hybrid Systems with Uncertain Guards

Authors: J. Joe Payne, Nathan J. Kong, Aaron M. Johnson

Abstract: In this paper we present a method for updating robotic state belief through contact with uncertain surfaces and apply this update to a Kalman filter for more accurate state estimation. Examining how guard surface uncertainty affects the time spent in each mode, we derive a guard saltation matrix - which maps perturbations prior to hybrid events to perturbations after - accounting for additional va… ▽ More In this paper we present a method for updating robotic state belief through contact with uncertain surfaces and apply this update to a Kalman filter for more accurate state estimation. Examining how guard surface uncertainty affects the time spent in each mode, we derive a guard saltation matrix - which maps perturbations prior to hybrid events to perturbations after - accounting for additional variation in the resulting state. Additionally, we propose the use of parameterized reset functions - capturing how unknown parameters change how states are mapped from one mode to the next - the Jacobian of which accounts for the additional uncertainty in the resulting state. The accuracy of these map**s is shown by simulating sampled distributions through uncertain transition events and comparing the resulting covariances. Finally, we integrate these additional terms into the "uncertainty aware Salted Kalman Filter", uaSKF, and show a peak reduction in average estimation error by 24-60% on a variety of test conditions and systems. △ Less

Submitted 29 July, 2022; v1 submitted 25 February, 2022; originally announced February 2022.

Comments: To appear in IROS 2022

Hole-robust Wireframe Detection

Authors: Nae** Kong, Kiwoong Park, Harshith Goka

Abstract: "Wireframe" is a line segment based representation designed to well capture large-scale visual properties of regular, structural shaped man-made scenes surrounding us. Unlike the wireframes, conventional edges or line segments focus on all visible edges and lines without particularly distinguishing which of them are more salient to man-made structural information. Existing wireframe detection mode… ▽ More "Wireframe" is a line segment based representation designed to well capture large-scale visual properties of regular, structural shaped man-made scenes surrounding us. Unlike the wireframes, conventional edges or line segments focus on all visible edges and lines without particularly distinguishing which of them are more salient to man-made structural information. Existing wireframe detection models rely on supervising the annotated data but do not explicitly pay attention to understand how to compose the structural shapes of the scene. In addition, we often face that many foreground objects occluding the background scene interfere with proper inference of the full scene structure behind them. To resolve these problems, we first time in the field, propose new conditional data generation and training that help the model understand how to ignore occlusion indicated by holes, such as foreground object regions masked out on the image. In addition, we first time combine GAN in the model to let the model better predict underlying scene structure even beyond large holes. We also introduce pseudo labeling to further enlarge the model capacity to overcome small-scale labeled data. We show qualitatively and quantitatively that our approach significantly outperforms previous works unable to handle holes, as well as improves ordinary detection without holes given. △ Less

Submitted 29 November, 2021; originally announced November 2021.

Comments: To appear in Proceedings of the 2022 IEEE Winter Conference on Applications of Computer Vision (WACV 2022)

Hybrid Event Sha** to Stabilize Periodic Hybrid Orbits

Authors: James Zhu, Nathan J. Kong, George Council, Aaron M. Johnson

Abstract: Many controllers for legged robotic systems leverage open- or closed-loop control at discrete hybrid events to enhance stability. These controllers appear in several well studied phenomena such as the Raibert step** controller, paddle juggling and swing leg retraction. This work introduces hybrid event sha** (HES): a generalized method for analyzing and producing stable hybrid event controller… ▽ More Many controllers for legged robotic systems leverage open- or closed-loop control at discrete hybrid events to enhance stability. These controllers appear in several well studied phenomena such as the Raibert step** controller, paddle juggling and swing leg retraction. This work introduces hybrid event sha** (HES): a generalized method for analyzing and producing stable hybrid event controllers. HES utilizes the saltation matrix, which gives a closed-form equation for the effect that hybrid events have on stability. We also introduce shape parameters, which are higher order terms that can be tuned completely independently from the system dynamics to promote stability. Optimization methods are used to produce values of these parameters that optimize a stability measure. Hybrid event sha** captures previously developed control methods while also producing new optimally stable trajectories without the need for continuous-domain feedback. △ Less

Submitted 3 July, 2022; v1 submitted 3 October, 2021; originally announced October 2021.

Comments: Presented at IEEE ICRA 2022

Resolution-robust Large Mask Inpainting with Fourier Convolutions

Authors: Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Nae** Kong, Harshith Goka, Kiwoong Park, Victor Lempitsky

Abstract: Modern image inpainting systems, despite the significant progress, often struggle with large missing areas, complex geometric structures, and high-resolution images. We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function. To alleviate this issue, we propose a new method called large mask inpainting (LaMa). LaMa… ▽ More Modern image inpainting systems, despite the significant progress, often struggle with large missing areas, complex geometric structures, and high-resolution images. We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function. To alleviate this issue, we propose a new method called large mask inpainting (LaMa). LaMa is based on i) a new inpainting network architecture that uses fast Fourier convolutions (FFCs), which have the image-wide receptive field; ii) a high receptive field perceptual loss; iii) large training masks, which unlocks the potential of the first two components. Our inpainting network improves the state-of-the-art across a range of datasets and achieves excellent performance even in challenging scenarios, e.g. completion of periodic structures. Our model generalizes surprisingly well to resolutions that are higher than those seen at train time, and achieves this at lower parameter&time costs than the competitive baselines. The code is available at \url{https://github.com/saic-mdal/lama}. △ Less

Submitted 10 November, 2021; v1 submitted 15 September, 2021; originally announced September 2021.

Comments: Winter Conference on Applications of Computer Vision (WACV 2022)

iLQR for Piecewise-Smooth Hybrid Dynamical Systems

Authors: Nathan J. Kong, George Council, Aaron M. Johnson

Abstract: Trajectory optimization is a popular strategy for planning trajectories for robotic systems. However, many robotic tasks require changing contact conditions, which is difficult due to the hybrid nature of the dynamics. The optimal sequence and timing of these modes are typically not known ahead of time. In this work, we extend the Iterative Linear Quadratic Regulator (iLQR) method to a class of pi… ▽ More Trajectory optimization is a popular strategy for planning trajectories for robotic systems. However, many robotic tasks require changing contact conditions, which is difficult due to the hybrid nature of the dynamics. The optimal sequence and timing of these modes are typically not known ahead of time. In this work, we extend the Iterative Linear Quadratic Regulator (iLQR) method to a class of piecewise smooth hybrid dynamical systems by allowing for changing hybrid modes in the forward pass, using the saltation matrix to update the gradient information in the backwards pass, and using a reference extension to account for mode mismatch. We demonstrate these changes on a variety of hybrid systems and compare the different strategies for computing the gradients. △ Less

Submitted 6 September, 2021; v1 submitted 26 March, 2021; originally announced March 2021.

Comments: To Appear in IEEE CDC 2021

A Review of Critical Features and General Issues of Freely Available mHealth Apps For Dietary Assessment

Authors: Ghalib Ahmed Tahir, Chu Kiong Loo, Foong Ming Moy, Nadine Kong

Abstract: Obesity is known to lower the quality of life substantially. It is often associated with increased chances of non-communicable diseases such as diabetes, cardiovascular problems, various cancers, etc. Evidence suggests that diet-related mobile applications play a vital role in assisting individuals in making healthier choices and kee** track of food intake. However, due to an abundance of simila… ▽ More Obesity is known to lower the quality of life substantially. It is often associated with increased chances of non-communicable diseases such as diabetes, cardiovascular problems, various cancers, etc. Evidence suggests that diet-related mobile applications play a vital role in assisting individuals in making healthier choices and kee** track of food intake. However, due to an abundance of similar applications, it becomes pertinent to evaluate each of them in terms of functionality, usability, and possible design issues to truly determine state-of-the-art solutions for the future. Since these applications involve implementing multiple user requirements and recommendations from different dietitians, the evaluation becomes quite complex. Therefore, this study aims to review existing dietary applications at length to highlight key features and problems that enhance or undermine an application's usability. For this purpose, we have examined the published literature from various scientific databases of the PUBMED, CINAHL (January 2010-December 2019) and Science Direct (2010-2019). We followed PRISMA guidelines, and out of our findings, fifty-six primary studies met our inclusion criteria after identification, screening, eligibility and full-text evaluation. We analyzed 35 apps from the selected studies and extracted the data of each of the identified apps.Following our detailed analysis on the comprehensiveness of freely available mHealth applications, we specified potential future research challenges and stated recommendations to help grow clinically accurate diet-related applications. △ Less

Submitted 11 July, 2021; v1 submitted 22 August, 2020; originally announced August 2020.

The Salted Kalman Filter: Kalman Filtering on Hybrid Dynamical Systems

Authors: Nathan J. Kong, J. Joe Payne, George Council, Aaron M. Johnson

Abstract: Many state estimation and control algorithms require knowledge of how probability distributions propagate through dynamical systems. However, despite hybrid dynamical systems becoming increasingly important in many fields, there has been little work on utilizing the knowledge of how probability distributions map through hybrid transitions. Here, we make use of a propagation law that employs the sa… ▽ More Many state estimation and control algorithms require knowledge of how probability distributions propagate through dynamical systems. However, despite hybrid dynamical systems becoming increasingly important in many fields, there has been little work on utilizing the knowledge of how probability distributions map through hybrid transitions. Here, we make use of a propagation law that employs the saltation matrix (a first-order update to the sensitivity equation) to create the Salted Kalman Filter (SKF), a natural extension of the Kalman Filter and Extended Kalman Filter to hybrid dynamical systems. Away from hybrid events, the SKF is a standard Kalman filter. When a hybrid event occurs, the saltation matrix plays an analogous role as that of the system dynamics, subsequently inducing a discrete modification to both the prediction and update steps. The SKF outperforms a naive variational update - the Jacobian of the reset map - by having a reduced mean squared error in state estimation, especially immediately after a hybrid transition event. Compared a hybrid particle filter, the particle filter outperforms the SKF in mean squared error only when a large number of particles are used, likely due to a more accurate accounting of the split distribution near a hybrid transition. △ Less

Submitted 8 February, 2021; v1 submitted 23 July, 2020; originally announced July 2020.

Comments: Submitted to Automatica

Applied Federated Learning: Improving Google Keyboard Query Suggestions

Authors: Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, Françoise Beaufays

Abstract: Federated learning is a distributed form of machine learning where both the training data and model training are decentralized. In this paper, we use federated learning in a commercial, global-scale setting to train, evaluate and deploy a model to improve virtual keyboard search suggestion quality without direct access to the underlying user data. We describe our observations in federated training… ▽ More Federated learning is a distributed form of machine learning where both the training data and model training are decentralized. In this paper, we use federated learning in a commercial, global-scale setting to train, evaluate and deploy a model to improve virtual keyboard search suggestion quality without direct access to the underlying user data. We describe our observations in federated training, compare metrics to live deployments, and present resulting quality increases. In whole, we demonstrate how federated learning can be applied end-to-end to both improve user experiences and enhance user privacy. △ Less

Submitted 6 December, 2018; originally announced December 2018.