Showing 1–27 of 27 results for author: Tian, P

RS-DFM: A Remote Sensing Distributed Foundation Model for Diverse Downstream Tasks

Authors: Zhechao Wang, Peirui Cheng, Pengju Tian, Yuchao Wang, Mingxin Chen, Shu**g Duan, Zhirui Wang, Xinming Li, Xian Sun

Abstract: Remote sensing lightweight foundation models have achieved notable success in online perception within remote sensing. However, their capabilities are restricted to performing online inference solely based on their own observations and models, thus lacking a comprehensive understanding of large-scale remote sensing scenarios. To overcome this limitation, we propose a Remote Sensing Distributed Fou… ▽ More Remote sensing lightweight foundation models have achieved notable success in online perception within remote sensing. However, their capabilities are restricted to performing online inference solely based on their own observations and models, thus lacking a comprehensive understanding of large-scale remote sensing scenarios. To overcome this limitation, we propose a Remote Sensing Distributed Foundation Model (RS-DFM) based on generalized information map** and interaction. This model can realize online collaborative perception across multiple platforms and various downstream tasks by map** observations into a unified space and implementing a task-agnostic information interaction strategy. Specifically, we leverage the ground-based geometric prior of remote sensing oblique observations to transform the feature map** from absolute depth estimation to relative depth estimation, thereby enhancing the model's ability to extract generalized features across diverse heights and perspectives. Additionally, we present a dual-branch information compression module to decouple high-frequency and low-frequency feature information, achieving feature-level compression while preserving essential task-agnostic details. In support of our research, we create a multi-task simulation dataset named AirCo-MultiTasks for multi-UAV collaborative observation. We also conduct extensive experiments, including 3D object detection, instance segmentation, and trajectory prediction. The numerous results demonstrate that our RS-DFM achieves state-of-the-art performance across various downstream tasks. △ Less

Submitted 11 June, 2024; originally announced June 2024.

UCDNet: Multi-UAV Collaborative 3D Object Detection Network by Reliable Feature Map**

Authors: Pengju Tian, Peirui Cheng, Yuchao Wang, Zhechao Wang, Zhirui Wang, Menglong Yan, Xue Yang, Xian Sun

Abstract: Multi-UAV collaborative 3D object detection can perceive and comprehend complex environments by integrating complementary information, with applications encompassing traffic monitoring, delivery services and agricultural management. However, the extremely broad observations in aerial remote sensing and significant perspective differences across multiple UAVs make it challenging to achieve precise… ▽ More Multi-UAV collaborative 3D object detection can perceive and comprehend complex environments by integrating complementary information, with applications encompassing traffic monitoring, delivery services and agricultural management. However, the extremely broad observations in aerial remote sensing and significant perspective differences across multiple UAVs make it challenging to achieve precise and consistent feature map** from 2D images to 3D space in multi-UAV collaborative 3D object detection paradigm. To address the problem, we propose an unparalleled camera-based multi-UAV collaborative 3D object detection paradigm called UCDNet. Specifically, the depth information from the UAVs to the ground is explicitly utilized as a strong prior to provide a reference for more accurate and generalizable feature map**. Additionally, we design a homologous points geometric consistency loss as an auxiliary self-supervision, which directly influences the feature map** module, thereby strengthening the global consistency of multi-view perception. Experiments on AeroCollab3D and CoPerception-UAVs datasets show our method increases 4.7% and 10% mAP respectively compared to the baseline, which demonstrates the superiority of UCDNet. △ Less

Submitted 7 June, 2024; originally announced June 2024.

UVCPNet: A UAV-Vehicle Collaborative Perception Network for 3D Object Detection

Authors: Yuchao Wang, Peirui Cheng, Pengju Tian, Ziyang Yuan, Liang** Zhao, **g Tian, Wensheng Wang, Zhirui Wang, Xian Sun

Abstract: With the advancement of collaborative perception, the role of aerial-ground collaborative perception, a crucial component, is becoming increasingly important. The demand for collaborative perception across different perspectives to construct more comprehensive perceptual information is growing. However, challenges arise due to the disparities in the field of view between cross-domain agents and th… ▽ More With the advancement of collaborative perception, the role of aerial-ground collaborative perception, a crucial component, is becoming increasingly important. The demand for collaborative perception across different perspectives to construct more comprehensive perceptual information is growing. However, challenges arise due to the disparities in the field of view between cross-domain agents and their varying sensitivity to information in images. Additionally, when we transform image features into Bird's Eye View (BEV) features for collaboration, we need accurate depth information. To address these issues, we propose a framework specifically designed for aerial-ground collaboration. First, to mitigate the lack of datasets for aerial-ground collaboration, we develop a virtual dataset named V2U-COO for our research. Second, we design a Cross-Domain Cross-Adaptation (CDCA) module to align the target information obtained from different domains, thereby achieving more accurate perception results. Finally, we introduce a Collaborative Depth Optimization (CDO) module to obtain more precise depth estimation results, leading to more accurate perception outcomes. We conduct extensive experiments on both our virtual dataset and a public dataset to validate the effectiveness of our framework. Our experiments on the V2U-COO dataset and the DAIR-V2X dataset demonstrate that our method improves detection accuracy by 6.1% and 2.7%, respectively. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyond

Authors: Zhechao Wang, Peirui Cheng, Mingxin Chen, Pengju Tian, Zhirui Wang, Xinming Li, Xue Yang, Xian Sun

Abstract: Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encounters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction r… ▽ More Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encounters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction requirements within the constrained drone-based communication bandwidth. To address these problems, we propose a novel framework named "Drones Help Drones" (DHD). Firstly, we incorporate the ground priors provided by the drone's inclined observation to estimate the distance between objects and drones, leading to more precise BEV generation. Secondly, we design a selective mechanism based on the local feature discrepancy to prioritize the critical information contributing to prediction tasks during inter-drone interactions. Additionally, we create the first dataset for multi-drone collaborative prediction, named "Air-Co-Pred", and conduct quantitative and qualitative experiments to validate the effectiveness of our DHD framework.The results demonstrate that compared to state-of-the-art approaches, DHD reduces position deviation in BEV representations by over 20% and requires only a quarter of the transmission ratio for interactions while achieving comparable prediction performance. Moreover, DHD also shows promising generalization to the collaborative 3D object detection in CoPerception-UAVs. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models

Authors: Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying Zhang, Yu Wang, Shizhu Liu, **chuan Tian, Yu** Wang, Yuxuan Wang

Abstract: Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP). Although convenient for research and practical applications, open-source LLMs with fewer parameters often suffer from severe hallucinations compared to their larger counterparts. This paper focuses on measuring and reducing hallucinations in BLOOM 7B, a representative of such weaker open-source LLMs that are public… ▽ More Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP). Although convenient for research and practical applications, open-source LLMs with fewer parameters often suffer from severe hallucinations compared to their larger counterparts. This paper focuses on measuring and reducing hallucinations in BLOOM 7B, a representative of such weaker open-source LLMs that are publicly available for research and commercial applications. We introduce HaloCheck, a lightweight BlackBox knowledge-free framework designed to quantify the severity of hallucinations in LLMs. Additionally, we explore techniques like knowledge injection and teacher-student approaches to alleviate hallucinations in low-parameter LLMs. Our experiments effectively demonstrate the reduction of hallucinations in challenging domains for these LLMs. △ Less

Submitted 13 September, 2023; v1 submitted 22 August, 2023; originally announced August 2023.

Understanding Forward Process of Convolutional Neural Network

Authors: Peixin Tian

Abstract: This paper reveal the selective rotation in the CNNs' forward processing. It elucidates the activation function as a discerning mechanism that unifies and quantizes the rotational aspects of the input data. Experiments show how this defined methodology reflects the progress network distinguish inputs based on statistical indicators, which can be comprehended or analyzed by applying structured math… ▽ More This paper reveal the selective rotation in the CNNs' forward processing. It elucidates the activation function as a discerning mechanism that unifies and quantizes the rotational aspects of the input data. Experiments show how this defined methodology reflects the progress network distinguish inputs based on statistical indicators, which can be comprehended or analyzed by applying structured mathematical tools. Our findings also unveil the consistency between artificial neural networks and the human brain in their data processing pattern. △ Less

Submitted 1 December, 2023; v1 submitted 26 July, 2023; originally announced July 2023.

Comments: something wrong in this paper

Flatness-Aware Minimization for Domain Generalization

Authors: Xingxuan Zhang, Renzhe Xu, Han Yu, Yancheng Dong, Pengfei Tian, Peng Cu

Abstract: Domain generalization (DG) seeks to learn robust models that generalize well under unknown distribution shifts. As a critical aspect of DG, optimizer selection has not been explored in depth. Currently, most DG methods follow the widely used benchmark, DomainBed, and utilize Adam as the default optimizer for all datasets. However, we reveal that Adam is not necessarily the optimal choice for the m… ▽ More Domain generalization (DG) seeks to learn robust models that generalize well under unknown distribution shifts. As a critical aspect of DG, optimizer selection has not been explored in depth. Currently, most DG methods follow the widely used benchmark, DomainBed, and utilize Adam as the default optimizer for all datasets. However, we reveal that Adam is not necessarily the optimal choice for the majority of current DG methods and datasets. Based on the perspective of loss landscape flatness, we propose a novel approach, Flatness-Aware Minimization for Domain Generalization (FAD), which can efficiently optimize both zeroth-order and first-order flatness simultaneously for DG. We provide theoretical analyses of the FAD's out-of-distribution (OOD) generalization error and convergence. Our experimental results demonstrate the superiority of FAD on various DG datasets. Additionally, we confirm that FAD is capable of discovering flatter optima in comparison to other zeroth-order and first-order flatness-aware optimization methods. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: Accepted by ICCV2023

Journal ref: ICCV2023

On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation

Authors: Matias D. Cattaneo, Jason M. Klusowski, Peter M. Tian

Abstract: Decision tree learning is increasingly being used for pointwise inference. Important applications include causal heterogenous treatment effects and dynamic policy decisions, as well as conditional quantile regression and design of experiments, where tree estimation and inference is conducted at specific values of the covariates. In this paper, we call into question the use of decision trees (train… ▽ More Decision tree learning is increasingly being used for pointwise inference. Important applications include causal heterogenous treatment effects and dynamic policy decisions, as well as conditional quantile regression and design of experiments, where tree estimation and inference is conducted at specific values of the covariates. In this paper, we call into question the use of decision trees (trained by adaptive recursive partitioning) for such purposes by demonstrating that they can fail to achieve polynomial rates of convergence in uniform norm with non-vanishing probability, even with pruning. Instead, the convergence may be arbitrarily slow or, in some important special cases, such as honest regression trees, fail completely. We show that random forests can remedy the situation, turning poor performing trees into nearly optimal procedures, at the cost of losing interpretability and introducing two additional tuning parameters. The two hallmarks of random forests, subsampling and the random feature selection mechanism, are seen to each distinctively contribute to achieving nearly optimal performance for the model class considered. △ Less

Submitted 6 February, 2024; v1 submitted 19 November, 2022; originally announced November 2022.

Tackling Data Scarcity with Transfer Learning: A Case Study of Thickness Characterization from Optical Spectra of Perovskite Thin Films

Authors: Siyu Isaac Parker Tian, Zekun Ren, Selvaraj Venkataraj, Yuanhang Cheng, Daniil Bash, Felipe Oviedo, J. Senthilnath, Vijila Chellappan, Yee-Fun Lim, Armin G. Aberle, Benjamin P MacLeod, Fraser G. L. Parlane, Curtis P. Berlinguette, Qianxiao Li, Tonio Buonassisi, Zhe Liu

Abstract: Transfer learning increasingly becomes an important tool in handling data scarcity often encountered in machine learning. In the application of high-throughput thickness as a downstream process of the high-throughput optimization of optoelectronic thin films with autonomous workflows, data scarcity occurs especially for new materials. To achieve high-throughput thickness characterization, we propo… ▽ More Transfer learning increasingly becomes an important tool in handling data scarcity often encountered in machine learning. In the application of high-throughput thickness as a downstream process of the high-throughput optimization of optoelectronic thin films with autonomous workflows, data scarcity occurs especially for new materials. To achieve high-throughput thickness characterization, we propose a machine learning model called thicknessML that predicts thickness from UV-Vis spectrophotometry input and an overarching transfer learning workflow. We demonstrate the transfer learning workflow from generic source domain of generic band-gapped materials to specific target domain of perovskite materials, where the target domain data only come from limited number (18) of refractive indices from literature. The target domain can be easily extended to other material classes with a few literature data. Defining thickness prediction accuracy to be within-10% deviation, thicknessML achieves 92.2% (with a deviation of 3.6%) accuracy with transfer learning compared to 81.8% (with a deviation of 3.6%) 11.7% without (lower mean and larger standard deviation). Experimental validation on six deposited perovskite films also corroborates the efficacy of the proposed workflow by yielding a 10.5% mean absolute percentage error (MAPE). △ Less

Submitted 20 December, 2022; v1 submitted 14 June, 2022; originally announced July 2022.

Reducing the LQG Cost with Minimal Communication

Authors: Oron Sabag, Peida Tian, Victoria Kostina, Babak Hassibi

Abstract: We study the linear quadratic Gaussian (LQG) control problem, in which the controller's observation of the system state is such that a desired cost is unattainable. To achieve the desired LQG cost, we introduce a communication link from the observer (encoder) to the controller. We investigate the optimal trade-off between the improved LQG cost and the consumed communication (information) resources… ▽ More We study the linear quadratic Gaussian (LQG) control problem, in which the controller's observation of the system state is such that a desired cost is unattainable. To achieve the desired LQG cost, we introduce a communication link from the observer (encoder) to the controller. We investigate the optimal trade-off between the improved LQG cost and the consumed communication (information) resources, measured with the conditional directed information, across all encoding-decoding policies. The main result is a semidefinite programming formulation for that optimization problem in the finite-horizon scenario, which applies to time-varying linear dynamical systems. This result extends a seminal work by Tanaka et al., where the only information the controller knows about the system state arrives via a communication channel, to the scenario where the controller has also access to a noisy observation of the system state. As part of our derivation to show the optimiality of an encoder that transmits a memoryless Gaussian measurement of the state, we show that the presence of the controller's observations at the encoder can not reduce the minimal directed information. For time-invariant systems, where the optimal policy may be time-varying, we show in the infinite-horizon scenario that the optimal policy is time-invariant and can be computed explicitly from a solution of a finite-dimensional semidefinite programming. The results are demonstrated via examples that show that even low-quality measurements can have a significant impact on the required communication resources. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Comments: A short version published in https://ieeexplore.ieee.org/document/9304490

LibFewShot: A Comprehensive Library for Few-shot Learning

Authors: Wenbin Li, Ziyi, Wang, Xuesong Yang, Chuanqi Dong, Pinzhuo Tian, Tiexin Qin, **g Huo, Yinghuan Shi, Lei Wang, Yang Gao, Jiebo Luo

Abstract: Few-shot learning, especially few-shot image classification, has received increasing attention and witnessed significant advances in recent years. Some recent studies implicitly show that many generic techniques or ``tricks'', such as data augmentation, pre-training, knowledge distillation, and self-supervision, may greatly boost the performance of a few-shot learning method. Moreover, different w… ▽ More Few-shot learning, especially few-shot image classification, has received increasing attention and witnessed significant advances in recent years. Some recent studies implicitly show that many generic techniques or ``tricks'', such as data augmentation, pre-training, knowledge distillation, and self-supervision, may greatly boost the performance of a few-shot learning method. Moreover, different works may employ different software platforms, backbone architectures and input image sizes, making fair comparisons difficult and practitioners struggle with reproducibility. To address these situations, we propose a comprehensive library for few-shot learning (LibFewShot) by re-implementing eighteen state-of-the-art few-shot learning methods in a unified framework with the same single codebase in PyTorch. Furthermore, based on LibFewShot, we provide comprehensive evaluations on multiple benchmarks with various backbone architectures to evaluate common pitfalls and effects of different training tricks. In addition, with respect to the recent doubts on the necessity of meta- or episodic-training mechanism, our evaluation results confirm that such a mechanism is still necessary especially when combined with pre-training. We hope our work can not only lower the barriers for beginners to enter the area of few-shot learning but also elucidate the effects of nontrivial tricks to facilitate intrinsic research on few-shot learning. The source code is available from https://github.com/RL-VIG/LibFewShot. △ Less

Submitted 15 September, 2022; v1 submitted 10 September, 2021; originally announced September 2021.

Comments: 17 pages

Improving the Generalization of Meta-learning on Unseen Domains via Adversarial Shift

Authors: Pinzhuo Tian, Yao Gao

Abstract: Meta-learning provides a promising way for learning to efficiently learn and achieves great success in many applications. However, most meta-learning literature focuses on dealing with tasks from a same domain, making it brittle to generalize to tasks from the other unseen domains. In this work, we address this problem by simulating tasks from the other unseen domains to improve the generalization… ▽ More Meta-learning provides a promising way for learning to efficiently learn and achieves great success in many applications. However, most meta-learning literature focuses on dealing with tasks from a same domain, making it brittle to generalize to tasks from the other unseen domains. In this work, we address this problem by simulating tasks from the other unseen domains to improve the generalization and robustness of meta-learning method. Specifically, we propose a model-agnostic shift layer to learn how to simulate the domain shift and generate pseudo tasks, and develop a new adversarial learning-to-learn mechanism to train it. Based on the pseudo tasks, the meta-learning model can learn cross-domain meta-knowledge, which can generalize well on unseen domains. We conduct extensive experiments under the domain generalization setting. Experimental results demonstrate that the proposed shift layer is applicable to various meta-learning frameworks. Moreover, our method also leads to state-of-the-art performance on different cross-domain few-shot classification benchmarks and produces good results on cross-domain few-shot regression. △ Less

Submitted 23 July, 2021; originally announced July 2021.

Comments: v1

Time Series Anomaly Detection for Cyber-Physical Systems via Neural System Identification and Bayesian Filtering

Authors: Cheng Feng, Pengwei Tian

Abstract: Recent advances in AIoT technologies have led to an increasing popularity of utilizing machine learning algorithms to detect operational failures for cyber-physical systems (CPS). In its basic form, an anomaly detection module monitors the sensor measurements and actuator states from the physical plant, and detects anomalies in these measurements to identify abnormal operation status. Nevertheless… ▽ More Recent advances in AIoT technologies have led to an increasing popularity of utilizing machine learning algorithms to detect operational failures for cyber-physical systems (CPS). In its basic form, an anomaly detection module monitors the sensor measurements and actuator states from the physical plant, and detects anomalies in these measurements to identify abnormal operation status. Nevertheless, building effective anomaly detection models for CPS is rather challenging as the model has to accurately detect anomalies in presence of highly complicated system dynamics and unknown amount of sensor noise. In this work, we propose a novel time series anomaly detection method called Neural System Identification and Bayesian Filtering (NSIBF) in which a specially crafted neural network architecture is posed for system identification, i.e., capturing the dynamics of CPS in a dynamical state-space model; then a Bayesian filtering algorithm is naturally applied on top of the "identified" state-space model for robust anomaly detection by tracking the uncertainty of the hidden state of the system recursively over time. We provide qualitative as well as quantitative experiments with the proposed method on a synthetic and three real-world CPS datasets, showing that NSIBF compares favorably to the state-of-the-art methods with considerable improvements on anomaly detection in CPS. △ Less

Submitted 6 January, 2022; v1 submitted 15 June, 2021; originally announced June 2021.

Comments: Appeared in KDD 2021

Journal ref: KDD 2021

On the Sample Complexity of Rank Regression from Pairwise Comparisons

Authors: Berkan Kadioglu, Peng Tian, Jennifer Dy, Deniz Erdogmus, Stratis Ioannidis

Abstract: We consider a rank regression setting, in which a dataset of $N$ samples with features in $\mathbb{R}^d$ is ranked by an oracle via $M$ pairwise comparisons. Specifically, there exists a latent total ordering of the samples; when presented with a pair of samples, a noisy oracle identifies the one ranked higher with respect to the underlying total ordering. A learner observes a dataset of such comp… ▽ More We consider a rank regression setting, in which a dataset of $N$ samples with features in $\mathbb{R}^d$ is ranked by an oracle via $M$ pairwise comparisons. Specifically, there exists a latent total ordering of the samples; when presented with a pair of samples, a noisy oracle identifies the one ranked higher with respect to the underlying total ordering. A learner observes a dataset of such comparisons and wishes to regress sample ranks from their features. We show that to learn the model parameters with $ε> 0$ accuracy, it suffices to conduct $M \in Ω(dN\log^3 N/ε^2)$ comparisons uniformly at random when $N$ is $Ω(d/ε^2)$. △ Less

Submitted 4 May, 2021; originally announced May 2021.

Large Scale Prediction with Decision Trees

Authors: Jason M. Klusowski, Peter M. Tian

Abstract: This paper shows that decision trees constructed with Classification and Regression Trees (CART) and C4.5 methodology are consistent for regression and classification tasks, even when the number of predictor variables grows sub-exponentially with the sample size, under natural 0-norm and 1-norm sparsity constraints. The theory applies to a wide range of models, including (ordinary or logistic) add… ▽ More This paper shows that decision trees constructed with Classification and Regression Trees (CART) and C4.5 methodology are consistent for regression and classification tasks, even when the number of predictor variables grows sub-exponentially with the sample size, under natural 0-norm and 1-norm sparsity constraints. The theory applies to a wide range of models, including (ordinary or logistic) additive regression models with component functions that are continuous, of bounded variation, or, more generally, Borel measurable. Consistency holds for arbitrary joint distributions of the predictor variables, thereby accommodating continuous, discrete, and/or dependent data. Finally, we show that these qualitative properties of individual trees are inherited by Breiman's random forests. A key step in the analysis is the establishment of an oracle inequality, which allows for a precise characterization of the goodness-of-fit and complexity tradeoff for a mis-specified model. △ Less

Submitted 13 November, 2023; v1 submitted 28 April, 2021; originally announced April 2021.

Nonparametric Variable Screening with Optimal Decision Stumps

Authors: Jason M. Klusowski, Peter M. Tian

Abstract: Decision trees and their ensembles are endowed with a rich set of diagnostic tools for ranking and screening variables in a predictive model. Despite the widespread use of tree based variable importance measures, pinning down their theoretical properties has been challenging and therefore largely unexplored. To address this gap between theory and practice, we derive finite sample performance guara… ▽ More Decision trees and their ensembles are endowed with a rich set of diagnostic tools for ranking and screening variables in a predictive model. Despite the widespread use of tree based variable importance measures, pinning down their theoretical properties has been challenging and therefore largely unexplored. To address this gap between theory and practice, we derive finite sample performance guarantees for variable selection in nonparametric models using a single-level CART decision tree (a decision stump). Under standard operating assumptions in variable screening literature, we find that the marginal signal strength of each variable and ambient dimensionality can be considerably weaker and higher, respectively, than state-of-the-art nonparametric variable selection methods. Furthermore, unlike previous marginal screening methods that attempt to directly estimate each marginal projection via a truncated basis expansion, the fitted model used here is a simple, parsimonious decision stump, thereby eliminating the need for tuning the number of basis terms. Thus, surprisingly, even though decision stumps are highly inaccurate for estimation purposes, they can still be used to perform consistent model selection. △ Less

Submitted 10 December, 2020; v1 submitted 5 November, 2020; originally announced November 2020.

An invertible crystallographic representation for general inverse design of inorganic crystals with targeted properties

Authors: Zekun Ren, Siyu Isaac Parker Tian, Juhwan Noh, Felipe Oviedo, Guangzong Xing, Jiali Li, Qiaohao Liang, Ruiming Zhu, Armin G. Aberle, Shi**g Sun, Xiaonan Wang, Yi Liu, Qianxiao Li, Senthilnath Jayavelu, Kedar Hippalgaonkar, Yousung Jung, Tonio Buonassisi

Abstract: Realizing general inverse design could greatly accelerate the discovery of new materials with user-defined properties. However, state-of-the-art generative models tend to be limited to a specific composition or crystal structure. Herein, we present a framework capable of general inverse design (not limited to a given set of elements or crystal structures), featuring a generalized invertible repres… ▽ More Realizing general inverse design could greatly accelerate the discovery of new materials with user-defined properties. However, state-of-the-art generative models tend to be limited to a specific composition or crystal structure. Herein, we present a framework capable of general inverse design (not limited to a given set of elements or crystal structures), featuring a generalized invertible representation that encodes crystals in both real and reciprocal space, and a property-structured latent space from a variational autoencoder (VAE). In three design cases, the framework generates 142 new crystals with user-defined formation energies, bandgap, thermoelectric (TE) power factor, and combinations thereof. These generated crystals, absent in the training database, are validated by first-principles calculations. The success rates (number of first-principles-validated target-satisfying crystals/number of designed crystals) ranges between 7.1% and 38.9%. These results represent a significant step toward property-driven general inverse design using generative models, although practical challenges remain when coupled with experimental synthesis. △ Less

Submitted 15 December, 2021; v1 submitted 15 May, 2020; originally announced May 2020.

RelSen: An Optimization-based Framework for Simultaneously Sensor Reliability Monitoring and Data Cleaning

Authors: Cheng Feng, Xiao Liang, Daniel Schneegass, PengWei Tian

Abstract: Recent advances in the Internet of Things (IoT) technology have led to a surge on the popularity of sensing applications. As a result, people increasingly rely on information obtained from sensors to make decisions in their daily life. Unfortunately, in most sensing applications, sensors are known to be error-prone and their measurements can become misleading at any unexpected time. Therefore, in… ▽ More Recent advances in the Internet of Things (IoT) technology have led to a surge on the popularity of sensing applications. As a result, people increasingly rely on information obtained from sensors to make decisions in their daily life. Unfortunately, in most sensing applications, sensors are known to be error-prone and their measurements can become misleading at any unexpected time. Therefore, in order to enhance the reliability of sensing applications, apart from the physical phenomena/processes of interest, we believe it is also highly important to monitor the reliability of sensors and clean the sensor data before analysis on them being conducted. Existing studies often regard sensor reliability monitoring and sensor data cleaning as separate problems. In this work, we propose RelSen, a novel optimization-based framework to address the two problems simultaneously via utilizing the mutual dependence between them. Furthermore, RelSen is not application-specific as its implementation assumes a minimal prior knowledge of the process dynamics under monitoring. This significantly improves its generality and applicability in practice. In our experiments, we apply RelSen on an outdoor air pollution monitoring system and a condition monitoring system for a cement rotary kiln. Experimental results show that our framework can timely identify unreliable sensors and remove sensor measurement errors caused by three types of most commonly observed sensor faults. △ Less

Submitted 6 August, 2020; v1 submitted 18 April, 2020; originally announced April 2020.

Comments: accepted in CIKM 2020

Differentiable Meta-learning Model for Few-shot Semantic Segmentation

Authors: Pinzhuo Tian, Zhangkai Wu, Lei Qi, Lei Wang, Yinghuan Shi, Yang Gao

Abstract: To address the annotation scarcity issue in some cases of semantic segmentation, there have been a few attempts to develop the segmentation model in the few-shot learning paradigm. However, most existing methods only focus on the traditional 1-way segmentation setting (i.e., one image only contains a single object). This is far away from practical semantic segmentation tasks where the K-way settin… ▽ More To address the annotation scarcity issue in some cases of semantic segmentation, there have been a few attempts to develop the segmentation model in the few-shot learning paradigm. However, most existing methods only focus on the traditional 1-way segmentation setting (i.e., one image only contains a single object). This is far away from practical semantic segmentation tasks where the K-way setting (K>1) is usually required by performing the accurate multi-object segmentation. To deal with this issue, we formulate the few-shot semantic segmentation task as a learning-based pixel classification problem and propose a novel framework called MetaSegNet based on meta-learning. In MetaSegNet, an architecture of embedding module consisting of the global and local feature branches is developed to extract the appropriate meta-knowledge for the few-shot segmentation. Moreover, we incorporate a linear model into MetaSegNet as a base learner to directly predict the label of each pixel for the multi-object segmentation. Furthermore, our MetaSegNet can be trained by the episodic training mechanism in an end-to-end manner from scratch. Experiments on two popular semantic segmentation datasets, i.e., PASCAL VOC and COCO, reveal the effectiveness of the proposed MetaSegNet in the K-way few-shot semantic segmentation task. △ Less

Submitted 23 November, 2019; originally announced November 2019.

Comments: Accepted by AAAI2020

Formations and generalized Davenport-Schinzel sequences

Authors: Jesse Geneson, Peter Tian, Katherine Tung

Abstract: Let $up(r, t) = (a_1 a_2 \dots a_r)^t$. We investigate the problem of determining the maximum possible integer $n(r, t)$ for which there exist $2t-1$ permutations $π_1, π_2, \dots, π_{2t-1}$ of $1, 2, \dots, n(r, t)$ such that the concatenated sequence $π_1 π_2 \dots π_{2t-1}$ has no subsequence isomorphic to $up(r,t)$. This quantity has been used to obtain an upper bound on the maximum number of… ▽ More Let $up(r, t) = (a_1 a_2 \dots a_r)^t$. We investigate the problem of determining the maximum possible integer $n(r, t)$ for which there exist $2t-1$ permutations $π_1, π_2, \dots, π_{2t-1}$ of $1, 2, \dots, n(r, t)$ such that the concatenated sequence $π_1 π_2 \dots π_{2t-1}$ has no subsequence isomorphic to $up(r,t)$. This quantity has been used to obtain an upper bound on the maximum number of edges in $k$-quasiplanar graphs. It was proved by (Geneson, Prasad, and Tidor, Electronic Journal of Combinatorics, 2014) that $n(r, t) \le (r-1)^{2^{2t-2}}$. We prove that $n(r,t) = Θ(r^{2t-1 \choose t})$, where the constant in the bound depends only on $t$. Using our upper bound in the case $t = 2$, we also sharpen an upper bound of (Klazar, Integers, 2002), who proved that $Ex(up(r,2),n) < (2n+1)L$ where $L = Ex(up(r,2),K-1)+1$, $K = (r-1)^4 + 1$, and $Ex(u, n)$ denotes the extremal function for forbidden generalized Davenport-Schinzel sequences. We prove that $K = (r-1)^4 + 1$ in Klazar's bound can be replaced with $K = (r-1) \binom{r}{2}+1$. We also prove a conjecture from (Geneson, Prasad, and Tidor, Electronic Journal of Combinatorics, 2014) by showing for $t \geq 1$ that $Ex(a b c (a c b)^{t} a b c, n) = n 2^{\frac{1}{t!}α(n)^{t} \pm O(α(n)^{t-1})}$. In addition, we prove that $Ex(a b c a c b (a b c)^{t} a c b, n) = n 2^{\frac{1}{(t+1)!}α(n)^{t+1} \pm O(α(n)^{t})}$ for all $t \geq 1$. △ Less

Submitted 13 September, 2021; v1 submitted 19 September, 2019; originally announced September 2019.

MSC Class: 05D99

A Search for the Underlying Equation Governing Similar Systems

Authors: Changwei Loh, Daniel Schneegass, Pengwei Tian

Abstract: We show a data-driven approach to discover the underlying structural form of the mathematical equation governing the dynamics of multiple but similar systems induced by the same mechanisms. This approach hinges on theories that we lay out involving arguments based on the nature of physical systems. In the same vein, we also introduce a metric to search for the best candidate equation using the dat… ▽ More We show a data-driven approach to discover the underlying structural form of the mathematical equation governing the dynamics of multiple but similar systems induced by the same mechanisms. This approach hinges on theories that we lay out involving arguments based on the nature of physical systems. In the same vein, we also introduce a metric to search for the best candidate equation using the datasets generated from the systems. This approach involves symbolic regression by means of genetic programming and regressions to compute the strength of the interplay between the extrinsic parameters in a candidate equation. We relate these extrinsic parameters to the hidden properties of the data-generating systems. The behavior of a new similar system can be predicted easily by utilizing the discovered structural form of the general equation. As illustrations, we apply the approach to identify candidate structural forms of the underlying equation governing two cases: the changes in a sensor measurement of degrading engines; and the search for the governing equation of systems with known variations of an intrinsic parameter. △ Less

Submitted 27 August, 2019; originally announced August 2019.

Nonstationary Gauss-Markov Processes: Parameter Estimation and Dispersion

Authors: Peida Tian, Victoria Kostina

Abstract: This paper provides a precise error analysis for the maximum likelihood estimate $\hat{a}_{\text{ML}}(u_1^n)$ of the parameter $a$ given samples $u_1^n = (u_1, \ldots, u_n)'$ drawn from a nonstationary Gauss-Markov process $U_i = a U_{i-1} + Z_i,~i\geq 1$, where $U_0 = 0$, $a> 1$, and $Z_i$'s are independent Gaussian random variables with zero mean and variance $σ^2$. We show a tight nonasymptotic… ▽ More This paper provides a precise error analysis for the maximum likelihood estimate $\hat{a}_{\text{ML}}(u_1^n)$ of the parameter $a$ given samples $u_1^n = (u_1, \ldots, u_n)'$ drawn from a nonstationary Gauss-Markov process $U_i = a U_{i-1} + Z_i,~i\geq 1$, where $U_0 = 0$, $a> 1$, and $Z_i$'s are independent Gaussian random variables with zero mean and variance $σ^2$. We show a tight nonasymptotic exponentially decaying bound on the tail probability of the estimation error. Unlike previous works, our bound is tight already for a sample size of the order of hundreds. We apply the new estimation bound to find the dispersion for lossy compression of nonstationary Gauss-Markov sources. We show that the dispersion is given by the same integral formula that we derived previously for the asymptotically stationary Gauss-Markov sources, i.e., $|a| < 1$. New ideas in the nonstationary case include separately bounding the maximum eigenvalue (which scales exponentially) and the other eigenvalues (which are bounded by constants that depend only on $a$) of the covariance matrix of the source sequence, and new techniques in the derivation of our estimation error bound. △ Less

Submitted 23 March, 2021; v1 submitted 29 June, 2019; originally announced July 2019.

Comments: 25 pages, 5 figures

Journal ref: in IEEE Transactions on Information Theory, vol. 67, no. 4, pp. 2426-2449, April 2021

Fast and interpretable classification of small X-ray diffraction datasets using data augmentation and deep neural networks

Authors: Felipe Oviedo, Zekun Ren, Shi**g Sun, Charlie Settens, Zhe Liu, Noor Titan Putri Hartono, Ramasamy Savitha, Brian L. DeCost, Siyu I. P. Tian, Giuseppe Romano, Aaron Gilad Kusne, Tonio Buonassisi

Abstract: X-ray diffraction (XRD) data acquisition and analysis is among the most time-consuming steps in the development cycle of novel thin-film materials. We propose a machine-learning-enabled approach to predict crystallographic dimensionality and space group from a limited number of thin-film XRD patterns. We overcome the scarce-data problem intrinsic to novel materials development by coupling a superv… ▽ More X-ray diffraction (XRD) data acquisition and analysis is among the most time-consuming steps in the development cycle of novel thin-film materials. We propose a machine-learning-enabled approach to predict crystallographic dimensionality and space group from a limited number of thin-film XRD patterns. We overcome the scarce-data problem intrinsic to novel materials development by coupling a supervised machine learning approach with a model agnostic, physics-informed data augmentation strategy using simulated data from the Inorganic Crystal Structure Database (ICSD) and experimental data. As a test case, 115 thin-film metal halides spanning 3 dimensionalities and 7 space-groups are synthesized and classified. After testing various algorithms, we develop and implement an all convolutional neural network, with cross validated accuracies for dimensionality and space-group classification of 93% and 89%, respectively. We propose average class activation maps, computed from a global average pooling layer, to allow high model interpretability by human experimentalists, elucidating the root causes of misclassification. Finally, we systematically evaluate the maximum XRD pattern step size (data acquisition rate) before loss of predictive accuracy occurs, and determine it to be 0.16°, which enables an XRD pattern to be obtained and classified in 5.5 minutes or less. △ Less

Submitted 23 April, 2019; v1 submitted 20 November, 2018; originally announced November 2018.

Comments: Accepted with minor revisions in npj Computational Materials, Presented in NIPS 2018 Workshop: Machine Learning for Molecules and Materials

The Dispersion of the Gauss-Markov Source

Authors: Peida Tian, Victoria Kostina

Abstract: The Gauss-Markov source produces $U_i = aU_{i-1} + Z_i$ for $i\geq 1$, where $U_0 = 0$, $|a|<1$ and $Z_i\sim\mathcal{N}(0, σ^2)$ are i.i.d. Gaussian random variables. We consider lossy compression of a block of $n$ samples of the Gauss-Markov source under squared error distortion. We obtain the Gaussian approximation for the Gauss-Markov source with excess-distortion criterion for any distortion… ▽ More The Gauss-Markov source produces $U_i = aU_{i-1} + Z_i$ for $i\geq 1$, where $U_0 = 0$, $|a|<1$ and $Z_i\sim\mathcal{N}(0, σ^2)$ are i.i.d. Gaussian random variables. We consider lossy compression of a block of $n$ samples of the Gauss-Markov source under squared error distortion. We obtain the Gaussian approximation for the Gauss-Markov source with excess-distortion criterion for any distortion $d>0$, and we show that the dispersion has a reverse waterfilling representation. This is the \emph{first} finite blocklength result for lossy compression of \emph{sources with memory}. We prove that the finite blocklength rate-distortion function $R(n,d,ε)$ approaches the rate-distortion function $\mathbb{R}(d)$ as $R(n,d,ε) = \mathbb{R}(d) + \sqrt{\frac{V(d)}{n}}Q^{-1}(ε) + o\left(\frac{1}{\sqrt{n}}\right)$, where $V(d)$ is the dispersion, $ε\in (0,1)$ is the excess-distortion probability, and $Q^{-1}$ is the inverse of the $Q$-function. We give a reverse waterfilling integral representation for the dispersion $V(d)$, which parallels that of the rate-distortion functions for Gaussian processes. Remarkably, for all $0 < d\leq \frac{σ^2}{(1+|a|)^2}$, $R(n,d,ε)$ of the Gauss-Markov source coincides with that of $Z_k$, the i.i.d. Gaussian noise driving the process, up to the second-order term. Among novel technical tools developed in this paper is a sharp approximation of the eigenvalues of the covariance matrix of $n$ samples of the Gauss-Markov source, and a construction of a typical set using the maximum likelihood estimate of the parameter $a$ based on $n$ observations. △ Less

Submitted 24 September, 2019; v1 submitted 25 April, 2018; originally announced April 2018.

Comments: 30 pages, 8 figures, shorter version published in the proceedings of IEEE ISIT 2018

Journal ref: in IEEE Transactions on Information Theory, vol. 65, no. 10, pp. 6355-6384, Oct. 2019

Arbitrarily Varying Networks: Capacity-achieving Computationally Efficient Codes

Authors: Peida Tian, Sidharth Jaggi, Mayank Bakshi, Oliver Kosut

Abstract: We consider the problem of communication over a network containing a hidden and malicious adversary that can control a subset of network resources, and aims to disrupt communications. We focus on omniscient node-based adversaries, i.e., the adversaries can control a subset of nodes, and know the message, network code and packets on all links. Characterizing information-theoretically optimal commun… ▽ More We consider the problem of communication over a network containing a hidden and malicious adversary that can control a subset of network resources, and aims to disrupt communications. We focus on omniscient node-based adversaries, i.e., the adversaries can control a subset of nodes, and know the message, network code and packets on all links. Characterizing information-theoretically optimal communication rates as a function of network parameters and bounds on the adversarially controlled network is in general open, even for unicast (single source, single destination) problems. In this work we characterize the information-theoretically optimal randomized capacity of such problems, i.e., under the assumption that the source node shares (an asymptotically negligible amount of) independent common randomness with each network node a priori (for instance, as part of network design). We propose a novel computationally-efficient communication scheme whose rate matches a natural information-theoretically "erasure outer bound" on the optimal rate. Our schemes require no prior knowledge of network topology, and can be implemented in a distributed manner as an overlay on top of classical distributed linear network coding. △ Less

Submitted 6 May, 2016; originally announced May 2016.

Comments: 13 pages, 7 figures, long version of ISIT paper

Extremal Functions of Forbidden Multidimensional Matrices

Authors: Jesse T. Geneson, Peter M. Tian

Abstract: Pattern avoidance is a central topic in graph theory and combinatorics. Pattern avoidance in matrices has applications in computer science and engineering, such as robot motion planning and VLSI circuit design. A $d$-dimensional zero-one matrix $A$ avoids another $d$-dimensional zero-one matrix $P$ if no submatrix of $A$ can be transformed to $P$ by changing some ones to zeros. A fundamental probl… ▽ More Pattern avoidance is a central topic in graph theory and combinatorics. Pattern avoidance in matrices has applications in computer science and engineering, such as robot motion planning and VLSI circuit design. A $d$-dimensional zero-one matrix $A$ avoids another $d$-dimensional zero-one matrix $P$ if no submatrix of $A$ can be transformed to $P$ by changing some ones to zeros. A fundamental problem is to study the maximum number of nonzero entries in a $d$-dimensional $n \times \cdots \times n$ matrix that avoids $P$. This maximum number, denoted by $f(n,P,d)$, is called the extremal function. We advance the extremal theory of matrices in two directions. The methods that we use come from combinatorics, probability, and analysis. Firstly, we obtain non-trivial lower and upper bounds on $f(n,P,d)$ when $n$ is large for every $d$-dimensional block permutation matrix $P$. We establish the tight bound $Θ(n^{d-1})$ on $f(n,P,d)$ for every $d$-dimensional tuple permutation matrix $P$. This tight bound has the lowest possible order that an extremal function of a nontrivial matrix can ever achieve. Secondly, we show that $f(n,P,d)$ is super-homogeneous for a class of matrices $P$. We use this super-homogeneity to show that the limit inferior of the sequence $\{ {f(n,P,d) \over n^{d-1}}\}$ has a lower bound $2^{Ω(k^{1/ d})}$ for a family of $k \times \cdots \times k$ permutation matrices $P$. We also improve the upper bound on the limit superior from $2^{O(k \log k)}$ to $2^{O(k)}$ for all $k \times \cdots \times k$ permutation matrices and show that the new upper bound also holds for tuple permutation matrices. △ Less

Submitted 11 June, 2015; originally announced June 2015.

MSC Class: 05D99

Sequences of formation width $4$ and alternation length $5$

Authors: Jesse Geneson, Peter Tian

Abstract: Sequence pattern avoidance is a central topic in combinatorics. A sequence $s$ contains a sequence $u$ if some subsequence of $s$ can be changed into $u$ by a one-to-one renaming of its letters. If $s$ does not contain $u$, then $s$ avoids $u$. A widely studied extremal function related to pattern avoidance is $Ex(u, n)$, the maximum length of an $n$-letter sequence that avoids $u$ and has every… ▽ More Sequence pattern avoidance is a central topic in combinatorics. A sequence $s$ contains a sequence $u$ if some subsequence of $s$ can be changed into $u$ by a one-to-one renaming of its letters. If $s$ does not contain $u$, then $s$ avoids $u$. A widely studied extremal function related to pattern avoidance is $Ex(u, n)$, the maximum length of an $n$-letter sequence that avoids $u$ and has every $r$ consecutive letters pairwise distinct, where $r$ is the number of distinct letters in $u$. We bound $Ex(u, n)$ using the formation width function, $fw(u)$, which is the minimum $s$ for which there exists $r$ such that any concatenation of $s$ permutations, each on the same $r$ letters, contains $u$. In particular, we identify every sequence $u$ such that $fw(u)=4$ and $u$ contains $ababa$. The significance of this result lies in its implication that, for every such sequence $u$, we have $Ex(u, n) = Θ(n α(n))$, where $α(n)$ denotes the incredibly slow-growing inverse Ackermann function. We have thus identified the extremal function of many infinite classes of previously unidentified sequences. △ Less

Submitted 13 February, 2015; originally announced February 2015.

Comments: 20 pages

MSC Class: 05D99