Showing 1–43 of 43 results for author: Cui, A

On the Relative Completeness of Satisfaction-based Probabilistic Hoare Logic With While Loop

Authors: Xin Sun, Xingchi Su, Xiaoning Bian, Anran Cui

Abstract: Probabilistic Hoare logic (PHL) is an extension of Hoare logic and is specifically useful in verifying randomized programs. It allows researchers to formally reason about the behavior of programs with stochastic elements, ensuring the desired probabilistic properties are upheld. The relative completeness of satisfaction-based PHL has been an open problem ever since the birth of the first PHL in 19… ▽ More Probabilistic Hoare logic (PHL) is an extension of Hoare logic and is specifically useful in verifying randomized programs. It allows researchers to formally reason about the behavior of programs with stochastic elements, ensuring the desired probabilistic properties are upheld. The relative completeness of satisfaction-based PHL has been an open problem ever since the birth of the first PHL in 1979. More specifically, no satisfaction-based PHL with While-loop has been proven to be relatively complete yet. This paper solves this problem by establishing a new PHL with While-loop and prove its relative completeness. The programming language concerned in our PHL is expressively equivalent to the existing PHL systems but brings a lot of convenience in showing completeness. The weakest preterm for While-loop command reveals how it changes the probabilistic properties of computer states, considering both execution branches that halt and infinite runs. We prove the relative completeness of our PHL in two steps. We first establish a semantics and proof system of Hoare triples with probabilistic programs and deterministic assertions. Then, by utilizing the weakest precondition of deterministic assertions, we construct the weakest preterm calculus of probabilistic expressions. The relative completeness of our PHL is then obtained as a consequence of the weakest preterm calculus. △ Less

Submitted 23 June, 2024; originally announced June 2024.

Comments: 13 pages. arXiv admin note: text overlap with arXiv:2405.01940

MSC Class: 03B70 Logic in computer science ACM Class: F.3

DeTra: A Unified Model for Object Detection and Trajectory Forecasting

Authors: Sergio Casas, Ben Agro, Jiageng Mao, Thomas Gilles, Alexander Cui, Thomas Li, Raquel Urtasun

Abstract: The tasks of object detection and trajectory forecasting play a crucial role in understanding the scene for autonomous driving. These tasks are typically executed in a cascading manner, making them prone to compounding errors. Furthermore, there is usually a very thin interface between the two tasks, creating a lossy information bottleneck. To address these challenges, our approach formulates the… ▽ More The tasks of object detection and trajectory forecasting play a crucial role in understanding the scene for autonomous driving. These tasks are typically executed in a cascading manner, making them prone to compounding errors. Furthermore, there is usually a very thin interface between the two tasks, creating a lossy information bottleneck. To address these challenges, our approach formulates the union of the two tasks as a trajectory refinement problem, where the first pose is the detection (current time), and the subsequent poses are the waypoints of the multiple forecasts (future time). To tackle this unified task, we design a refinement transformer that infers the presence, pose, and multi-modal future behaviors of objects directly from LiDAR point clouds and high-definition maps. We call this model DeTra, short for object Detection and Trajectory forecasting. In our experiments, we observe that \ourmodel{} outperforms the state-of-the-art on Argoverse 2 Sensor and Waymo Open Dataset by a large margin, across a broad range of metrics. Last but not least, we perform extensive ablation studies that show the value of refinement for this task, that every proposed component contributes positively to its performance, and that key design choices were made. △ Less

Submitted 13 June, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Authors: Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, Andrea Tacchetti, Colin Gaffney, Samira Daruki, Olcan Sercinoglu, Zach Gleicher, Juliette Love , et al. (1092 additional authors not shown)

Abstract: In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February… ▽ More In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content. △ Less

Submitted 14 June, 2024; v1 submitted 8 March, 2024; originally announced March 2024.

Interferometric Single-Shot Parity Measurement in an InAs-Al Hybrid Device

Authors: Morteza Aghaee, Alejandro Alcaraz Ramirez, Zulfi Alam, Rizwan Ali, Mariusz Andrzejczuk, Andrey Antipov, Mikhail Astafev, Amin Barzegar, Bela Bauer, Jonathan Becker, Umesh Kumar Bhaskar, Alex Bocharov, Srini Boddapati, David Bohn, Jouri Bommer, Leo Bourdet, Arnaud Bousquet, Samuel Boutin, Lucas Casparis, Benjamin James Chapman, Sohail Chatoor, Anna Wulff Christensen, Cassandra Chua, Patrick Codd, William Cole , et al. (137 additional authors not shown)

Abstract: The fusion of non-Abelian anyons or topological defects is a fundamental operation in measurement-only topological quantum computation. In topological superconductors, this operation amounts to a determination of the shared fermion parity of Majorana zero modes. As a step towards this, we implement a single-shot interferometric measurement of fermion parity in indium arsenide-aluminum heterostruct… ▽ More The fusion of non-Abelian anyons or topological defects is a fundamental operation in measurement-only topological quantum computation. In topological superconductors, this operation amounts to a determination of the shared fermion parity of Majorana zero modes. As a step towards this, we implement a single-shot interferometric measurement of fermion parity in indium arsenide-aluminum heterostructures with a gate-defined nanowire. The interferometer is formed by tunnel-coupling the proximitized nanowire to quantum dots. The nanowire causes a state-dependent shift of these quantum dots' quantum capacitance of up to 1 fF. Our quantum capacitance measurements show flux h/2e-periodic bimodality with a signal-to-noise ratio of 1 in 3.7 $μ$s at optimal flux values. From the time traces of the quantum capacitance measurements, we extract a dwell time in the two associated states that is longer than 1 ms at in-plane magnetic fields of approximately 2 T. These results are consistent with a measurement of the fermion parity encoded in a pair of Majorana zero modes that are separated by approximately 3 $μ$m and subjected to a low rate of poisoning by non-equilibrium quasiparticles. The large capacitance shift and long poisoning time enable a parity measurement error probability of 1%. △ Less

Submitted 2 April, 2024; v1 submitted 17 January, 2024; originally announced January 2024.

Comments: Added data on a second measurement of device A and a measurement of device B, expanded discussion of a trivial scenario. Refs added, author list updated

Gemini: A Family of Highly Capable Multimodal Models

Authors: Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee , et al. (1325 additional authors not shown)

Abstract: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr… ▽ More This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI. △ Less

Submitted 17 June, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images

Authors: Aiyu Cui, Jay Mahajan, Viraj Shah, Preeti Gomathinayagam, Chang Liu, Svetlana Lazebnik

Abstract: Most existing methods for virtual try-on focus on studio person images with a limited range of poses and clean backgrounds. They can achieve plausible results for this studio try-on setting by learning to warp a garment image to fit a person's body from paired training data, i.e., garment images paired with images of people wearing the same garment. Such data is often collected from commercial web… ▽ More Most existing methods for virtual try-on focus on studio person images with a limited range of poses and clean backgrounds. They can achieve plausible results for this studio try-on setting by learning to warp a garment image to fit a person's body from paired training data, i.e., garment images paired with images of people wearing the same garment. Such data is often collected from commercial websites, where each garment is demonstrated both by itself and on several models. By contrast, it is hard to collect paired data for in-the-wild scenes, and therefore, virtual try-on for casual images of people with more diverse poses against cluttered backgrounds is rarely studied. In this work, we fill the gap by introducing a StreetTryOn benchmark to evaluate in-the-wild virtual try-on performance and proposing a novel method that can learn it without paired data, from a set of in-the-wild person images directly. Our method achieves robust performance across shop and street domains using a novel DensePose war** correction method combined with diffusion-based conditional inpainting. Our experiments show competitive performance for standard studio try-on tasks and SOTA performance for street try-on and cross-domain try-on tasks. △ Less

Submitted 18 April, 2024; v1 submitted 27 November, 2023; originally announced November 2023.

PST: Improving Quantitative Trading via Program Sketch-based Tuning

Authors: Zhiming Li, Junzhe Jiang, Yushi Cao, Aixin Cui, Bozhi Wu, Bo Li, Yang Liu, Dongning Sun

Abstract: Deep reinforcement learning (DRL) has revolutionized quantitative finance by achieving decent performance without significant human expert knowledge. Despite its achievements, we observe that the current state-of-the-art DRL models are still ineffective in identifying the market trend, causing them to miss good trading opportunities or suffer from large drawdowns when encountering market crashes.… ▽ More Deep reinforcement learning (DRL) has revolutionized quantitative finance by achieving decent performance without significant human expert knowledge. Despite its achievements, we observe that the current state-of-the-art DRL models are still ineffective in identifying the market trend, causing them to miss good trading opportunities or suffer from large drawdowns when encountering market crashes. To tackle this limitation, a natural idea is to embed human expert knowledge regarding the market trend. Whereas, such knowledge is abstract and hard to be quantified. In this paper, we propose a universal neuro-symbolic tuning framework, called program sketch-based tuning (PST). Particularly, PST first proposes using a novel symbolic program sketch to embed the abstract human expert knowledge of market trends. Then we utilize the program sketch to tune a trained DRL policy according to the different market trend of the moment. Finally, in order to optimize this neural-symbolic framework, we propose a novel hybrid optimization method. Extensive evaluations on two popular quantitative trading tasks demonstrate that PST can significantly enhance the performance of previous state-of-the-art DRL strategies while being extremely lightweight. △ Less

Submitted 24 April, 2024; v1 submitted 9 October, 2023; originally announced October 2023.

PQA: Exploring the Potential of Product Quantization in DNN Hardware Acceleration

Authors: Ahmed F. AbouElhamayed, Angela Cui, Javier Fernandez-Marques, Nicholas D. Lane, Mohamed S. Abdelfattah

Abstract: Conventional multiply-accumulate (MAC) operations have long dominated computation time for deep neural networks (DNNs), espcially convolutional neural networks (CNNs). Recently, product quantization (PQ) has been applied to these workloads, replacing MACs with memory lookups to pre-computed dot products. To better understand the efficiency tradeoffs of product-quantized DNNs (PQ-DNNs), we create a… ▽ More Conventional multiply-accumulate (MAC) operations have long dominated computation time for deep neural networks (DNNs), espcially convolutional neural networks (CNNs). Recently, product quantization (PQ) has been applied to these workloads, replacing MACs with memory lookups to pre-computed dot products. To better understand the efficiency tradeoffs of product-quantized DNNs (PQ-DNNs), we create a custom hardware accelerator to parallelize and accelerate nearest-neighbor search and dot-product lookups. Additionally, we perform an empirical study to investigate the efficiency--accuracy tradeoffs of different PQ parameterizations and training methods. We identify PQ configurations that improve performance-per-area for ResNet20 by up to 3.1$\times$, even when compared to a highly optimized conventional DNN accelerator, with similar improvements on two additional compact DNNs. When comparing to recent PQ solutions, we outperform prior work by $4\times$ in terms of performance-per-area with a 0.6% accuracy degradation. Finally, we reduce the bitwidth of PQ operations to investigate the impact on both hardware efficiency and accuracy. With only 2-6-bit precision on three compact DNNs, we were able to maintain DNN accuracy eliminating the need for DSPs. △ Less

Submitted 28 March, 2024; v1 submitted 25 May, 2023; originally announced May 2023.

Comments: ACM Transactions on Reconfigurable Technology and Systems (TRETS) - FCCM 2024 Journal Track

One-Shot Stylization for Full-Body Human Images

Authors: Aiyu Cui, Svetlana Lazebnik

Abstract: The goal of human stylization is to transfer full-body human photos to a style specified by a single art character reference image. Although previous work has succeeded in example-based stylization of faces and generic scenes, full-body human stylization is a more complex domain. This work addresses several unique challenges of stylizing full-body human images. We propose a method for one-shot fin… ▽ More The goal of human stylization is to transfer full-body human photos to a style specified by a single art character reference image. Although previous work has succeeded in example-based stylization of faces and generic scenes, full-body human stylization is a more complex domain. This work addresses several unique challenges of stylizing full-body human images. We propose a method for one-shot fine-tuning of a pose-guided human generator to preserve the "content" (garments, face, hair, pose) of the input photo and the "style" of the artistic reference. Since body shape deformation is an essential component of an art character's style, we incorporate a novel skeleton deformation module to reshape the pose of the input person and modify the DiOr pose-guided person generator to be more robust to the rescaled poses falling outside the distribution of the realistic poses that the generator is originally trained on. Several human studies verify the effectiveness of our approach. △ Less

Submitted 13 April, 2023; originally announced April 2023.

Learning Garment DensePose for Robust War** in Virtual Try-On

Authors: Aiyu Cui, Sen He, Tao Xiang, Antoine Toisoul

Abstract: Virtual try-on, i.e making people virtually try new garments, is an active research area in computer vision with great commercial applications. Current virtual try-on methods usually work in a two-stage pipeline. First, the garment image is warped on the person's pose using a flow estimation network. Then in the second stage, the warped garment is fused with the person image to render a new try-on… ▽ More Virtual try-on, i.e making people virtually try new garments, is an active research area in computer vision with great commercial applications. Current virtual try-on methods usually work in a two-stage pipeline. First, the garment image is warped on the person's pose using a flow estimation network. Then in the second stage, the warped garment is fused with the person image to render a new try-on image. Unfortunately, such methods are heavily dependent on the quality of the garment war** which often fails when dealing with hard poses (e.g., a person lifting or crossing arms). In this work, we propose a robust war** method for virtual try-on based on a learned garment DensePose which has a direct correspondence with the person's DensePose. Due to the lack of annotated data, we show how to leverage an off-the-shelf person DensePose model and a pretrained flow model to learn the garment DensePose in a weakly supervised manner. The garment DensePose allows a robust war** to any person's pose without any additional computation. Our method achieves the state-of-the-art equivalent on virtual try-on benchmarks and shows war** robustness on in-the-wild person images with hard poses, making it more suited for real-world virtual try-on applications. △ Less

Submitted 30 March, 2023; originally announced March 2023.

Comments: 6 pages

GoRela: Go Relative for Viewpoint-Invariant Motion Forecasting

Authors: Alexander Cui, Sergio Casas, Kelvin Wong, Simon Suo, Raquel Urtasun

Abstract: The task of motion forecasting is critical for self-driving vehicles (SDVs) to be able to plan a safe maneuver. Towards this goal, modern approaches reason about the map, the agents' past trajectories and their interactions in order to produce accurate forecasts. The predominant approach has been to encode the map and other agents in the reference frame of each target agent. However, this approach… ▽ More The task of motion forecasting is critical for self-driving vehicles (SDVs) to be able to plan a safe maneuver. Towards this goal, modern approaches reason about the map, the agents' past trajectories and their interactions in order to produce accurate forecasts. The predominant approach has been to encode the map and other agents in the reference frame of each target agent. However, this approach is computationally expensive for multi-agent prediction as inference needs to be run for each agent. To tackle the scaling challenge, the solution thus far has been to encode all agents and the map in a shared coordinate frame (e.g., the SDV frame). However, this is sample inefficient and vulnerable to domain shift (e.g., when the SDV visits uncommon states). In contrast, in this paper, we propose an efficient shared encoding for all agents and the map without sacrificing accuracy or generalization. Towards this goal, we leverage pair-wise relative positional encodings to represent geometric relationships between the agents and the map elements in a heterogeneous spatial graph. This parameterization allows us to be invariant to scene viewpoint, and save online computation by re-using map embeddings computed offline. Our decoder is also viewpoint agnostic, predicting agent goals on the lane graph to enable diverse and context-aware multimodal prediction. We demonstrate the effectiveness of our approach on the urban Argoverse 2 benchmark as well as a novel highway dataset. △ Less

Submitted 8 November, 2022; v1 submitted 4 November, 2022; originally announced November 2022.

Local Relighting of Real Scenes

Authors: Audrey Cui, Ali Jahanian, Agata Lapedriza, Antonio Torralba, Shahin Mahdizadehaghdam, Rohit Kumar, David Bau

Abstract: We introduce the task of local relighting, which changes a photograph of a scene by switching on and off the light sources that are visible within the image. This new task differs from the traditional image relighting problem, as it introduces the challenge of detecting light sources and inferring the pattern of light that emanates from them. We propose an approach for local relighting that trains… ▽ More We introduce the task of local relighting, which changes a photograph of a scene by switching on and off the light sources that are visible within the image. This new task differs from the traditional image relighting problem, as it introduces the challenge of detecting light sources and inferring the pattern of light that emanates from them. We propose an approach for local relighting that trains a model without supervision of any novel image dataset by using synthetically generated image pairs from another model. Concretely, we collect paired training images from a stylespace-manipulated GAN; then we use these images to train a conditional image-to-image model. To benchmark local relighting, we introduce Lonoff, a collection of 306 precisely aligned images taken in indoor spaces with different combinations of lights switched on. We show that our method significantly outperforms baseline methods based on GAN inversion. Finally, we demonstrate extensions of our method that control different light sources separately. We invite the community to tackle this new task of local relighting. △ Less

Submitted 6 July, 2022; originally announced July 2022.

Comments: 15 pages, 15 figures

InAs-Al Hybrid Devices Passing the Topological Gap Protocol

Authors: Morteza Aghaee, Arun Akkala, Zulfi Alam, Rizwan Ali, Alejandro Alcaraz Ramirez, Mariusz Andrzejczuk, Andrey E Antipov, Pavel Aseev, Mikhail Astafev, Bela Bauer, Jonathan Becker, Srini Boddapati, Frenk Boekhout, Jouri Bommer, Esben Bork Hansen, Tom Bosma, Leo Bourdet, Samuel Boutin, Philippe Caroff, Lucas Casparis, Maja Cassidy, Anna Wulf Christensen, Noah Clay, William S Cole, Fabiano Corsetti , et al. (102 additional authors not shown)

Abstract: We present measurements and simulations of semiconductor-superconductor heterostructure devices that are consistent with the observation of topological superconductivity and Majorana zero modes. The devices are fabricated from high-mobility two-dimensional electron gases in which quasi-one-dimensional wires are defined by electrostatic gates. These devices enable measurements of local and non-loca… ▽ More We present measurements and simulations of semiconductor-superconductor heterostructure devices that are consistent with the observation of topological superconductivity and Majorana zero modes. The devices are fabricated from high-mobility two-dimensional electron gases in which quasi-one-dimensional wires are defined by electrostatic gates. These devices enable measurements of local and non-local transport properties and have been optimized via extensive simulations to ensure robustness against non-uniformity and disorder. Our main result is that several devices, fabricated according to the design's engineering specifications, have passed the topological gap protocol defined in Pikulin et al. [arXiv:2103.12217]. This protocol is a stringent test composed of a sequence of three-terminal local and non-local transport measurements performed while varying the magnetic field, semiconductor electron density, and junction transparencies. Passing the protocol indicates a high probability of detection of a topological phase hosting Majorana zero modes as determined by large-scale disorder simulations. Our experimental results are consistent with a quantum phase transition into a topological superconducting phase that extends over several hundred millitesla in magnetic field and several millivolts in gate voltage, corresponding to approximately one hundred micro-electron-volts in Zeeman energy and chemical potential in the semiconducting wire. These regions feature a closing and re-opening of the bulk gap, with simultaneous zero-bias conductance peaks at both ends of the devices that withstand changes in the junction transparencies. The extracted maximum topological gaps in our devices are 20-60 $μ$eV. This demonstration is a prerequisite for experiments involving fusion and braiding of Majorana zero modes. △ Less

Submitted 8 March, 2024; v1 submitted 6 July, 2022; originally announced July 2022.

Comments: Final version

Estimating beneficiaries of the child tax credit: past, present, and future

Authors: Ashley Nunes, Chung Yi See, Lucas Woodley, Nicole A. Divers, Audrey L. Cui

Abstract: Government efforts to address child poverty commonly encompass economic assistance programs that bolster household income. The Child Tax Credit (CTC) is the most prominent example of this. Introduced by the United States Congress in 1997, the program endeavors to help working parents via income stabilization. Our work examines the extent to which the CTC has done so. Our study, which documents cle… ▽ More Government efforts to address child poverty commonly encompass economic assistance programs that bolster household income. The Child Tax Credit (CTC) is the most prominent example of this. Introduced by the United States Congress in 1997, the program endeavors to help working parents via income stabilization. Our work examines the extent to which the CTC has done so. Our study, which documents clear, consistent, and compelling evidence of gender inequity in benefits realization, yields four key findings. First, stringent requisite income thresholds disproportionally disadvantage single mothers, a reflection of the high concentration of this demographic in lower segments of the income distribution. Second, married parents and, to a lesser extent, single fathers, are the primary beneficiaries of the CTC program when benefits are structured as credits rather than refunds. Third, making program benefits more generous disproportionally reduces how many single mothers, relative to married parents and single fathers, can claim this benefit. Fourth and finally, increasing credit refundability can mitigate gender differences in relief eligibility, although doing so imposes externalities of its own. Our findings can inform public policy discourse surrounding the efficacy of programs like the CTC and the effectiveness of programs aimed at alleviating child poverty. △ Less

Submitted 2 May, 2022; originally announced May 2022.

Comments: 64 pages, 7 figures

Selective Differential Privacy for Language Modeling

Authors: Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, Zhou Yu

Abstract: With the increasing applications of language models, it has become crucial to protect these models from leaking private information. Previous work has attempted to tackle this challenge by training RNN-based language models with differential privacy guarantees. However, applying classical differential privacy to language models leads to poor model performance as the underlying privacy notion is ov… ▽ More With the increasing applications of language models, it has become crucial to protect these models from leaking private information. Previous work has attempted to tackle this challenge by training RNN-based language models with differential privacy guarantees. However, applying classical differential privacy to language models leads to poor model performance as the underlying privacy notion is over-pessimistic and provides undifferentiated protection for all tokens in the data. Given that the private information in natural language is sparse (for example, the bulk of an email might not carry personally identifiable information), we propose a new privacy notion, selective differential privacy, to provide rigorous privacy guarantees on the sensitive portion of the data to improve model utility. To realize such a new notion, we develop a corresponding privacy mechanism, Selective-DPSGD, for RNN-based language models. Besides language modeling, we also apply the method to a more concrete application--dialog systems. Experiments on both language modeling and dialog system building show that the proposed privacy-preserving mechanism achieves better utilities while remaining safe under various privacy attacks compared to the baselines. The data and code are released at https://github.com/wyshi/lm_privacy to facilitate future research . △ Less

Submitted 16 July, 2022; v1 submitted 29 August, 2021; originally announced August 2021.

Comments: NAACL 2022

Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing

Authors: Aiyu Cui, Daniel McKee, Svetlana Lazebnik

Abstract: We propose a flexible person generation framework called Dressing in Order (DiOr), which supports 2D pose transfer, virtual try-on, and several fashion editing tasks. The key to DiOr is a novel recurrent generation pipeline to sequentially put garments on a person, so that trying on the same garments in different orders will result in different looks. Our system can produce dressing effects not ac… ▽ More We propose a flexible person generation framework called Dressing in Order (DiOr), which supports 2D pose transfer, virtual try-on, and several fashion editing tasks. The key to DiOr is a novel recurrent generation pipeline to sequentially put garments on a person, so that trying on the same garments in different orders will result in different looks. Our system can produce dressing effects not achievable by existing work, including different interactions of garments (e.g., wearing a top tucked into the bottom or over it), as well as layering of multiple garments of the same type (e.g., jacket over shirt over t-shirt). DiOr explicitly encodes the shape and texture of each garment, enabling these elements to be edited separately. Joint training on pose transfer and inpainting helps with detail preservation and coherence of generated garments. Extensive evaluations show that DiOr outperforms other recent methods like ADGAN in terms of output quality, and handles a wide range of editing functions for which there is no direct supervision. △ Less

Submitted 18 October, 2022; v1 submitted 14 April, 2021; originally announced April 2021.

Comments: ICCV 2021

Paint by Word

Authors: Alex Andonian, Sabrina Osmany, Audrey Cui, YeonHwan Park, Ali Jahanian, Antonio Torralba, David Bau

Abstract: We investigate the problem of zero-shot semantic image painting. Instead of painting modifications into an image using only concrete colors or a finite set of semantic concepts, we ask how to create semantic paint based on open full-text descriptions: our goal is to be able to point to a location in a synthesized image and apply an arbitrary new concept such as "rustic" or "opulent" or "happy dog.… ▽ More We investigate the problem of zero-shot semantic image painting. Instead of painting modifications into an image using only concrete colors or a finite set of semantic concepts, we ask how to create semantic paint based on open full-text descriptions: our goal is to be able to point to a location in a synthesized image and apply an arbitrary new concept such as "rustic" or "opulent" or "happy dog." To do this, our method combines a state-of-the art generative model of realistic images with a state-of-the-art text-image semantic similarity network. We find that, to make large changes, it is important to use non-gradient methods to explore latent space, and it is important to relax the computations of the GAN to target changes to a specific region. We conduct user studies to compare our methods to several baselines. △ Less

Submitted 23 March, 2023; v1 submitted 19 March, 2021; originally announced March 2021.

Comments: 10 pages, 9 figures

ACM Class: I.2.10; I.4; I.3

LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

Authors: Alexander Cui, Sergio Casas, Abbas Sadat, Renjie Liao, Raquel Urtasun

Abstract: In this paper, we present LookOut, a novel autonomy system that perceives the environment, predicts a diverse set of futures of how the scene might unroll and estimates the trajectory of the SDV by optimizing a set of contingency plans over these future realizations. In particular, we learn a diverse joint distribution over multi-agent future trajectories in a traffic scene that covers a wide rang… ▽ More In this paper, we present LookOut, a novel autonomy system that perceives the environment, predicts a diverse set of futures of how the scene might unroll and estimates the trajectory of the SDV by optimizing a set of contingency plans over these future realizations. In particular, we learn a diverse joint distribution over multi-agent future trajectories in a traffic scene that covers a wide range of future modes with high sample efficiency while leveraging the expressive power of generative models. Unlike previous work in diverse motion forecasting, our diversity objective explicitly rewards sampling future scenarios that require distinct reactions from the self-driving vehicle for improved safety. Our contingency planner then finds comfortable and non-conservative trajectories that ensure safe reactions to a wide range of future scenarios. Through extensive evaluations, we show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset as well as safer and less-conservative motion plans in long-term closed-loop simulations when compared to current state-of-the-art models. △ Less

Submitted 7 May, 2021; v1 submitted 16 January, 2021; originally announced January 2021.

Graph Neural Networks for the Prediction of Substrate-Specific Organic Reaction Conditions

Authors: Serim Ryou, Michael R. Maser, Alexander Y. Cui, Travis J. DeLano, Yisong Yue, Sarah E. Reisman

Abstract: We present a systematic investigation using graph neural networks (GNNs) to model organic chemical reactions. To do so, we prepared a dataset collection of four ubiquitous reactions from the organic chemistry literature. We evaluate seven different GNN architectures for classification tasks pertaining to the identification of experimental reagents and conditions. We find that models are able to id… ▽ More We present a systematic investigation using graph neural networks (GNNs) to model organic chemical reactions. To do so, we prepared a dataset collection of four ubiquitous reactions from the organic chemistry literature. We evaluate seven different GNN architectures for classification tasks pertaining to the identification of experimental reagents and conditions. We find that models are able to identify specific graph features that affect reaction conditions and lead to accurate predictions. The results herein show great promise in advancing molecular machine learning. △ Less

Submitted 9 July, 2020; v1 submitted 8 July, 2020; originally announced July 2020.

Comments: 23 pages, 10 tables, 13 figures, to appear in the ICML 2020 Workshop on Graph Representation Learning and Beyond (GRLB)

Transparent Gatable Superconducting Shadow Junctions

Authors: Sabbir A. Khan, Charalampos Lampadaris, Ajuan Cui, Lukas Stampfer, Yu Liu, S. J. Pauka, Martin E. Cachaza, Elisabetta M. Fiordaliso, Jung-Hyun Kang, Svetlana Korneychuk, Timo Mutas, Joachim E. Sestoft, Filip Krizek, Rawa Tanta, M. C. Cassidy, Thomas S. Jespersen, Peter Krogstrup

Abstract: Gate tunable junctions are key elements in quantum devices based on hybrid semiconductor-superconductor materials. They serve multiple purposes ranging from tunnel spectroscopy probes to voltage-controlled qubit operations in gatemon and topological qubits. Common to all is that junction transparency plays a critical role. In this study, we grow single crystalline InAs, InSb and… ▽ More Gate tunable junctions are key elements in quantum devices based on hybrid semiconductor-superconductor materials. They serve multiple purposes ranging from tunnel spectroscopy probes to voltage-controlled qubit operations in gatemon and topological qubits. Common to all is that junction transparency plays a critical role. In this study, we grow single crystalline InAs, InSb and $\mathrm{InAs_{1-x}Sb_x}$ nanowires with epitaxial superconductors and in-situ shadowed junctions in a single-step molecular beam epitaxy process. We investigate correlations between fabrication parameters, junction morphologies, and electronic transport properties of the junctions and show that the examined in-situ shadowed junctions are of significantly higher quality than the etched junctions. By varying the edge sharpness of the shadow junctions we show that the sharpest edges yield the highest junction transparency for all three examined semiconductors. Further, critical supercurrent measurements reveal an extraordinarily high $I_\mathrm{C} R_\mathrm{N}$, close to the KO$-$2 limit. This study demonstrates a promising engineering path towards reliable gate-tunable superconducting qubits. △ Less

Submitted 9 March, 2020; originally announced March 2020.

Comments: 10 pages, 5 figures, 48 references

Adaptive iterative singular value thresholding algorithm to low-rank matrix recovery

Authors: Angang Cui, Jigen Peng, Haiyang Li

Abstract: The problem of recovering a low-rank matrix from the linear constraints, known as affine matrix rank minimization problem, has been attracting extensive attention in recent years. In general, affine matrix rank minimization problem is a NP-hard. In our latest work, a non-convex fraction function is studied to approximate the rank function in affine matrix rank minimization problem and translate th… ▽ More The problem of recovering a low-rank matrix from the linear constraints, known as affine matrix rank minimization problem, has been attracting extensive attention in recent years. In general, affine matrix rank minimization problem is a NP-hard. In our latest work, a non-convex fraction function is studied to approximate the rank function in affine matrix rank minimization problem and translate the NP-hard affine matrix rank minimization problem into a transformed affine matrix rank minimization problem. A scheme of iterative singular value thresholding algorithm is generated to solve the regularized transformed affine matrix rank minimization problem. However, one of the drawbacks for our iterative singular value thresholding algorithm is that the parameter $a$, which influences the behaviour of non-convex fraction function in the regularized transformed affine matrix rank minimization problem, needs to be determined manually in every simulation. In fact, how to determine the optimal parameter $a$ is not an easy problem. Here instead, in this paper, we will generate an adaptive iterative singular value thresholding algorithm to solve the regularized transformed affine matrix rank minimization problem. When doing so, our new algorithm will be intelligent both for the choice of the regularized parameter $λ$ and the parameter $a$. △ Less

Submitted 30 January, 2020; v1 submitted 16 January, 2020; originally announced January 2020.

Nonconvex fraction function recovery sparse signal by convex optimization algorithm

Authors: Angang Cui, Jigen Peng, Haiyang Li, Meng Wen

Abstract: In this paper, we will generate a convex iterative FP thresholding algorithm to solve the problem $(FP^λ_{a})$. Two schemes of convex iterative FP thresholding algorithms are generated. One is convex iterative FP thresholding algorithm-Scheme 1 and the other is convex iterative FP thresholding algorithm-Scheme 2. A global convergence theorem is proved for the convex iterative FP thresholding algor… ▽ More In this paper, we will generate a convex iterative FP thresholding algorithm to solve the problem $(FP^λ_{a})$. Two schemes of convex iterative FP thresholding algorithms are generated. One is convex iterative FP thresholding algorithm-Scheme 1 and the other is convex iterative FP thresholding algorithm-Scheme 2. A global convergence theorem is proved for the convex iterative FP thresholding algorithm-Scheme 1. Under an adaptive rule, the convex iterative FP thresholding algorithm-Scheme 2 will be adaptive both for the choice of the regularized parameter $λ$ and parameter $a$. These are the advantages for our two schemes of convex iterative FP thresholding algorithm compared with our previous proposed two schemes of iterative FP thresholding algorithm. At last, we provide a series of numerical simulations to test the performance of the convex iterative FP thresholding algorithm-Scheme 2, and the simulation results show that our convex iterative FP thresholding algorithm-Scheme 2 performs very well in recovering a sparse signal. △ Less

Submitted 28 May, 2019; v1 submitted 14 May, 2019; originally announced May 2019.

A Taxonomy of Crystallographic Sphere Packings

Authors: Debra Chait, Alisa Cui, Zachary Stier

Abstract: The Apollonian circle packing, generated from three mutually-tangent circles in the plane, has inspired over the past half-century the study of other classes of space-filling packings, both in two and in higher dimensions. Recently, Kontorovich and Nakamura introduced the notion of crystallographic sphere packings, $n$-dimensional packings of spheres with symmetry groups that are isometries of… ▽ More The Apollonian circle packing, generated from three mutually-tangent circles in the plane, has inspired over the past half-century the study of other classes of space-filling packings, both in two and in higher dimensions. Recently, Kontorovich and Nakamura introduced the notion of crystallographic sphere packings, $n$-dimensional packings of spheres with symmetry groups that are isometries of $\mathbb{H}^{n+1}$. There exist at least three sources which give rise to crystallographic packings, namely polyhedra, reflective extended Bianchi groups, and various higher dimensional quadratic forms. When applied in conjunction with the Koebe-Andreev-Thurston Theorem, Kontorovich and Nakamura's Structure Theorem guarantees crystallographic packings to be generated from polyhedra in $n=2$. The Structure Theorem similarly allows us to generate packings from the reflective extended Bianchi groups in $n=2$ by applying Vinberg's algorithm to obtain the appropriate Coxeter diagrams. In $n>2$, the Structure Theorem when used with Vinberg's algorithm allows us to explore whether certain Coxeter diagrams in $\mathbb{H}^{n+1}$ for a given quadratic form admit a packing at all. Kontorovich and Nakamura's Finiteness Theorem shows that there exist only finitely many classes of superintegral such packings, all of which exist in dimensions $n\le20$. In this work, we systematically determine all known examples of crystallographic sphere packings. △ Less

Submitted 6 March, 2019; originally announced March 2019.

A non-convex approach to low-rank and sparse matrix decomposition

Authors: Angang Cui, Meng Wen, Haiyang Li, Jigen Peng

Abstract: In this paper, we develop a nonconvex approach to the problem of low-rank and sparse matrix decomposition. In our nonconvex method, we replace the rank function and the $l_{0}$-norm of a given matrix with a non-convex fraction function on the singular values and the elements of the matrix respectively. An alternative direction method of multipliers algorithm is utilized to solve our proposed nonco… ▽ More In this paper, we develop a nonconvex approach to the problem of low-rank and sparse matrix decomposition. In our nonconvex method, we replace the rank function and the $l_{0}$-norm of a given matrix with a non-convex fraction function on the singular values and the elements of the matrix respectively. An alternative direction method of multipliers algorithm is utilized to solve our proposed nonconvex problem with the nonconvex fraction function penalty. Numerical experiments on some low-rank and sparse matrix decomposition problems show that our method performs very well in recovering low-rank matrices which are heavily corrupted by large sparse errors. △ Less

Submitted 11 May, 2019; v1 submitted 1 July, 2018; originally announced July 2018.

A New Nonconvex Strategy to Affine Matrix Rank Minimization Problem

Authors: Angang Cui, Jigen Peng, Haiyang Li, Junxiong Jia, Meng Wen

Abstract: The affine matrix rank minimization (AMRM) problem is to find a matrix of minimum rank that satisfies a given linear system constraint. It has many applications in some important areas such as control, recommender systems, matrix completion and network localization. However, the problem (AMRM) is NP-hard in general due to the combinational nature of the matrix rank function. There are many alterna… ▽ More The affine matrix rank minimization (AMRM) problem is to find a matrix of minimum rank that satisfies a given linear system constraint. It has many applications in some important areas such as control, recommender systems, matrix completion and network localization. However, the problem (AMRM) is NP-hard in general due to the combinational nature of the matrix rank function. There are many alternative functions have been proposed to substitute the matrix rank function, which lead to many corresponding alternative minimization problems solved efficiently by some popular convex or nonconvex optimization algorithms. In this paper, we propose a new nonconvex function, namely, $TL_α^ε$ function (with $0\leqα<1$ and $ε>0$), to approximate the rank function, and translate the NP-hard problem (AMRM) into the $TL_{p}^ε$ function affine matrix rank minimization (TLAMRM) problem. Firstly, we study the equivalence of problem (AMRM) and (TLAMRM), and proved that the uniqueness of global minimizer of the problem (TLAMRM) also solves the NP-hard problem (AMRM) if the linear map $\mathcal{A}$ satisfies a restricted isometry property (RIP). Secondly, an iterative thresholding algorithm is proposed to solve the regularization problem (RTLAMRM) for all $0\leqα<1$ and $ε>0$. At last, some numerical results on low-rank matrix completion problems illustrated that our algorithm is able to recover a low-rank matrix, and the extensive numerical on image inpainting problems shown that our algorithm performs the best in finding a low-rank image compared with some state-of-art methods. △ Less

Submitted 22 November, 2018; v1 submitted 29 April, 2018; originally announced April 2018.

Iterative thresholding algorithm based on non-convex method for modified lp-norm regularization minimization

Authors: Angang Cui, Jigen Peng, Haiyang Li, Meng Wen, Jiajun Xiong

Abstract: Recently, the $ł_{p}$-norm regularization minimization problem $(P_{p}^λ)$ has attracted great attention in compressed sensing. However, the $ł_{p}$-norm $\|x\|_{p}^{p}$ in problem $(P_{p}^λ)$ is nonconvex and non-Lipschitz for all $p\in(0,1)$, and there are not many optimization theories and methods are proposed to solve this problem. In fact, it is NP-hard for all $p\in(0,1)$ and $λ>0$. In this… ▽ More Recently, the $ł_{p}$-norm regularization minimization problem $(P_{p}^λ)$ has attracted great attention in compressed sensing. However, the $ł_{p}$-norm $\|x\|_{p}^{p}$ in problem $(P_{p}^λ)$ is nonconvex and non-Lipschitz for all $p\in(0,1)$, and there are not many optimization theories and methods are proposed to solve this problem. In fact, it is NP-hard for all $p\in(0,1)$ and $λ>0$. In this paper, we study two modified $ł_{p}$ regularization minimization problems to approximate the NP-hard problem $(P_{p}^λ)$. Inspired by the good performance of Half algorithm and $2/3$ algorithm in some sparse signal recovery problems, two iterative thresholding algorithms are proposed to solve the problems $(P_{p,1/2,ε}^λ)$ and $(P_{p,2/3,ε}^λ)$ respectively. Numerical results show that our algorithms perform effectively in finding the sparse signal in some sparse signal recovery problems for some proper $p\in(0,1)$. △ Less

Submitted 25 April, 2018; originally announced April 2018.

Modified lp-norm regularization minimization for sparse signal recovery

Authors: Angang Cui, Jigen Peng, Haiyang Li

Abstract: In numerous substitution models for the $ł_{0}$-norm minimization problem $(P_{0})$, the $ł_{p}$-norm minimization $(P_{p})$ with $0<p<1$ have been considered as the most natural choice. However, the non-convex optimization problem $(P_{p})$ are much more computational challenges, and are also NP-hard. Meanwhile, the algorithms corresponding to the proximal map** of the regularization $ł_{p}$-no… ▽ More In numerous substitution models for the $ł_{0}$-norm minimization problem $(P_{0})$, the $ł_{p}$-norm minimization $(P_{p})$ with $0<p<1$ have been considered as the most natural choice. However, the non-convex optimization problem $(P_{p})$ are much more computational challenges, and are also NP-hard. Meanwhile, the algorithms corresponding to the proximal map** of the regularization $ł_{p}$-norm minimization $(P_{p}^λ)$ are limited to few specific values of parameter $p$. In this paper, we replace the $\ell_{p}$-norm $\|x\|_{p}^{p}$ with a modified function $\sum_{i=1}^{n}\frac{|x_{i}|}{(|x_{i}|+ε_{i})^{1-p}}$. With change the parameter $ε>0$, this modified function would like to interpolate the $ł_{p}$-norm $\|x\|_{p}^{p}$. By this transformation, we translated the $ł_{p}$-norm regularization minimization $(P_{p}^λ)$ into a modified $ł_{p}$-norm regularization minimization $(P_{p}^{λ,ε})$. Then, we develop the thresholding representation theory of the problem $(P_{p}^{λ,ε})$, and based on it, the IT algorithm is proposed to solve the problem $(P_{p}^{λ,ε})$ for all $0<p<1$. Indeed, we could get some much better results by choosing proper $p$, which is one of the advantages for our algorithm compared with other methods. Numerical results also show that, for some proper $p$, our algorithm performs the best in some sparse signal recovery problems compared with some state-of-art methods. △ Less

Submitted 26 April, 2018; v1 submitted 27 January, 2018; originally announced January 2018.

Sparse Portfolio Selection via Non-convex Fraction Function

Authors: Angang Cui, Jigen Peng, Chengyi Zhang, Haiyang Li, Meng Wen

Abstract: In this paper, a continuous and non-convex promoting sparsity fraction function is studied in two sparse portfolio selection models with and without short-selling constraints. Firstly, we study the properties of the optimal solution to the problem $(FP_{a,λ,η})$ including the first-order and the second optimality condition and the lower and upper bound of the absolute value for its nonzero entries… ▽ More In this paper, a continuous and non-convex promoting sparsity fraction function is studied in two sparse portfolio selection models with and without short-selling constraints. Firstly, we study the properties of the optimal solution to the problem $(FP_{a,λ,η})$ including the first-order and the second optimality condition and the lower and upper bound of the absolute value for its nonzero entries. Secondly, we develop the thresholding representation theory of the problem $(FP_{a,λ,η})$. Based on it, we prove the existence of the resolvent operator of gradient of $P_{a}(x)$, calculate its analytic expression, and propose an iterative fraction penalty thresholding (IFPT) algorithm to solve the problem $(FP_{a,λ,η})$. Moreover, we also prove that the value of the regularization parameter $λ>0$ can not be chosen too large. Indeed, there exists $\barλ>0$ such that the optimal solution to the problem $(FP_{a,λ,η})$ is equal to zero for any $λ>\barλ$. At last, inspired by the thresholding representation theory of the problem $(FP_{a,λ,η})$, we propose an iterative nonnegative fraction penalty thresholding (INFPT) algorithm to solve the problem $(FP_{a,λ,η}^{\geq})$. Empirical results show that our methods, for some proper $a>0$, perform effective in finding the sparse portfolio weights with and without short-selling constraints. △ Less

Submitted 27 January, 2018; originally announced January 2018.

Recovering Sparse Nonnegative Signals via Non-convex Fraction Function Penalty

Authors: Angang Cui, Haiyang Li, Meng Wen, Jigen Peng

Abstract: Many real world practical problems can be formulated as $\ell_{0}$-minimization problems with nonnegativity constraints, which seek the sparsest nonnegative signals to underdetermined linear systems. They have been widely applied in signal and image processing, machine learning, pattern recognition and computer vision. Unfortunately, this $\ell_{0}$-minimization problem with nonnegativity constrai… ▽ More Many real world practical problems can be formulated as $\ell_{0}$-minimization problems with nonnegativity constraints, which seek the sparsest nonnegative signals to underdetermined linear systems. They have been widely applied in signal and image processing, machine learning, pattern recognition and computer vision. Unfortunately, this $\ell_{0}$-minimization problem with nonnegativity constraint is computational and NP-hard because of the discrete and discontinuous nature of the $\ell_{0}$-norm. In this paper, we replace the $\ell_{0}$-norm with a non-convex fraction function, and study the minimization problem of this non-convex fraction function in recovering the sparse nonnegative signals from an underdetermined linear system. Firstly, we discuss the equivalence between $(P_{0}^{\geq})$ and $(FP_{a}^{\geq})$, and the equivalence between $(FP_{a}^{\geq})$ and $(FP_{a,λ}^{\geq})$. It is proved that the optimal solution of the problem $(P_{0}^{\geq})$ could be approximately obtained by solving the regularization problem $(FP_{a,λ}^{\geq})$ if some specific conditions satisfied. Secondly, we propose a nonnegative iterative thresholding algorithm to solve the regularization problem $(FP_{a,λ}^{\geq})$ for all $a>0$. Finally, some numerical experiments on sparse nonnegative siganl recovery problems show that our method performs effective in finding sparse nonnegative signals compared with the linear programming. △ Less

Submitted 26 August, 2017; v1 submitted 20 July, 2017; originally announced July 2017.

Generalized singular value thresholding operator to affine matrix rank minimization problem

Authors: Angang Cui, Haiyang Li, Jigen Peng, Junxiong Jia

Abstract: It is well known that the affine matrix rank minimization problem is NP-hard and all known algorithms for exactly solving it are doubly exponential in theory and in practice due to the combinational nature of the rank function. In this paper, a generalized singular value thresholding operator is generated to solve the affine matrix rank minimization problem. Numerical experiments show that our alg… ▽ More It is well known that the affine matrix rank minimization problem is NP-hard and all known algorithms for exactly solving it are doubly exponential in theory and in practice due to the combinational nature of the rank function. In this paper, a generalized singular value thresholding operator is generated to solve the affine matrix rank minimization problem. Numerical experiments show that our algorithm performs effectively in finding a low-rank matrix compared with some state-of-art methods. △ Less

Submitted 21 April, 2018; v1 submitted 3 July, 2017; originally announced July 2017.

Minimization of fraction function penalty in compressed sensing

Authors: Haiyang Li, Qian Zhang, Angang Cui, Jigen Peng

Abstract: In the paper, we study the minimization problem of a non-convex sparsity promoting penalty function $$P_{a}(x)=\sum_{i=1}^{n}p_{a}(x_{i})=\sum_{i=1}^{n}\frac{a|x_{i}|}{1+a|x_{i}|}$$ in compressed sensing, which is called fraction function. Firstly, we discuss the equivalence of $\ell_{0}$ minimization and fraction function minimization. It is proved that there corresponds a constant $a^{**}>0$ suc… ▽ More In the paper, we study the minimization problem of a non-convex sparsity promoting penalty function $$P_{a}(x)=\sum_{i=1}^{n}p_{a}(x_{i})=\sum_{i=1}^{n}\frac{a|x_{i}|}{1+a|x_{i}|}$$ in compressed sensing, which is called fraction function. Firstly, we discuss the equivalence of $\ell_{0}$ minimization and fraction function minimization. It is proved that there corresponds a constant $a^{**}>0$ such that, whenever $a>a^{**}$, every solution to $(FP_{a})$ also solves $(P_{0})$, that the uniqueness of global minimizer of $(FP_{a})$ and its equivalence to $(P_{0})$ if the sensing matrix $A$ satisfies a restricted isometry property (RIP) and, last but the most important, that the optimal solution to the regularization problem $(FP_{a}^λ)$ also solves $(FP_{a})$ if the certain condition is satisfied, which is similar to the regularization problem in convex optimal theory. Secondly, we study the properties of the optimal solution to the regularization problem $(FP^λ_{a})$ including the first-order and the second optimality condition and the lower and upper bound of the absolute value for its nonzero entries. Finally, we derive the closed form representation of the optimal solution to the regularization problem ($FP_{a}^λ$) for all positive values of parameter $a$, and propose an iterative $FP$ thresholding algorithm to solve the regularization problem $(FP_{a}^λ)$. We also provide a series of experiments to assess performance of the $FP$ algorithm, and the experiment results show that, compared with soft thresholding algorithm and half thresholding algorithms, the $FP$ algorithm performs the best in sparse signal recovery with and without measurement noise. △ Less

Submitted 17 July, 2019; v1 submitted 17 May, 2017; originally announced May 2017.

Comments: 12 pages

Non-convex Fraction Function Penalty: Sparse Signals Recovered from Quasi-linear Systems

Authors: Angang Cui, Jigen Peng, Haiyang Li

Abstract: The goal of compressed sensing is to reconstruct a sparse signal under a few linear measurements far less than the dimension of the ambient space of the signal. However, many real-life applications in physics and biomedical sciences carry some strongly nonlinear structures, and the linear model is no longer suitable. Compared with the compressed sensing under the linear circumstance, this nonlinea… ▽ More The goal of compressed sensing is to reconstruct a sparse signal under a few linear measurements far less than the dimension of the ambient space of the signal. However, many real-life applications in physics and biomedical sciences carry some strongly nonlinear structures, and the linear model is no longer suitable. Compared with the compressed sensing under the linear circumstance, this nonlinear compressed sensing is much more difficult, in fact also NP-hard, combinatorial problem, because of the discrete and discontinuous nature of the $\ell_{0}$-norm and the nonlinearity. In order to get a convenience for sparse signal recovery, we set most of the nonlinear models have a smooth quasi-linear nature in this paper, and study a non-convex fraction function $ρ_{a}$ in this quasi-linear compressed sensing. We propose an iterative fraction thresholding algorithm to solve the regularization problem $(QP_{a}^λ)$ for all $a>0$. With the change of parameter $a>0$, our algorithm could get a promising result, which is one of the advantages for our algorithm compared with other algorithms. Numerical experiments show that our method performs much better compared with some state-of-art methods. △ Less

Submitted 26 August, 2017; v1 submitted 2 May, 2017; originally announced May 2017.

Exact recovery low-rank matrix via transformed affine matrix rank minimization

Authors: Angang Cui, Jigen Peng, Haiyang Li

Abstract: The goal of affine matrix rank minimization problem is to reconstruct a low-rank or approximately low-rank matrix under linear constraints. In general, this problem is combinatorial and NP-hard. In this paper, a nonconvex fraction function is studied to approximate the rank of a matrix and translate this NP-hard problem into a transformed affine matrix rank minimization problem. The equivalence be… ▽ More The goal of affine matrix rank minimization problem is to reconstruct a low-rank or approximately low-rank matrix under linear constraints. In general, this problem is combinatorial and NP-hard. In this paper, a nonconvex fraction function is studied to approximate the rank of a matrix and translate this NP-hard problem into a transformed affine matrix rank minimization problem. The equivalence between these two problems is established, and we proved that the uniqueness of the global minimizer of transformed affine matrix rank minimization problem also solves affine matrix rank minimization problem if some conditions are satisfied. Moreover, we also proved that the optimal solution to the transformed affine matrix rank minimization problem can be approximately obtained by solving its regularization problem for some proper smaller $λ>0$. Lastly, the DC algorithm is utilized to solve the regularization transformed affine matrix rank minimization problem and the numerical experiments on image inpainting problems show that our method performs effectively in recovering low-rank images compared with some state-of-art algorithms. △ Less

Submitted 19 June, 2018; v1 submitted 11 December, 2016; originally announced December 2016.

Comments: arXiv admin note: text overlap with arXiv:1611.07777

Affine matrix rank minimization problem via non-convex fraction function penalty

Authors: Angang Cui, Jigen Peng, Haiyang Li, Chengyi Zhang, Yongchao Yu

Abstract: Affine matrix rank minimization problem is a fundamental problem with a lot of important applications in many fields. It is well known that this problem is combinatorial and NP-hard in general. In this paper, a continuous promoting low rank non-convex fraction function is studied to replace the rank function in this NP-hard problem. Inspired by our former work in compressed sensing, an iterative s… ▽ More Affine matrix rank minimization problem is a fundamental problem with a lot of important applications in many fields. It is well known that this problem is combinatorial and NP-hard in general. In this paper, a continuous promoting low rank non-convex fraction function is studied to replace the rank function in this NP-hard problem. Inspired by our former work in compressed sensing, an iterative singular value thresholding algorithm is proposed to solve the regularization transformed affine matrix rank minimization problem. For different $a>0$, we could get a much better result by adjusting the different value of $a$, which is one of the advantages for the iterative singular value thresholding algorithm compared with some state-of-art methods. Some convergence results are established and numerical experiments show that this thresholding algorithm is feasible for solving the regularization transformed affine matrix rank minimization problem. Moreover, we proved that the value of the regularization parameter $λ>0$ can not be chosen too large. Indeed, there exists $\barλ>0$ such that the optimal solution of the regularization transformed affine matrix rank minimization problem is equal to zero for any $λ>\barλ$. Numerical experiments on matrix completion problems show that our method performs powerful in finding a low-rank matrix and the numerical experiments about image inpainting problems show that our algorithm has better performances than some state-of-art methods. △ Less

Submitted 30 April, 2017; v1 submitted 23 November, 2016; originally announced November 2016.

Neural Contextual Conversation Learning with Labeled Question-Answering Pairs

Authors: Kun Xiong, Anqi Cui, Zefeng Zhang, Ming Li

Abstract: Neural conversational models tend to produce generic or safe responses in different contexts, e.g., reply \textit{"Of course"} to narrative statements or \textit{"I don't know"} to questions. In this paper, we propose an end-to-end approach to avoid such problem in neural generative models. Additional memory mechanisms have been introduced to standard sequence-to-sequence (seq2seq) models, so that… ▽ More Neural conversational models tend to produce generic or safe responses in different contexts, e.g., reply \textit{"Of course"} to narrative statements or \textit{"I don't know"} to questions. In this paper, we propose an end-to-end approach to avoid such problem in neural generative models. Additional memory mechanisms have been introduced to standard sequence-to-sequence (seq2seq) models, so that context can be considered while generating sentences. Three seq2seq models, which memorize a fix-sized contextual vector from hidden input, hidden input/output and a gated contextual attention structure respectively, have been trained and tested on a dataset of labeled question-answering pairs in Chinese. The model with contextual attention outperforms others including the state-of-the-art seq2seq models on perplexity test. The novel contextual model generates diverse and robust responses, and is able to carry out conversations on a wide range of topics appropriately. △ Less

Submitted 19 July, 2016; originally announced July 2016.

Efficient allocation of heterogeneous response times in information spreading process

Authors: Ai-Xiang Cui, Wei Wang, Ming Tang, Yan Fu, Xiaoming Liang, Younghae Do

Abstract: Recently, the impacts of spatiotemporal heterogeneities of human activities on spreading dynamics have attracted extensive attention. In this paper, to study heterogeneous response times on information spreading, we focus on the susceptible-infected spreading dynamics with adjustable power-law response time distribution based on uncorrelated scale-free networks. We find that the stronger the heter… ▽ More Recently, the impacts of spatiotemporal heterogeneities of human activities on spreading dynamics have attracted extensive attention. In this paper, to study heterogeneous response times on information spreading, we focus on the susceptible-infected spreading dynamics with adjustable power-law response time distribution based on uncorrelated scale-free networks. We find that the stronger the heterogeneity of response times is, the faster the information spreading is in the early and middle stages. Following a given heterogeneity, the procedure of reducing the correlation between the response times and degrees of individuals can also accelerate the spreading dynamics in the early and middle stages. However, the dynamics in the late stage is slightly more complicated, and there is an optimal value of the full prevalence time changing with the heterogeneity of response times and the response time-degree correlation, respectively. The optimal phenomena results from the efficient allocation of heterogeneous response times. △ Less

Submitted 22 January, 2014; originally announced January 2014.

Comments: 8 pages, 6 figures

Strong ties promote the epidemic prevalence in susceptible-infected-susceptible spreading dynamics

Authors: Ai-Xiang Cui, Zimo Yang, Tao Zhou

Abstract: Understanding spreading dynamics will benefit society as a whole in better preventing and controlling diseases, as well as facilitating the socially responsible information while depressing destructive rumors. In network-based spreading dynamics, edges with different weights may play far different roles: a friend from afar usually brings novel stories, and an intimate relationship is highly risky… ▽ More Understanding spreading dynamics will benefit society as a whole in better preventing and controlling diseases, as well as facilitating the socially responsible information while depressing destructive rumors. In network-based spreading dynamics, edges with different weights may play far different roles: a friend from afar usually brings novel stories, and an intimate relationship is highly risky for a flu epidemic. In this article, we propose a weighted susceptible-infected-susceptible model on complex networks, where the weight of an edge is defined by the topological proximity of the two associated nodes. Each infected individual is allowed to select limited number of neighbors to contact, and a tunable parameter is introduced to control the preference to contact through high-weight or low-weight edges. Experimental results on six real networks show that the epidemic prevalence can be largely promoted when strong ties are favored in the spreading process. By comparing with two statistical null models respectively with randomized topology and randomly redistributed weights, we show that the distribution pattern of weights, rather than the topology, mainly contributes to the experimental observations. Further analysis suggests that the weight-weight correlation strongly affects the results: high-weight edges are more significant in kee** high epidemic prevalence when the weight-weight correlation is present. △ Less

Submitted 22 November, 2013; originally announced November 2013.

Comments: 7 pages, 6 figures, and 1 Table. arXiv admin note: substantial text overlap with arXiv:1204.0100

Journal ref: PLoS ONE 9(12): e113457 (2014)

A Training effect on electrical properties in nanoscale BiFeO$_3$

Authors: Sudipta Goswami, Dipten Bhattacharya, Wuxia Li, Ajuan Cui, QianQing Jiang, Chang-zhi Gu

Abstract: We report our observation of the training effect on dc electrical properties in a nanochain of BiFeO$_3$ as a result of large scale migration of defects under combined influence of electric field and Joule heating. We show that an optimum number of cycles of electric field within the range zero to $\sim$1.0 MV/cm across a temperature range 80-300 K helps in reaching the stable state via a glass-tr… ▽ More We report our observation of the training effect on dc electrical properties in a nanochain of BiFeO$_3$ as a result of large scale migration of defects under combined influence of electric field and Joule heating. We show that an optimum number of cycles of electric field within the range zero to $\sim$1.0 MV/cm across a temperature range 80-300 K helps in reaching the stable state via a glass-transition-like process in the defect structure. Further treatment does not give rise to any substantial modification. We conclude that such a training effect is ubiquitous in pristine nanowires or chains of oxides and needs to be addressed for applications in nanoelectronic devices. △ Less

Submitted 8 April, 2013; originally announced April 2013.

Comments: 11 pages with 5 figures; published in Nanotechnology

Journal ref: Nanotechnology 24, 135705 (2013)

Emergence of scale-free close-knit friendship structure in online social networks

Authors: Ai-xiang Cui, Zi-ke Zhang, Ming Tang, Pak Ming Hui, Yan Fu

Abstract: Despite the structural properties of online social networks have attracted much attention, the properties of the close-knit friendship structures remain an important question. Here, we mainly focus on how these mesoscale structures are affected by the local and global structural properties. Analyzing the data of four large-scale online social networks reveals several common structural properties.… ▽ More Despite the structural properties of online social networks have attracted much attention, the properties of the close-knit friendship structures remain an important question. Here, we mainly focus on how these mesoscale structures are affected by the local and global structural properties. Analyzing the data of four large-scale online social networks reveals several common structural properties. It is found that not only the local structures given by the indegree, outdegree, and reciprocal degree distributions follow a similar scaling behavior, the mesoscale structures represented by the distributions of close-knit friendship structures also exhibit a similar scaling law. The degree correlation is very weak over a wide range of the degrees. We propose a simple directed network model that captures the observed properties. The model incorporates two mechanisms: reciprocation and preferential attachment. Through rate equation analysis of our model, the local-scale and mesoscale structural properties are derived. In the local-scale, the same scaling behavior of indegree and outdegree distributions stems from indegree and outdegree of nodes both growing as the same function of the introduction time, and the reciprocal degree distribution also shows the same power-law due to the linear relationship between the reciprocal degree and in/outdegree of nodes. In the mesoscale, the distributions of four closed triples representing close-knit friendship structures are found to exhibit identical power-laws, a behavior attributed to the negligible degree correlations. Intriguingly, all the power-law exponents of the distributions in the local-scale and mesoscale depend only on one global parameter -- the mean in/outdegree, while both the mean in/outdegree and the reciprocity together determine the ratio of the reciprocal degree of a node to its in/outdegree. △ Less

Submitted 16 December, 2012; v1 submitted 11 May, 2012; originally announced May 2012.

Comments: 48 pages, 34 figures

Slow dynamics of Zero Range Process in the Framework of Traps Model

Authors: Kai Qi, Ming Tang, Aixiang Cui, Yan Fu

Abstract: The relaxation dynamics of zero range process (ZRP) has always been an interesting problem. In this study, we set up the relationship between ZRP and traps model, and investigate the slow dynamics of ZRP in the framework of traps model. Through statistical quantities such as the average rest time, the particle distribution, the two-time correlation function and the average escape time, we find tha… ▽ More The relaxation dynamics of zero range process (ZRP) has always been an interesting problem. In this study, we set up the relationship between ZRP and traps model, and investigate the slow dynamics of ZRP in the framework of traps model. Through statistical quantities such as the average rest time, the particle distribution, the two-time correlation function and the average escape time, we find that the particle interaction, especially the resulted condensation, can significantly influence the dynamics. In the stationary state, both the average rest time and the average escape time caused by the attraction among particles are obtained analytically. In the transient state, a hierarchical nature of the aging dynamics is revealed by both simulations and scaling analysis. Moreover, by comparing the particle diffusion in both the transient state and the stationary state, we find that the closer ZRP systems approach the stationary state, the more slowly particles diffuse. △ Less

Submitted 3 April, 2012; originally announced April 2012.

Comments: 5 pages, 4 figures

MSC Class: 82C05

Roles of Ties in Spreading

Authors: Ai-Xiang Cui, Zimo Yang, Tao Zhou

Abstract: Background: Controlling global epidemics in the real world and accelerating information propagation in the artificial world are of great significance, which have activated an upsurge in the studies on networked spreading dynamics. Lots of efforts have been made to understand the impacts of macroscopic statistics (e.g., degree distribution and average distance) and mesoscopic structures (e.g., comm… ▽ More Background: Controlling global epidemics in the real world and accelerating information propagation in the artificial world are of great significance, which have activated an upsurge in the studies on networked spreading dynamics. Lots of efforts have been made to understand the impacts of macroscopic statistics (e.g., degree distribution and average distance) and mesoscopic structures (e.g., communities and rich clubs) on spreading processes while the microscopic elements are less concerned. In particular, roles of ties are not yet clear to the academic community. Methodology/Principle Findings: Every edges is stamped by its strength that is defined solely based on the local topology. According to a weighted susceptible-infected-susceptible model, the steady-state infected density and spreading speed are respectively optimized by adjusting the relationship between edge's strength and spreading ability. Experiments on six real networks show that the infected density is increased when strong ties are favored in the spreading, while the speed is enhanced when weak ties are favored. Significance of these findings is further demonstrated by comparing with a null model. Conclusions/Significance: Experimental results indicate that strong and weak ties play distinguishable roles in spreading dynamics: the former enlarge the infected density while the latter fasten the process. The proposed method provides a quantitative way to reveal the qualitatively different roles of ties, which could find applications in analyzing many networked dynamical processes with multiple performance indices, such as synchronizability and converging time in synchronization and throughput and delivering time in transportation. △ Less

Submitted 31 March, 2012; originally announced April 2012.

Comments: 8 pages, 4 figures, 1 table

Impact of Heterogeneous Human Activities on Epidemic Spreading

Authors: Zimo Yang, Ai-Xiang Cui, Tao Zhou

Abstract: Recent empirical observations suggest a heterogeneous nature of human activities. The heavy-tailed inter-event time distribution at population level is well accepted, while whether the individual acts in a heterogeneous way is still under debate. Motivated by the impact of temporal heterogeneity of human activities on epidemic spreading, this paper studies the susceptible-infected model on a fully… ▽ More Recent empirical observations suggest a heterogeneous nature of human activities. The heavy-tailed inter-event time distribution at population level is well accepted, while whether the individual acts in a heterogeneous way is still under debate. Motivated by the impact of temporal heterogeneity of human activities on epidemic spreading, this paper studies the susceptible-infected model on a fully mixed population, where each individual acts in a completely homogeneous way but different individuals have different mean activities. Extensive simulations show that the heterogeneity of activities at population level remarkably affects the speed of spreading, even though each individual behaves regularly. Further more, the spreading speed of this model is more sensitive to the change of system heterogeneity compared with the model consisted of individuals acting with heavy-tailed inter-event time distribution. This work refines our understanding of the impact of heterogeneous human activities on epidemic spreading. △ Less

Submitted 17 June, 2011; originally announced June 2011.

Comments: 11 pages, 4 figures

Journal ref: Physica A 390 (2011) 4543-4548

On canonically fibred algebraic 3-folds - some new examples

Authors: Meng Chen, Aoxiang Cui

Abstract: This note aims to improve known numerical bounds proved earlier by Chen \cite{PAMS} and Chen-Hacon \cite{Chen-Hacon} and to present some new examples of smooth minimal 3-folds canonically fibred by surfaces (resp. curves) of geometric genus as large as 19 (resp. 13). As an interesting by-product, we present a new class of general type surfaces which are canonically fibred by curves of genus 13. This note aims to improve known numerical bounds proved earlier by Chen \cite{PAMS} and Chen-Hacon \cite{Chen-Hacon} and to present some new examples of smooth minimal 3-folds canonically fibred by surfaces (resp. curves) of geometric genus as large as 19 (resp. 13). As an interesting by-product, we present a new class of general type surfaces which are canonically fibred by curves of genus 13. △ Less

Submitted 21 April, 2011; v1 submitted 21 May, 2010; originally announced May 2010.

Comments: 20 pages, To appear in Pure and Applied Mathematics Quarterly (Special volume for Eckart Viehweg)

Journal ref: Pure and Applied Mathematics Quarterly 7(2011), 1105--1128