Showing 1–8 of 8 results for author: Ramamurthy, A

Deep Generative Model for Efficient 3D Airfoil Parameterization and Generation

Authors: Wei Chen, Arun Ramamurthy

Abstract: In aerodynamic shape optimization, the convergence and computational cost are greatly affected by the representation capacity and compactness of the design space. Previous research has demonstrated that using a deep generative model to parameterize two-dimensional (2D) airfoils achieves high representation capacity/compactness, which significantly benefits shape optimization. In this paper, we pro… ▽ More In aerodynamic shape optimization, the convergence and computational cost are greatly affected by the representation capacity and compactness of the design space. Previous research has demonstrated that using a deep generative model to parameterize two-dimensional (2D) airfoils achieves high representation capacity/compactness, which significantly benefits shape optimization. In this paper, we propose a deep generative model, Free-Form Deformation Generative Adversarial Networks (FFD-GAN), that provides an efficient parameterization for three-dimensional (3D) aerodynamic/hydrodynamic shapes like aircraft wings, turbine blades, car bodies, and hulls. The learned model maps a compact set of design variables to 3D surface points representing the shape. We ensure the surface smoothness and continuity of generated geometries by incorporating an FFD layer into the generative model. We demonstrate FFD-GAN's performance using a wing shape design example. The results show that FFD-GAN can generate realistic designs and form a reasonable parameterization. We further demonstrate FFD-GAN's high representation compactness and capacity by testing its design space coverage, the feasibility ratio of the design space, and its performance in design optimization. We demonstrate that over 94% feasibility ratio is achieved among wings randomly generated by the FFD-GAN, while FFD and B-spline only achieve less than 31%. We also show that the FFD-GAN leads to an order of magnitude faster convergence in a wing shape optimization problem, compared to the FFD and the B-spline parameterizations. △ Less

Submitted 7 January, 2021; originally announced January 2021.

Answering Any-hop Open-domain Questions with Iterative Document Reranking

Authors: ** Nie, Yuyu Zhang, Arun Ramamurthy, Le Song

Abstract: Existing approaches for open-domain question answering (QA) are typically designed for questions that require either single-hop or multi-hop reasoning, which make strong assumptions of the complexity of questions to be answered. Also, multi-step document retrieval often incurs higher number of relevant but non-supporting documents, which dampens the downstream noise-sensitive reader module for ans… ▽ More Existing approaches for open-domain question answering (QA) are typically designed for questions that require either single-hop or multi-hop reasoning, which make strong assumptions of the complexity of questions to be answered. Also, multi-step document retrieval often incurs higher number of relevant but non-supporting documents, which dampens the downstream noise-sensitive reader module for answer extraction. To address these challenges, we propose a unified QA framework to answer any-hop open-domain questions, which iteratively retrieves, reranks and filters documents, and adaptively determines when to stop the retrieval process. To improve the retrieval accuracy, we propose a graph-based reranking model that perform multi-document interaction as the core of our iterative reranking framework. Our method consistently achieves performance comparable to or better than the state-of-the-art on both single-hop and multi-hop open-domain QA datasets, including Natural Questions Open, SQuAD Open, and HotpotQA. △ Less

Submitted 24 May, 2021; v1 submitted 16 September, 2020; originally announced September 2020.

Comments: Accepted by SIGIR 2021

Conditional Neural Architecture Search

Authors: Sheng-Chun Kao, Arun Ramamurthy, Reed Williams, Tushar Krishna

Abstract: Designing resource-efficient Deep Neural Networks (DNNs) is critical to deploy deep learning solutions over edge platforms due to diverse performance, power, and memory budgets. Unfortunately, it is often the case a well-trained ML model does not fit to the constraint of deploying edge platforms, causing a long iteration of model reduction and retraining process. Moreover, a ML model optimized for… ▽ More Designing resource-efficient Deep Neural Networks (DNNs) is critical to deploy deep learning solutions over edge platforms due to diverse performance, power, and memory budgets. Unfortunately, it is often the case a well-trained ML model does not fit to the constraint of deploying edge platforms, causing a long iteration of model reduction and retraining process. Moreover, a ML model optimized for platform-A often may not be suitable when we deploy it on another platform-B, causing another iteration of model retraining. We propose a conditional neural architecture search method using GAN, which produces feasible ML models for different platforms. We present a new workflow to generate constraint-optimized DNN models. This is the first work of bringing in condition and adversarial technique into Neural Architecture Search domain. We verify the method with regression problems and classification on CIFAR-10. The proposed workflow can successfully generate resource-optimized MLP or CNN-based networks. △ Less

Submitted 6 June, 2020; originally announced June 2020.

Generative Design of Hardware-aware DNNs

Authors: Sheng-Chun Kao, Arun Ramamurthy, Tushar Krishna

Abstract: To efficiently run DNNs on the edge/cloud, many new DNN inference accelerators are being designed and deployed frequently. To enhance the resource efficiency of DNNs, model quantization is a widely-used approach. However, different accelerator/HW has different resources leading to the need for specialized quantization strategy of each HW. Moreover, using the same quantization for every layer may b… ▽ More To efficiently run DNNs on the edge/cloud, many new DNN inference accelerators are being designed and deployed frequently. To enhance the resource efficiency of DNNs, model quantization is a widely-used approach. However, different accelerator/HW has different resources leading to the need for specialized quantization strategy of each HW. Moreover, using the same quantization for every layer may be sub-optimal, increasing the designspace of possible quantization choices. This makes manual-tuning infeasible. Recent work in automatically determining quantization for each layer is driven by optimization methods such as reinforcement learning. However, these approaches need re-training the RL for every new HW platform. We propose a new way for autonomous quantization and HW-aware tuning. We propose a generative model, AQGAN, which takes a target accuracy as the condition and generates a suite of quantization configurations. With the conditional generative model, the user can autonomously generate different configurations with different targets in inference time. Moreover, we propose a simplified HW-tuning flow, which uses the generative model to generate proposals and execute simple selection based on the HW resource budget, whose process is fast and interactive. We evaluate our model on five of the widely-used efficient models on the ImageNet dataset. We compare with existing uniform quantization and state-of-the-art autonomous quantization methods. Our generative model shows competitive achieved accuracy, however, with around two degrees less search cost for each design point. Our generative model shows the generated quantization configuration can lead to less than 3.5% error across all experiments. △ Less

Submitted 12 July, 2020; v1 submitted 6 June, 2020; originally announced June 2020.

DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding

Authors: Yuyu Zhang, ** Nie, Xiubo Geng, Arun Ramamurthy, Le Song, Daxin Jiang

Abstract: Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT. State-of-the-art approaches typically follow the "retrieve and read" pipeline and employ BERT-based reranker to filter retrieved documents before feeding them into the reader module. The BERT retriever takes as input the concatenation of question and each… ▽ More Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT. State-of-the-art approaches typically follow the "retrieve and read" pipeline and employ BERT-based reranker to filter retrieved documents before feeding them into the reader module. The BERT retriever takes as input the concatenation of question and each retrieved document. Despite the success of these approaches in terms of QA accuracy, due to the concatenation, they can barely handle high-throughput of incoming questions each with a large collection of retrieved documents. To address the efficiency problem, we propose DC-BERT, a decoupled contextual encoding framework that has dual BERT models: an online BERT which encodes the question only once, and an offline BERT which pre-encodes all the documents and caches their encodings. On SQuAD Open and Natural Questions Open datasets, DC-BERT achieves 10x speedup on document retrieval, while retaining most (about 98%) of the QA performance compared to state-of-the-art approaches for open-domain question answering. △ Less

Submitted 28 February, 2020; originally announced February 2020.

Efficient Probabilistic Logic Reasoning with Graph Neural Networks

Authors: Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Ramamurthy, Bo Li, Yuan Qi, Le Song

Abstract: Markov Logic Networks (MLNs), which elegantly combine logic rules and probabilistic graphical models, can be used to address many knowledge graph problems. However, inference in MLN is computationally intensive, making the industrial-scale application of MLN very difficult. In recent years, graph neural networks (GNNs) have emerged as efficient and effective tools for large-scale graph problems. N… ▽ More Markov Logic Networks (MLNs), which elegantly combine logic rules and probabilistic graphical models, can be used to address many knowledge graph problems. However, inference in MLN is computationally intensive, making the industrial-scale application of MLN very difficult. In recent years, graph neural networks (GNNs) have emerged as efficient and effective tools for large-scale graph problems. Nevertheless, GNNs do not explicitly incorporate prior logic rules into the models, and may require many labeled examples for a target task. In this paper, we explore the combination of MLNs and GNNs, and use graph neural networks for variational inference in MLN. We propose a GNN variant, named ExpressGNN, which strikes a nice balance between the representation power and the simplicity of the model. Our extensive experiments on several benchmark datasets demonstrate that ExpressGNN leads to effective and efficient probabilistic logic reasoning. △ Less

Submitted 3 February, 2020; v1 submitted 29 January, 2020; originally announced January 2020.

Can Graph Neural Networks Help Logic Reasoning?

Authors: Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Ramamurthy, Bo Li, Yuan Qi, Le Song

Abstract: Effectively combining logic reasoning and probabilistic inference has been a long-standing goal of machine learning: the former has the ability to generalize with small training data, while the latter provides a principled framework for dealing with noisy data. However, existing methods for combining the best of both worlds are typically computationally intensive. In this paper, we focus on Markov… ▽ More Effectively combining logic reasoning and probabilistic inference has been a long-standing goal of machine learning: the former has the ability to generalize with small training data, while the latter provides a principled framework for dealing with noisy data. However, existing methods for combining the best of both worlds are typically computationally intensive. In this paper, we focus on Markov Logic Networks and explore the use of graph neural networks (GNNs) for representing probabilistic logic inference. It is revealed from our analysis that the representation power of GNN alone is not enough for such a task. We instead propose a more expressive variant, called ExpressGNN, which can perform effective probabilistic logic inference while being able to scale to a large number of entities. We demonstrate by several benchmark datasets that ExpressGNN has the potential to advance probabilistic logic reasoning to the next stage. △ Less

Submitted 20 September, 2019; v1 submitted 5 June, 2019; originally announced June 2019.

On Improving Capacity of Full-Duplex Small Cells with D2D

Authors: Arun Ramamurthy, Vanlin Sathya, Shrestha Ghosh, Antony Franklin, Bheemarjuna Reddy Tamma

Abstract: The recent developments in full duplex (FD) communication promise doubling the capacity of cellular networks using self interference cancellation (SIC) techniques. FD small cells with device-to-device (D2D) communication links could achieve the expected capacity of the future cellular networks (5G). In this work, we consider joint scheduling and dynamic power algorithm (DPA) for a single cell FD s… ▽ More The recent developments in full duplex (FD) communication promise doubling the capacity of cellular networks using self interference cancellation (SIC) techniques. FD small cells with device-to-device (D2D) communication links could achieve the expected capacity of the future cellular networks (5G). In this work, we consider joint scheduling and dynamic power algorithm (DPA) for a single cell FD small cell network with D2D links (D2DLs). We formulate the optimal user selection and power control as a non-linear programming (NLP) optimization problem to get the optimal user scheduling and transmission power in a given TTI. Our numerical results show that using DPA gives better overall throughput performance than full power transmission algorithm (FPA). Also, simultaneous transmissions (combination of uplink (UL), downlink (DL), and D2D occur 80% of the time thereby increasing the spectral efficiency and network capacity. △ Less

Submitted 23 June, 2016; originally announced June 2016.

Comments: Submitted to IEEE Globecom Conference 2016