-
DeepJEB: 3D Deep Learning-based Synthetic Jet Engine Bracket Dataset
Authors:
Seongjun Hong,
Yongmin Kwon,
Dongju Shin,
Jangseop Park,
Namwoo Kang
Abstract:
Recent advancements in artificial intelligence (AI) have significantly influenced various fields, including mechanical engineering. Nonetheless, the development of high-quality, diverse datasets for structural analysis still needs to be improved. Although traditional datasets, such as simulated jet engine bracket dataset, are useful, they are constrained by a small number of samples, which must be…
▽ More
Recent advancements in artificial intelligence (AI) have significantly influenced various fields, including mechanical engineering. Nonetheless, the development of high-quality, diverse datasets for structural analysis still needs to be improved. Although traditional datasets, such as simulated jet engine bracket dataset, are useful, they are constrained by a small number of samples, which must be improved for develo** robust data-driven surrogate models. This study presents the DeepJEB dataset, which has been created using deep generative models and automated engineering simulation pipelines, to overcome these challenges. Moreover, this study provides comprehensive 3D geometries and their corresponding structural analysis data.
Key experiments validated the effectiveness of the DeepJEB dataset, demonstrating significant improvements in the prediction accuracy and reliability of surrogate models trained on this data. The enhanced dataset showed a broader design space and better generalization capabilities than traditional datasets. These findings highlight the potential of DeepJEB as a benchmark dataset for develo** reliable surrogate models in structural engineering. The DeepJEB dataset supports advanced modeling techniques, such as graph neural networks (GNNs) and high-dimensional convolutional networks (CNNs), leveraging node-level field data for precise predictions. This dataset is set to drive innovation in engineering design applications, enabling more accurate and efficient structural performance predictions. The DeepJEB dataset is publicly accessible at: https://www.narnia.ai/dataset
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
Enhancing Graph U-Nets for Mesh-Agnostic Spatio-Temporal Flow Prediction
Authors:
Sunwoong Yang,
Ricardo Vinuesa,
Namwoo Kang
Abstract:
This study aims to overcome the conventional deep-learning approaches based on convolutional neural networks, whose applicability to complex geometries and unstructured meshes is limited due to their inherent mesh dependency. We propose novel approaches to improve mesh-agnostic spatio-temporal prediction of transient flow fields using graph U-Nets, enabling accurate prediction on diverse mesh conf…
▽ More
This study aims to overcome the conventional deep-learning approaches based on convolutional neural networks, whose applicability to complex geometries and unstructured meshes is limited due to their inherent mesh dependency. We propose novel approaches to improve mesh-agnostic spatio-temporal prediction of transient flow fields using graph U-Nets, enabling accurate prediction on diverse mesh configurations. Key enhancements to the graph U-Net architecture, including the Gaussian mixture model convolutional operator and noise injection approaches, provide increased flexibility in modeling node dynamics: the former reduces prediction error by 95\% compared to conventional convolutional operators, while the latter improves long-term prediction robustness, resulting in an error reduction of 86\%. We also investigate transductive and inductive-learning perspectives of graph U-Nets with proposed improvements. In the transductive setting, they effectively predict quantities for unseen nodes within the trained graph. In the inductive setting, they successfully perform in mesh scenarios with different vortex-shedding periods, showing 98\% improvement in predicting the future flow fields compared to a model trained without the inductive settings. It is found that graph U-Nets without pooling operations, i.e. without reducing and restoring the node dimensionality of the graph data, perform better in inductive settings due to their ability to learn from the detailed structure of each graph. Meanwhile, we also discover that the choice of normalization technique significantly impacts graph U-Net performance.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Bayesian Mesh Optimization for Graph Neural Networks to Enhance Engineering Performance Prediction
Authors:
Jangseop Park,
Namwoo Kang
Abstract:
In engineering design, surrogate models are widely employed to replace computationally expensive simulations by leveraging design variables and geometric parameters from computer-aided design (CAD) models. However, these models often lose critical information when simplified to lower dimensions and face challenges in parameter definition, especially with the complex 3D shapes commonly found in ind…
▽ More
In engineering design, surrogate models are widely employed to replace computationally expensive simulations by leveraging design variables and geometric parameters from computer-aided design (CAD) models. However, these models often lose critical information when simplified to lower dimensions and face challenges in parameter definition, especially with the complex 3D shapes commonly found in industrial datasets. To address these limitations, we propose a Bayesian graph neural network (GNN) framework for a 3D deep-learning-based surrogate model that predicts engineering performance by directly learning geometric features from CAD using mesh representation. Our framework determines the optimal size of mesh elements through Bayesian optimization, resulting in a high-accuracy surrogate model. Additionally, it effectively handles the irregular and complex structures of 3D CADs, which differ significantly from the regular and uniform pixel structures of 2D images typically used in deep learning. Experimental results demonstrate that the quality of the mesh significantly impacts the prediction accuracy of the surrogate model, with an optimally sized mesh achieving superior performance. We compare the performance of models based on various 3D representations such as voxel, point cloud, and graph, and evaluate the computational costs of Monte Carlo simulation and Bayesian optimization methods to find the optimal mesh size. We anticipate that our proposed framework has the potential to be applied to mesh-based simulations across various engineering fields, leveraging physics-based information commonly used in computer-aided engineering.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
FNCC: Fast Notification Congestion Control in Data Center Networks
Authors:
**g Xu,
Zhan Wang,
Fan Yang,
Ning Kang,
Zhenlong Ma,
Guojun Yuan,
Guangming Tan,
Ninghui Sun
Abstract:
Congestion control plays a pivotal role in large-scale data centers, facilitating ultra-low latency, high bandwidth, and optimal utilization. Even with the deployment of data center congestion control mechanisms such as DCQCN and HPCC, these algorithms often respond to congestion sluggishly. This sluggishness is primarily due to the slow notification of congestion. It takes almost one round-trip t…
▽ More
Congestion control plays a pivotal role in large-scale data centers, facilitating ultra-low latency, high bandwidth, and optimal utilization. Even with the deployment of data center congestion control mechanisms such as DCQCN and HPCC, these algorithms often respond to congestion sluggishly. This sluggishness is primarily due to the slow notification of congestion. It takes almost one round-trip time (RTT) for the congestion information to reach the sender. In this paper, we introduce the Fast Notification Congestion Control (FNCC) mechanism, which achieves sub-RTT notification. FNCC leverages the acknowledgment packet (ACK) from the return path to carry in-network telemetry (INT) information of the request path, offering the sender more timely and accurate INT. To further accelerate the responsiveness of last-hop congestion control, we propose that the receiver notifies the sender of the number of concurrent congested flows, which can be used to adjust the congested flows to a fair rate quickly. Our experimental results demonstrate that FNCC reduces flow completion time by 27.4% and 88.9% compared to HPCC and DCQCN, respectively. Moreover, FNCC triggers minimal pause frames and maintains high utilization even at 400Gbps.
△ Less
Submitted 26 May, 2024; v1 submitted 13 May, 2024;
originally announced May 2024.
-
Learning Design Preferences through Design Feature Extraction and Weighted Ensemble
Authors:
Dongju Shin,
Sunghee Lee,
Namwoo Kang
Abstract:
Design is a factor that plays an important role in consumer purchase decisions. As the need for understanding and predicting various preferences for each customer increases along with the importance of mass customization, predicting individual design preferences has become a critical factor in product development. However, current methods for predicting design preferences have some limitations. Pr…
▽ More
Design is a factor that plays an important role in consumer purchase decisions. As the need for understanding and predicting various preferences for each customer increases along with the importance of mass customization, predicting individual design preferences has become a critical factor in product development. However, current methods for predicting design preferences have some limitations. Product design involves a vast amount of high-dimensional information, and personal design preference is a complex and heterogeneous area of emotion unique to each individual. To address these challenges, we propose an approach that utilizes dimensionality reduction model to transform design samples into low-dimensional feature vectors, enabling us to extract the key representational features of each design. For preference prediction models using feature vectors, by referring to the design preference tendencies of others, we can predict the individual-level design preferences more accurately. Our proposed framework overcomes the limitations of traditional methods to determine design preferences, allowing us to accurately identify design features and predict individual preferences for specific products. Through this framework, we can improve the effectiveness of product development and create personalized product recommendations that cater to the unique needs of each consumer.
△ Less
Submitted 12 May, 2024;
originally announced May 2024.
-
Deep Generative Design for Mass Production
Authors:
Jihoon Kim,
Yongmin Kwon,
Namwoo Kang
Abstract:
Generative Design (GD) has evolved as a transformative design approach, employing advanced algorithms and AI to create diverse and innovative solutions beyond traditional constraints. Despite its success, GD faces significant challenges regarding the manufacturability of complex designs, often necessitating extensive manual modifications due to limitations in standard manufacturing processes and t…
▽ More
Generative Design (GD) has evolved as a transformative design approach, employing advanced algorithms and AI to create diverse and innovative solutions beyond traditional constraints. Despite its success, GD faces significant challenges regarding the manufacturability of complex designs, often necessitating extensive manual modifications due to limitations in standard manufacturing processes and the reliance on additive manufacturing, which is not ideal for mass production. Our research introduces an innovative framework addressing these manufacturability concerns by integrating constraints pertinent to die casting and injection molding into GD, through the utilization of 2D depth images. This method simplifies intricate 3D geometries into manufacturable profiles, removing unfeasible features such as non-manufacturable overhangs and allowing for the direct consideration of essential manufacturing aspects like thickness and rib design. Consequently, designs previously unsuitable for mass production are transformed into viable solutions. We further enhance this approach by adopting an advanced 2D generative model, which offer a more efficient alternative to traditional 3D shape generation methods. Our results substantiate the efficacy of this framework, demonstrating the production of innovative, and, importantly, manufacturable designs. This shift towards integrating practical manufacturing considerations into GD represents a pivotal advancement, transitioning from purely inspirational concepts to actionable, production-ready solutions. Our findings underscore usefulness and potential of GD for broader industry adoption, marking a significant step forward in aligning GD with the demands of manufacturing challenges.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
Deep Generative Model-based Synthesis of Four-bar Linkage Mechanisms with Target Conditions
Authors:
Sumin Lee,
Jihoon Kim,
Namwoo Kang
Abstract:
Mechanisms are essential components designed to perform specific tasks in various mechanical systems. However, designing a mechanism that satisfies certain kinematic or quasi-static requirements is a challenging task. The kinematic requirements may include the workspace of a mechanism, while the quasi-static requirements of a mechanism may include its torque transmission, which refers to the abili…
▽ More
Mechanisms are essential components designed to perform specific tasks in various mechanical systems. However, designing a mechanism that satisfies certain kinematic or quasi-static requirements is a challenging task. The kinematic requirements may include the workspace of a mechanism, while the quasi-static requirements of a mechanism may include its torque transmission, which refers to the ability of the mechanism to transfer power and torque effectively. In this paper, we propose a deep learning-based generative model for generating multiple crank-rocker four-bar linkage mechanisms that satisfy both the kinematic and quasi-static requirements aforementioned. The proposed model is based on a conditional generative adversarial network (cGAN) with modifications for mechanism synthesis, which is trained to learn the relationship between the requirements of a mechanism with respect to linkage lengths. The results demonstrate that the proposed model successfully generates multiple distinct mechanisms that satisfy specific kinematic and quasi-static requirements. To evaluate the novelty of our approach, we provide a comparison of the samples synthesized by the proposed cGAN, traditional cVAE and NSGA-II. Our approach has several advantages over traditional design methods. It enables designers to efficiently generate multiple diverse and feasible design candidates while exploring a large design space. Also, the proposed model considers both the kinematic and quasi-static requirements, which can lead to more efficient and effective mechanisms for real-world use, making it a promising tool for linkage mechanism design.
△ Less
Submitted 21 February, 2024;
originally announced February 2024.
-
Multi-objective Generative Design Framework and Realization for Quasi-serial Manipulator: Considering Kinematic and Dynamic Performance
Authors:
Sumin Lee,
Sunwoong Yang,
Namwoo Kang
Abstract:
This paper proposes a framework that optimizes the linkage mechanism of the quasi-serial manipulator for target tasks. This process is explained through a case study of 2-degree-of-freedom linkage mechanisms, which significantly affect the workspace of the quasi-serial manipulator. First, a vast quasi-serial mechanism is generated with a workspace satisfying a target task and it converts it into a…
▽ More
This paper proposes a framework that optimizes the linkage mechanism of the quasi-serial manipulator for target tasks. This process is explained through a case study of 2-degree-of-freedom linkage mechanisms, which significantly affect the workspace of the quasi-serial manipulator. First, a vast quasi-serial mechanism is generated with a workspace satisfying a target task and it converts it into a 3D CAD model. Then, the workspace and required torque performance of each mechanism are evaluated through kinematic and dynamic analysis. A deep learning-based surrogate model is leveraged to efficiently predict mechanisms and performance during the optimization process. After model training, a multi-objective optimization problem is formulated under the mechanical and dynamic conditions of the manipulator. The design goal of the manipulator is to recommend quasi-serial mechanisms with optimized kinematic (workspace) and dynamic (joint torque) performance that satisfies the target task. To investigate the underlying physics from the obtained Pareto solutions, various data mining techniques are performed to extract design rules that can provide practical design guidance. Finally, the manipulator was designed in detail for realization with 3D printed parts, including topology optimization. Also, the task-based optimized manipulator is verified through a payload test. Based on these results, the proposed framework has the potential for other real applications as realized cases and provides a reasonable design plan through the design rule extraction.
△ Less
Submitted 7 January, 2024;
originally announced February 2024.
-
Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective
Authors:
Sunwoong Yang,
Ho** Kim,
Yoonpyo Hong,
Kwanjung Yee,
Romit Maulik,
Namwoo Kang
Abstract:
This study explores the potential of physics-informed neural networks (PINNs) for the realization of digital twins (DT) from various perspectives. First, various adaptive sampling approaches for collocation points are investigated to verify their effectiveness in the mesh-free framework of PINNs, which allows automated construction of virtual representation without manual mesh generation. Then, th…
▽ More
This study explores the potential of physics-informed neural networks (PINNs) for the realization of digital twins (DT) from various perspectives. First, various adaptive sampling approaches for collocation points are investigated to verify their effectiveness in the mesh-free framework of PINNs, which allows automated construction of virtual representation without manual mesh generation. Then, the overall performance of the data-driven PINNs (DD-PINNs) framework is examined, which can utilize the acquired datasets in DT scenarios. Its scalability to more general physics is validated within parametric Navier-Stokes equations, where PINNs do not need to be retrained as the Reynolds number varies. In addition, since datasets can be often collected from different fidelity/sparsity in practice, multi-fidelity DD-PINNs are also proposed and evaluated. They show remarkable prediction performance even in the extrapolation tasks, with $42\sim62\%$ improvement over the single-fidelity approach. Finally, the uncertainty quantification performance of multi-fidelity DD-PINNs is investigated by the ensemble method to verify their potential in DT, where an accurate measure of predictive uncertainty is critical. The DD-PINN frameworks explored in this study are found to be more suitable for DT scenarios than traditional PINNs from the above perspectives, bringing engineers one step closer to seamless DT realization.
△ Less
Submitted 19 May, 2024; v1 submitted 5 January, 2024;
originally announced January 2024.
-
A Continual Learning Paradigm for Non-differentiable Visual Programming Frameworks on Visual Reasoning Tasks
Authors:
Wentao Wan,
Nan Kang,
Zeqing Wang,
Zhuojie Yang,
Liang Lin,
Keze Wang
Abstract:
Recently, the visual programming framework (VisProg) has emerged as a significant framework for executing compositional visual tasks due to its interpretability and flexibility. However, the performance of VisProg on specific Visual Reasoning (VR) tasks is markedly inferior compared to well-trained task-specific models since its employed visual sub-modules have limited generalization capabilities.…
▽ More
Recently, the visual programming framework (VisProg) has emerged as a significant framework for executing compositional visual tasks due to its interpretability and flexibility. However, the performance of VisProg on specific Visual Reasoning (VR) tasks is markedly inferior compared to well-trained task-specific models since its employed visual sub-modules have limited generalization capabilities. Due to the non-differentiability of VisProg, it is quite challenging to improve these visual sub-modules within VisProg for the specific VR task while maintaining their generalizability on the un-seen tasks. Attempt to overcome these difficulties, we propose CLVP, a Continuous Learning paradigm for VisProg across various visual reasoning tasks. Specifically, our CLVP distills the capabilities of well-trained task-specific models into the visual sub-modules in a stepwise and anti-forgetting manner. This can continually improve the performance of VisProg on multiple visual tasks while preserving the flexibility of VisProg. Extensive and comprehensive experimental results demonstrate that our CLVP obtains significant performance gains on specific VR benchmarks, i.e., GQA (+1.4%) and NLVRv2 (+5.6%), compared to the VisProg baseline, and also maintains a promising generalizability for VR on un-seen and previous learned tasks.
△ Less
Submitted 30 November, 2023; v1 submitted 18 September, 2023;
originally announced September 2023.
-
Weighted Unsupervised Domain Adaptation Considering Geometry Features and Engineering Performance of 3D Design Data
Authors:
Seungyeon Shin,
Namwoo Kang
Abstract:
The product design process in manufacturing involves iterative design modeling and analysis to achieve the target engineering performance, but such an iterative process is time consuming and computationally expensive. Recently, deep learning-based engineering performance prediction models have been proposed to accelerate design optimization. However, they only guarantee predictions on training dat…
▽ More
The product design process in manufacturing involves iterative design modeling and analysis to achieve the target engineering performance, but such an iterative process is time consuming and computationally expensive. Recently, deep learning-based engineering performance prediction models have been proposed to accelerate design optimization. However, they only guarantee predictions on training data and may be inaccurate when applied to new domain data. In particular, 3D design data have complex features, which means domains with various distributions exist. Thus, the utilization of deep learning has limitations due to the heavy data collection and training burdens. We propose a bi-weighted unsupervised domain adaptation approach that considers the geometry features and engineering performance of 3D design data. It is specialized for deep learning-based engineering performance predictions. Domain-invariant features can be extracted through an adversarial training strategy by using hypothesis discrepancy, and a multi-output regression task can be performed with the extracted features to predict the engineering performance. In particular, we present a source instance weighting method suitable for 3D design data to avoid negative transfers. The developed bi-weighting strategy based on the geometry features and engineering performance of engineering structures is incorporated into the training process. The proposed model is tested on a wheel impact analysis problem to predict the magnitude of the maximum von Mises stress and the corresponding location of 3D road wheels. This mechanism can reduce the target risk for unlabeled target domains on the basis of weighted multi-source domain knowledge and can efficiently replace conventional finite element analysis.
△ Less
Submitted 7 September, 2023;
originally announced September 2023.
-
Performance Comparison of Design Optimization and Deep Learning-based Inverse Design
Authors:
Minyoung Jwa,
Jihoon Kim,
Seungyeon Shin,
Ah-hyeon **,
Dongju Shin,
Namwoo Kang
Abstract:
Surrogate model-based optimization has been increasingly used in the field of engineering design. It involves creating a surrogate model with objective functions or constraints based on the data obtained from simulations or real-world experiments, and then finding the optimal solution from the model using numerical optimization methods. Recent advancements in deep learning-based inverse design met…
▽ More
Surrogate model-based optimization has been increasingly used in the field of engineering design. It involves creating a surrogate model with objective functions or constraints based on the data obtained from simulations or real-world experiments, and then finding the optimal solution from the model using numerical optimization methods. Recent advancements in deep learning-based inverse design methods have made it possible to generate real-time optimal solutions for engineering design problems, eliminating the requirement for iterative optimization processes. Nevertheless, no comprehensive study has yet closely examined the specific advantages and disadvantages of this novel approach compared to the traditional design optimization method. The objective of this paper is to compare the performance of traditional design optimization methods with deep learning-based inverse design methods by employing benchmark problems across various scenarios. Based on the findings of this study, we provide guidelines that can be taken into account for the future utilization of deep learning-based inverse design. It is anticipated that these guidelines will enhance the practical applicability of this approach to real engineering design problems.
△ Less
Submitted 23 August, 2023;
originally announced August 2023.
-
Is end-to-end learning enough for fitness activity recognition?
Authors:
Antoine Mercier,
Guillaume Berger,
Sunny Panchal,
Florian Letsch,
Cornelius Boehm,
Nahua Kang,
Ingo Bax,
Roland Memisevic
Abstract:
End-to-end learning has taken hold of many computer vision tasks, in particular, related to still images, with task-specific optimization yielding very strong performance. Nevertheless, human-centric action recognition is still largely dominated by hand-crafted pipelines, and only individual components are replaced by neural networks that typically operate on individual frames. As a testbed to stu…
▽ More
End-to-end learning has taken hold of many computer vision tasks, in particular, related to still images, with task-specific optimization yielding very strong performance. Nevertheless, human-centric action recognition is still largely dominated by hand-crafted pipelines, and only individual components are replaced by neural networks that typically operate on individual frames. As a testbed to study the relevance of such pipelines, we present a new fully annotated video dataset of fitness activities. Any recognition capabilities in this domain are almost exclusively a function of human poses and their temporal dynamics, so pose-based solutions should perform well. We show that, with this labelled data, end-to-end learning on raw pixels can compete with state-of-the-art action recognition pipelines based on pose estimation. We also show that end-to-end learning can support temporally fine-grained tasks such as real-time repetition counting.
△ Less
Submitted 14 May, 2023;
originally announced May 2023.
-
A simple model of influence
Authors:
Colin Cooper,
Nan Kang,
Tomasz Radzik
Abstract:
We propose a simple model of influence in a network, based on edge density. In the model vertices (people) follow the opinion of the group they belong to. The opinion percolates down from an active vertex, the influencer, at the head of the group. Groups can merge, based on interactions between influencers (i.e., interactions along `active edges' of the network), so that the number of opinions is…
▽ More
We propose a simple model of influence in a network, based on edge density. In the model vertices (people) follow the opinion of the group they belong to. The opinion percolates down from an active vertex, the influencer, at the head of the group. Groups can merge, based on interactions between influencers (i.e., interactions along `active edges' of the network), so that the number of opinions is reduced. Eventually no active edges remain, and the groups and their opinions become static.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Temporal Interpolation Is All You Need for Dynamic Neural Radiance Fields
Authors:
Sungheon Park,
Minjung Son,
Seokhwan Jang,
Young Chun Ahn,
Ji-Yeon Kim,
Nahyup Kang
Abstract:
Temporal interpolation often plays a crucial role to learn meaningful representations in dynamic scenes. In this paper, we propose a novel method to train spatiotemporal neural radiance fields of dynamic scenes based on temporal interpolation of feature vectors. Two feature interpolation methods are suggested depending on underlying representations, neural networks or grids. In the neural represen…
▽ More
Temporal interpolation often plays a crucial role to learn meaningful representations in dynamic scenes. In this paper, we propose a novel method to train spatiotemporal neural radiance fields of dynamic scenes based on temporal interpolation of feature vectors. Two feature interpolation methods are suggested depending on underlying representations, neural networks or grids. In the neural representation, we extract features from space-time inputs via multiple neural network modules and interpolate them based on time frames. The proposed multi-level feature interpolation network effectively captures features of both short-term and long-term time ranges. In the grid representation, space-time features are learned via four-dimensional hash grids, which remarkably reduces training time. The grid representation shows more than 100 times faster training speed than the previous neural-net-based methods while maintaining the rendering quality. Concatenating static and dynamic features and adding a simple smoothness term further improve the performance of our proposed models. Despite the simplicity of the model architectures, our method achieved state-of-the-art performance both in rendering quality for the neural representation and in training speed for the grid representation.
△ Less
Submitted 29 March, 2023; v1 submitted 18 February, 2023;
originally announced February 2023.
-
Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Finance
Authors:
Gui** Son,
Hanwool Lee,
Nahyeon Kang,
Moonjeong Hahm
Abstract:
Extraction of sentiment signals from news text, stock message boards, and business reports, for stock movement prediction, has been a rising field of interest in finance. Building upon past literature, the most recent works attempt to better capture sentiment from sentences with complex syntactic structures by introducing aspect-level sentiment classification (ASC). Despite the growing interest, h…
▽ More
Extraction of sentiment signals from news text, stock message boards, and business reports, for stock movement prediction, has been a rising field of interest in finance. Building upon past literature, the most recent works attempt to better capture sentiment from sentences with complex syntactic structures by introducing aspect-level sentiment classification (ASC). Despite the growing interest, however, fine-grained sentiment analysis has not been fully explored in non-English literature due to the shortage of annotated finance-specific data. Accordingly, it is necessary for non-English languages to leverage datasets and pre-trained language models (PLM) of different domains, languages, and tasks to best their performance. To facilitate finance-specific ASC research in the Korean language, we build KorFinASC, a Korean aspect-level sentiment classification dataset for finance consisting of 12,613 human-annotated samples, and explore methods of intermediate transfer learning. Our experiments indicate that past research has been ignorant towards the potentially wrong knowledge of financial entities encoded during the training phase, which has overestimated the predictive power of PLMs. In our work, we use the term "non-stationary knowledge'' to refer to information that was previously correct but is likely to change, and present "TGT-Masking'', a novel masking pattern to restrict PLMs from speculating knowledge of the kind. Finally, through a series of transfer learning with TGT-Masking applied we improve 22.63% of classification accuracy compared to standalone models on KorFinASC.
△ Less
Submitted 24 January, 2023; v1 submitted 8 January, 2023;
originally announced January 2023.
-
PANeRF: Pseudo-view Augmentation for Improved Neural Radiance Fields Based on Few-shot Inputs
Authors:
Young Chun Ahn,
Seokhwan Jang,
Sungheon Park,
Ji-Yeon Kim,
Nahyup Kang
Abstract:
The method of neural radiance fields (NeRF) has been developed in recent years, and this technology has promising applications for synthesizing novel views of complex scenes. However, NeRF requires dense input views, typically numbering in the hundreds, for generating high-quality images. With a decrease in the number of input views, the rendering quality of NeRF for unseen viewpoints tends to deg…
▽ More
The method of neural radiance fields (NeRF) has been developed in recent years, and this technology has promising applications for synthesizing novel views of complex scenes. However, NeRF requires dense input views, typically numbering in the hundreds, for generating high-quality images. With a decrease in the number of input views, the rendering quality of NeRF for unseen viewpoints tends to degenerate drastically. To overcome this challenge, we propose pseudo-view augmentation of NeRF, a scheme that expands a sufficient amount of data by considering the geometry of few-shot inputs. We first initialized the NeRF network by leveraging the expanded pseudo-views, which efficiently minimizes uncertainty when rendering unseen views. Subsequently, we fine-tuned the network by utilizing sparse-view inputs containing precise geometry and color information. Through experiments under various settings, we verified that our model faithfully synthesizes novel-view images of superior quality and outperforms existing methods for multi-view datasets.
△ Less
Submitted 23 November, 2022;
originally announced November 2022.
-
Topology Optimization via Machine Learning and Deep Learning: A Review
Authors:
Seungyeon Shin,
Dongju Shin,
Namwoo Kang
Abstract:
Topology optimization (TO) is a method of deriving an optimal design that satisfies a given load and boundary conditions within a design domain. This method enables effective design without initial design, but has been limited in use due to high computational costs. At the same time, machine learning (ML) methodology including deep learning has made great progress in the 21st century, and accordin…
▽ More
Topology optimization (TO) is a method of deriving an optimal design that satisfies a given load and boundary conditions within a design domain. This method enables effective design without initial design, but has been limited in use due to high computational costs. At the same time, machine learning (ML) methodology including deep learning has made great progress in the 21st century, and accordingly, many studies have been conducted to enable effective and rapid optimization by applying ML to TO. Therefore, this study reviews and analyzes previous research on ML-based TO (MLTO). Two different perspectives of MLTO are used to review studies: (1) TO and (2) ML perspectives. The TO perspective addresses "why" to use ML for TO, while the ML perspective addresses "how" to apply ML to TO. In addition, the limitations of current MLTO research and future research directions are examined.
△ Less
Submitted 5 June, 2023; v1 submitted 19 October, 2022;
originally announced October 2022.
-
Adaptive Neural Network Ensemble Using Frequency Distribution
Authors:
Ungki Lee,
Namwoo Kang
Abstract:
Neural network (NN) ensembles can reduce large prediction variance of NN and improve prediction accuracy. For highly nonlinear problems with insufficient data set, the prediction accuracy of NN models becomes unstable, resulting in a decrease in the accuracy of ensembles. Therefore, this study proposes a frequency distribution-based ensemble that identifies core prediction values, which are expect…
▽ More
Neural network (NN) ensembles can reduce large prediction variance of NN and improve prediction accuracy. For highly nonlinear problems with insufficient data set, the prediction accuracy of NN models becomes unstable, resulting in a decrease in the accuracy of ensembles. Therefore, this study proposes a frequency distribution-based ensemble that identifies core prediction values, which are expected to be concentrated near the true prediction value. The frequency distribution-based ensemble classifies core prediction values supported by multiple prediction values by conducting statistical analysis with a frequency distribution, which is based on various prediction values obtained from a given prediction point. The frequency distribution-based ensemble can improve predictive performance by excluding prediction values with low accuracy and co** with the uncertainty of the most frequent value. An adaptive sampling strategy that sequentially adds samples based on the core prediction variance calculated as the variance of the core prediction values is proposed to improve the predictive performance of the frequency distribution-based ensemble efficiently. Results of various case studies show that the prediction accuracy of the frequency distribution-based ensemble is higher than that of Kriging and other existing ensemble methods. In addition, the proposed adaptive sampling strategy effectively improves the predictive performance of the frequency distribution-based ensemble compared with the previously developed space-filling and prediction variance-based strategies.
△ Less
Submitted 19 October, 2022;
originally announced October 2022.
-
Wheel Impact Test by Deep Learning: Prediction of Location and Magnitude of Maximum Stress
Authors:
Seungyeon Shin,
Ah-hyeon **,
Soyoung Yoo,
Sunghee Lee,
ChangGon Kim,
Sungpil Heo,
Namwoo Kang
Abstract:
For ensuring vehicle safety, the impact performance of wheels during wheel development must be ensured through a wheel impact test. However, manufacturing and testing a real wheel requires a significant time and money because develo** an optimal wheel design requires numerous iterative processes to modify the wheel design and verify the safety performance. Accordingly, wheel impact tests have be…
▽ More
For ensuring vehicle safety, the impact performance of wheels during wheel development must be ensured through a wheel impact test. However, manufacturing and testing a real wheel requires a significant time and money because develo** an optimal wheel design requires numerous iterative processes to modify the wheel design and verify the safety performance. Accordingly, wheel impact tests have been replaced by computer simulations such as finite element analysis (FEA); however, it still incurs high computational costs for modeling and analysis, and requires FEA experts. In this study, we present an aluminum road wheel impact performance prediction model based on deep learning that replaces computationally expensive and time-consuming 3D FEA. For this purpose, 2D disk-view wheel image data, 3D wheel voxel data, and barrier mass values used for the wheel impact test were utilized as the inputs to predict the magnitude of the maximum von Mises stress, corresponding location, and the stress distribution of the 2D disk-view. The input data were first compressed into a latent space with a 3D convolutional variational autoencoder (cVAE) and 2D convolutional autoencoder (cAE). Subsequently, the fully connected layers were used to predict the impact performance, and a decoder was used to predict the stress distribution heatmap of the 2D disk-view. The proposed model can replace the impact test in the early wheel-development stage by predicting the impact performance in real-time and can be used without domain knowledge. The time required for the wheel development process can be reduced by using this mechanism.
△ Less
Submitted 18 December, 2022; v1 submitted 3 October, 2022;
originally announced October 2022.
-
PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE Solvers
Authors:
Namgyu Kang,
Byeonghyeon Lee,
Youngjoon Hong,
Seok-Bae Yun,
Eunbyung Park
Abstract:
With the increases in computational power and advances in machine learning, data-driven learning-based methods have gained significant attention in solving PDEs. Physics-informed neural networks (PINNs) have recently emerged and succeeded in various forward and inverse PDE problems thanks to their excellent properties, such as flexibility, mesh-free solutions, and unsupervised training. However, t…
▽ More
With the increases in computational power and advances in machine learning, data-driven learning-based methods have gained significant attention in solving PDEs. Physics-informed neural networks (PINNs) have recently emerged and succeeded in various forward and inverse PDE problems thanks to their excellent properties, such as flexibility, mesh-free solutions, and unsupervised training. However, their slower convergence speed and relatively inaccurate solutions often limit their broader applicability in many science and engineering domains. This paper proposes a new kind of data-driven PDEs solver, physics-informed cell representations (PIXEL), elegantly combining classical numerical methods and learning-based approaches. We adopt a grid structure from the numerical methods to improve accuracy and convergence speed and overcome the spectral bias presented in PINNs. Moreover, the proposed method enjoys the same benefits in PINNs, e.g., using the same optimization frameworks to solve both forward and inverse PDE problems and readily enforcing PDE constraints with modern automatic differentiation techniques. We provide experimental results on various challenging PDEs that the original PINNs have struggled with and show that PIXEL achieves fast convergence speed and high accuracy. Project page: https://namgyukang.github.io/PIXEL/
△ Less
Submitted 4 February, 2023; v1 submitted 26 July, 2022;
originally announced July 2022.
-
PILC: Practical Image Lossless Compression with an End-to-end GPU Oriented Neural Framework
Authors:
Ning Kang,
Shanzhao Qiu,
Shifeng Zhang,
Zhenguo Li,
Shutao Xia
Abstract:
Generative model based image lossless compression algorithms have seen a great success in improving compression ratio. However, the throughput for most of them is less than 1 MB/s even with the most advanced AI accelerated chips, preventing them from most real-world applications, which often require 100 MB/s. In this paper, we propose PILC, an end-to-end image lossless compression framework that a…
▽ More
Generative model based image lossless compression algorithms have seen a great success in improving compression ratio. However, the throughput for most of them is less than 1 MB/s even with the most advanced AI accelerated chips, preventing them from most real-world applications, which often require 100 MB/s. In this paper, we propose PILC, an end-to-end image lossless compression framework that achieves 200 MB/s for both compression and decompression with a single NVIDIA Tesla V100 GPU, 10 times faster than the most efficient one before. To obtain this result, we first develop an AI codec that combines auto-regressive model and VQ-VAE which performs well in lightweight setting, then we design a low complexity entropy coder that works well with our codec. Experiments show that our framework compresses better than PNG by a margin of 30% in multiple datasets. We believe this is an important step to bring AI compression forward to commercial use.
△ Less
Submitted 9 June, 2022;
originally announced June 2022.
-
Split Hierarchical Variational Compression
Authors:
Tom Ryder,
Chen Zhang,
Ning Kang,
Shifeng Zhang
Abstract:
Variational autoencoders (VAEs) have witnessed great success in performing the compression of image datasets. This success, made possible by the bits-back coding framework, has produced competitive compression performance across many benchmarks. However, despite this, VAE architectures are currently limited by a combination of coding practicalities and compression ratios. That is, not only do stat…
▽ More
Variational autoencoders (VAEs) have witnessed great success in performing the compression of image datasets. This success, made possible by the bits-back coding framework, has produced competitive compression performance across many benchmarks. However, despite this, VAE architectures are currently limited by a combination of coding practicalities and compression ratios. That is, not only do state-of-the-art methods, such as normalizing flows, often demonstrate out-performance, but the initial bits required in coding makes single and parallel image compression challenging. To remedy this, we introduce Split Hierarchical Variational Compression (SHVC). SHVC introduces two novelties. Firstly, we propose an efficient autoregressive prior, the autoregressive sub-pixel convolution, that allows a generalisation between per-pixel autoregressions and fully factorised probability models. Secondly, we define our coding framework, the autoregressive initial bits, that flexibly supports parallel coding and avoids -- for the first time -- many of the practicalities commonly associated with bits-back coding. In our experiments, we demonstrate SHVC is able to achieve state-of-the-art compression performance across full-resolution lossless image compression tasks, with up to 100x fewer model parameters than competing VAE approaches.
△ Less
Submitted 5 April, 2022;
originally announced April 2022.
-
Deep Learning-Based Inverse Design for Engineering Systems: Multidisciplinary Design Optimization of Automotive Brakes
Authors:
Seongsin Kim,
Minyoung Jwa,
Soonwook Lee,
Sunghoon Park,
Namwoo Kang
Abstract:
The braking performance of the brake system is a target performance that must be considered for vehicle development. Apparent piston travel (APT) and drag torque are the most representative factors for evaluating braking performance. In particular, as the two performance factors have a conflicting relationship with each other, a multidisciplinary design optimization (MDO) approach is required for…
▽ More
The braking performance of the brake system is a target performance that must be considered for vehicle development. Apparent piston travel (APT) and drag torque are the most representative factors for evaluating braking performance. In particular, as the two performance factors have a conflicting relationship with each other, a multidisciplinary design optimization (MDO) approach is required for brake design. However, the computational cost of MDO increases as the number of disciplines increases. Recent studies on inverse design that use deep learning (DL) have established the possibility of instantly generating an optimal design that can satisfy the target performance without implementing an iterative optimization process. This study proposes a DL-based multidisciplinary inverse design (MID) that simultaneously satisfies multiple targets, such as the APT and drag torque of the brake system. Results show that the proposed inverse design can find the optimal design more efficiently compared with the conventional optimization methods, such as backpropagation and sequential quadratic programming. The MID achieved a similar performance to the single-disciplinary inverse design in terms of accuracy and computational cost. A novel design was derived on the basis of results, and the same performance was satisfied as that of the existing design.
△ Less
Submitted 27 February, 2022;
originally announced February 2022.
-
Parallel Neural Local Lossless Compression
Authors:
Mingtian Zhang,
James Townsend,
Ning Kang,
David Barber
Abstract:
The recently proposed Neural Local Lossless Compression (NeLLoC), which is based on a local autoregressive model, has achieved state-of-the-art (SOTA) out-of-distribution (OOD) generalization performance in the image compression task. In addition to the encouragement of OOD generalization, the local model also allows parallel inference in the decoding stage. In this paper, we propose two paralleli…
▽ More
The recently proposed Neural Local Lossless Compression (NeLLoC), which is based on a local autoregressive model, has achieved state-of-the-art (SOTA) out-of-distribution (OOD) generalization performance in the image compression task. In addition to the encouragement of OOD generalization, the local model also allows parallel inference in the decoding stage. In this paper, we propose two parallelization schemes for local autoregressive models. We discuss the practicalities of implementing the schemes and provide experimental evidence of significant gains in compression runtime compared to the previous, non-parallel implementation.
△ Less
Submitted 26 June, 2022; v1 submitted 13 January, 2022;
originally announced January 2022.
-
iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder
Authors:
Shifeng Zhang,
Ning Kang,
Tom Ryder,
Zhenguo Li
Abstract:
It was estimated that the world produced $59 ZB$ ($5.9 \times 10^{13} GB$) of data in 2020, resulting in the enormous costs of both data storage and transmission. Fortunately, recent advances in deep generative models have spearheaded a new class of so-called "neural compression" algorithms, which significantly outperform traditional codecs in terms of compression ratio. Unfortunately, the applica…
▽ More
It was estimated that the world produced $59 ZB$ ($5.9 \times 10^{13} GB$) of data in 2020, resulting in the enormous costs of both data storage and transmission. Fortunately, recent advances in deep generative models have spearheaded a new class of so-called "neural compression" algorithms, which significantly outperform traditional codecs in terms of compression ratio. Unfortunately, the application of neural compression garners little commercial interest due to its limited bandwidth; therefore, develo** highly efficient frameworks is of critical practical importance. In this paper, we discuss lossless compression using normalizing flows which have demonstrated a great capacity for achieving high compression ratios. As such, we introduce iFlow, a new method for achieving efficient lossless compression. We first propose Modular Scale Transform (MST) and a novel family of numerically invertible flow transformations based on MST. Then we introduce the Uniform Base Conversion System (UBCS), a fast uniform-distribution codec incorporated into iFlow, enabling efficient compression. iFlow achieves state-of-the-art compression ratios and is $5\times$ quicker than other high-performance schemes. Furthermore, the techniques presented in this paper can be used to accelerate coding time for a broad class of flow-based algorithms.
△ Less
Submitted 1 November, 2021;
originally announced November 2021.
-
NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models
Authors:
Hang Xu,
Ning Kang,
Gengwei Zhang,
Chuanlong Xie,
Xiaodan Liang,
Zhenguo Li
Abstract:
Fine-tuning from pre-trained ImageNet models has been a simple, effective, and popular approach for various computer vision tasks. The common practice of fine-tuning is to adopt a default hyperparameter setting with a fixed pre-trained model, while both of them are not optimized for specific tasks and time constraints. Moreover, in cloud computing or GPU clusters where the tasks arrive sequentiall…
▽ More
Fine-tuning from pre-trained ImageNet models has been a simple, effective, and popular approach for various computer vision tasks. The common practice of fine-tuning is to adopt a default hyperparameter setting with a fixed pre-trained model, while both of them are not optimized for specific tasks and time constraints. Moreover, in cloud computing or GPU clusters where the tasks arrive sequentially in a stream, faster online fine-tuning is a more desired and realistic strategy for saving money, energy consumption, and CO2 emission. In this paper, we propose a joint Neural Architecture Search and Online Adaption framework named NASOA towards a faster task-oriented fine-tuning upon the request of users. Specifically, NASOA first adopts an offline NAS to identify a group of training-efficient networks to form a pretrained model zoo. We propose a novel joint block and macro-level search space to enable a flexible and efficient search. Then, by estimating fine-tuning performance via an adaptive model by accumulating experience from the past tasks, an online schedule generator is proposed to pick up the most suitable model and generate a personalized training regime with respect to each desired task in a one-shot fashion. The resulting model zoo is more training efficient than SOTA models, e.g. 6x faster than RegNetY-16GF, and 1.7x faster than EfficientNetB3. Experiments on multiple datasets also show that NASOA achieves much better fine-tuning results, i.e. improving around 2.1% accuracy than the best performance in RegNet series under various constraints and tasks; 40x faster compared to the BOHB.
△ Less
Submitted 7 August, 2021;
originally announced August 2021.
-
Diversity, Fairness, and Sustainability in Population Protocols
Authors:
Nan Kang,
Frederik Mallmann-Trenn,
Nicolás Rivera
Abstract:
Over the years, population protocols with the goal of reaching consensus have been studied in great depth. However, many systems in the real-world do not result in all agents eventually reaching consensus, but rather in the opposite: they converge to a state of rich diversity. Consider for example task allocation in ants. If eventually all ants perform the same task, then the colony will perish (l…
▽ More
Over the years, population protocols with the goal of reaching consensus have been studied in great depth. However, many systems in the real-world do not result in all agents eventually reaching consensus, but rather in the opposite: they converge to a state of rich diversity. Consider for example task allocation in ants. If eventually all ants perform the same task, then the colony will perish (lack of food, no brood care, etc.). Then, it is vital for the survival of the colony to have a diverse set of tasks and enough ants working on each task. What complicates matters is that ants need to switch tasks periodically to adjust the needs of the colony; e.g., when too many foragers fell victim to other ant colonies. Moreover, all tasks are equally important and maybe they need to keep certain proportions in the distribution of the task. How can ants keep a healthy and balanced allocation of tasks?
To answer this question, we propose a simple population protocol for $n$ agents on a complete graph and an arbitrary initial distribution of $k$ colours (tasks). We assume that each colour $i$ has an associated weight (importance) $w_i \geq 1$. By denoting $w$ as the sum of the weights of different colours, we show that the protocol converges in $O(w^2 n \log n)$ rounds to a configuration where the number of agents supporting each colour $i$ is concentrated on the fair share $w_in/w$ and will stay concentrated for a large number of rounds, w.h.p.
Our protocol has many interesting properties: agents do not need to know other colours and weights in the system, and our protocol requires very little memory per agent. Furthermore, the protocol guarantees fairness meaning that over a long period each agent has each colour roughly a number of times proportional to the weight of the colour. Finally, our protocol also fulfils sustainability meaning that no colour ever vanishes.
△ Less
Submitted 7 June, 2021; v1 submitted 20 May, 2021;
originally announced May 2021.
-
Syft 0.5: A Platform for Universally Deployable Structured Transparency
Authors:
Adam James Hall,
Madhava Jay,
Tudor Cebere,
Bogdan Cebere,
Koen Lennart van der Veen,
George Muraru,
Tongye Xu,
Patrick Cason,
William Abramson,
Ayoub Benaissa,
Chinmay Shah,
Alan Aboudib,
Théo Ryffel,
Kritika Prakash,
Tom Titcombe,
Varun Kumar Khare,
Maddie Shang,
Ionesio Junior,
Animesh Gupta,
Jason Paumier,
Nahua Kang,
Vova Manannikov,
Andrew Trask
Abstract:
We present Syft 0.5, a general-purpose framework that combines a core group of privacy-enhancing technologies that facilitate a universal set of structured transparency systems. This framework is demonstrated through the design and implementation of a novel privacy-preserving inference information flow where we pass homomorphically encrypted activation signals through a split neural network for in…
▽ More
We present Syft 0.5, a general-purpose framework that combines a core group of privacy-enhancing technologies that facilitate a universal set of structured transparency systems. This framework is demonstrated through the design and implementation of a novel privacy-preserving inference information flow where we pass homomorphically encrypted activation signals through a split neural network for inference. We show that splitting the model further up the computation chain significantly reduces the computation time of inference and the payload size of activation signals at the cost of model secrecy. We evaluate our proposed flow with respect to its provision of the core structural transparency principles.
△ Less
Submitted 27 April, 2021; v1 submitted 26 April, 2021;
originally announced April 2021.
-
iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression
Authors:
Shifeng Zhang,
Chen Zhang,
Ning Kang,
Zhenguo Li
Abstract:
It is nontrivial to store rapidly growing big data nowadays, which demands high-performance lossless compression techniques. Likelihood-based generative models have witnessed their success on lossless compression, where flow based models are desirable in allowing exact data likelihood optimisation with bijective map**s. However, common continuous flows are in contradiction with the discreteness…
▽ More
It is nontrivial to store rapidly growing big data nowadays, which demands high-performance lossless compression techniques. Likelihood-based generative models have witnessed their success on lossless compression, where flow based models are desirable in allowing exact data likelihood optimisation with bijective map**s. However, common continuous flows are in contradiction with the discreteness of coding schemes, which requires either 1) imposing strict constraints on flow models that degrades the performance or 2) coding numerous bijective map** errors which reduces the efficiency. In this paper, we investigate volume preserving flows for lossless compression and show that a bijective map** without error is possible. We propose Numerical Invertible Volume Preserving Flow (iVPF) which is derived from the general volume preserving flows. By introducing novel computation algorithms on flow models, an exact bijective map** is achieved without any numerical error. We also propose a lossless compression algorithm based on iVPF. Experiments on various datasets show that the algorithm based on iVPF achieves state-of-the-art compression ratio over lightweight compression algorithms.
△ Less
Submitted 1 November, 2021; v1 submitted 30 March, 2021;
originally announced March 2021.
-
VEGA: Towards an End-to-End Configurable AutoML Pipeline
Authors:
Bochao Wang,
Hang Xu,
Jia** Zhang,
Chen Chen,
Xiaozhi Fang,
Yixing Xu,
Ning Kang,
Lanqing Hong,
Chenhan Jiang,
Xinyue Cai,
Jiawei Li,
Fengwei Zhou,
Yong Li,
Zhicheng Liu,
Xinghao Chen,
Kai Han,
Han Shu,
Dehua Song,
Yunhe Wang,
Wei Zhang,
Chun**g Xu,
Zhenguo Li,
Wenzhi Liu,
Tong Zhang
Abstract:
Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models. However, designing an integrated AutoML system faces four great challenges of configurability, scalability, integrability, and platform diversity. In this work, we present VEGA, an efficient and comprehensive AutoML framework that is compatible and optimize…
▽ More
Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models. However, designing an integrated AutoML system faces four great challenges of configurability, scalability, integrability, and platform diversity. In this work, we present VEGA, an efficient and comprehensive AutoML framework that is compatible and optimized for multiple hardware platforms. a) The VEGA pipeline integrates various modules of AutoML, including Neural Architecture Search (NAS), Hyperparameter Optimization (HPO), Auto Data Augmentation, Model Compression, and Fully Train. b) To support a variety of search algorithms and tasks, we design a novel fine-grained search space and its description language to enable easy adaptation to different search algorithms and tasks. c) We abstract the common components of deep learning frameworks into a unified interface. VEGA can be executed with multiple back-ends and hardwares. Extensive benchmark experiments on multiple tasks demonstrate that VEGA can improve the existing AutoML algorithms and discover new high-performance models against SOTA methods, e.g. the searched DNet model zoo for Ascend 10x faster than EfficientNet-B5 and 9.2x faster than RegNetX-32GF on ImageNet. VEGA is open-sourced at https://github.com/huawei-noah/vega.
△ Less
Submitted 26 November, 2020; v1 submitted 3 November, 2020;
originally announced November 2020.
-
Explainable Artificial Intelligence for Manufacturing Cost Estimation and Machining Feature Visualization
Authors:
Soyoung Yoo,
Namwoo Kang
Abstract:
Studies on manufacturing cost prediction based on deep learning have begun in recent years, but the cost prediction rationale cannot be explained because the models are still used as a black box. This study aims to propose a manufacturing cost prediction process for 3D computer-aided design (CAD) models using explainable artificial intelligence. The proposed process can visualize the machining fea…
▽ More
Studies on manufacturing cost prediction based on deep learning have begun in recent years, but the cost prediction rationale cannot be explained because the models are still used as a black box. This study aims to propose a manufacturing cost prediction process for 3D computer-aided design (CAD) models using explainable artificial intelligence. The proposed process can visualize the machining features of the 3D CAD model that are influencing the increase in manufacturing costs. The proposed process consists of (1) data collection and pre-processing, (2) 3D deep learning architecture exploration, and (3) visualization to explain the prediction results. The proposed deep learning model shows high predictability of manufacturing cost for the computer numerical control (CNC) machined parts. In particular, using 3D gradient-weighted class activation map** proves that the proposed model not only can detect the CNC machining features but also can differentiate the machining difficulty for the same feature. Using the proposed process, we can provide a design guidance to engineering designers in reducing manufacturing costs during the conceptual design phase. We can also provide real-time quotations and redesign proposals to online manufacturing platform customers.
△ Less
Submitted 13 June, 2021; v1 submitted 28 October, 2020;
originally announced October 2020.
-
Idle Vehicle Relocation Strategy through Deep Learning for Shared Autonomous Electric Vehicle System Optimization
Authors:
Seongsin Kim,
Ungki Lee,
Ik** Lee,
Namwoo Kang
Abstract:
In optimization of a shared autonomous electric vehicle (SAEV) system, idle vehicle relocation strategies are important to reduce operation costs and customers' wait time. However, for an on-demand service, continuous optimization for idle vehicle relocation is computationally expensive, and thus, not effective. This study proposes a deep learning-based algorithm that can instantly predict the opt…
▽ More
In optimization of a shared autonomous electric vehicle (SAEV) system, idle vehicle relocation strategies are important to reduce operation costs and customers' wait time. However, for an on-demand service, continuous optimization for idle vehicle relocation is computationally expensive, and thus, not effective. This study proposes a deep learning-based algorithm that can instantly predict the optimal solution to idle vehicle relocation problems under various traffic conditions. The proposed relocation process comprises three steps. First, a deep learning-based passenger demand prediction model using taxi big data is built. Second, idle vehicle relocation problems are solved based on predicted demands, and optimal solution data are collected. Finally, a deep learning model using the optimal solution data is built to estimate the optimal strategy without solving relocation. In addition, the proposed idle vehicle relocation model is validated by applying it to optimize the SAEV system. We present an optimal service system including the design of SAEV vehicles and charging stations. Further, we demonstrate that the proposed strategy can drastically reduce operation costs and wait times for on-demand services.
△ Less
Submitted 16 October, 2020;
originally announced October 2020.
-
Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs
Authors:
Seowoo Jang,
Soyoung Yoo,
Namwoo Kang
Abstract:
Generative design refers to computational design methods that can automatically conduct design exploration under constraints defined by designers. Among many approaches, topology optimization-based generative designs aim to explore diverse topology designs, which cannot be represented by conventional parametric design approaches. Recently, data-driven topology optimization research has started to…
▽ More
Generative design refers to computational design methods that can automatically conduct design exploration under constraints defined by designers. Among many approaches, topology optimization-based generative designs aim to explore diverse topology designs, which cannot be represented by conventional parametric design approaches. Recently, data-driven topology optimization research has started to exploit artificial intelligence, such as deep learning or machine learning, to improve the capability of design exploration. This study proposes a reinforcement learning (RL) based generative design process, with reward functions maximizing the diversity of topology designs. We formulate generative design as a sequential problem of finding optimal design parameter combinations in accordance with a given reference design. Proximal Policy Optimization is used as the learning framework, which is demonstrated in the case study of an automotive wheel design problem. To reduce the heavy computational burden of the wheel topology optimization process required by our RL formulation, we approximate the optimization process with neural networks. With efficient data preprocessing/augmentation and neural architecture, the neural networks achieve a generalized performance and symmetricity-reserving characteristics. We show that RL-based generative design produces a large number of diverse designs within a short inference time by exploiting GPU in a fully automated manner. It is different from the previous approach using CPU which takes much more processing time and involving human intervention.
△ Less
Submitted 16 February, 2021; v1 submitted 17 August, 2020;
originally announced August 2020.
-
The Effect of Robo-taxi User Experience on User Acceptance: Field Test Data Analysis
Authors:
Sunghee Lee,
Soyoung Yoo,
Seongsin Kim,
Eunji Kim,
Namwoo Kang
Abstract:
With the advancement of self-driving technology, the commercialization of Robo-taxi services is just a matter of time. However, there is some skepticism regarding whether such taxi services will be successfully accepted by real customers due to perceived safety-related concerns; therefore, studies focused on user experience have become more crucial. Although many studies statistically analyze user…
▽ More
With the advancement of self-driving technology, the commercialization of Robo-taxi services is just a matter of time. However, there is some skepticism regarding whether such taxi services will be successfully accepted by real customers due to perceived safety-related concerns; therefore, studies focused on user experience have become more crucial. Although many studies statistically analyze user experience data obtained by surveying individuals' perceptions of Robo-taxi or indirectly through simulators, there is a lack of research that statistically analyzes data obtained directly from actual Robo-taxi service experiences. Accordingly, based on the user experience data obtained by implementing a Robo-taxi service in the downtown of Seoul and Daejeon in South Korea, this study quantitatively analyzes the effect of user experience on user acceptance through structural equation modeling and path analysis. We also obtained balanced and highly valid insights by reanalyzing meaningful causal relationships obtained through statistical models based on in-depth interview results. Results revealed that the experience of the traveling stage had the greatest effect on user acceptance, and the cutting edge of the service and apprehension of technology were emotions that had a great effect on user acceptance. Based on these findings, we suggest guidelines for the design and marketing of future Robo-taxi services.
△ Less
Submitted 12 July, 2021; v1 submitted 30 June, 2020;
originally announced June 2020.
-
Integrating Deep Learning into CAD/CAE System: Generative Design and Evaluation of 3D Conceptual Wheel
Authors:
Soyoung Yoo,
Sunghee Lee,
Seongsin Kim,
Kwang Hyeon Hwang,
Jong Ho Park,
Namwoo Kang
Abstract:
Engineering design research integrating artificial intelligence (AI) into computer-aided design (CAD) and computer-aided engineering (CAE) is actively being conducted. This study proposes a deep learning-based CAD/CAE framework in the conceptual design phase that automatically generates 3D CAD designs and evaluates their engineering performance. The proposed framework comprises seven stages: (1) 2…
▽ More
Engineering design research integrating artificial intelligence (AI) into computer-aided design (CAD) and computer-aided engineering (CAE) is actively being conducted. This study proposes a deep learning-based CAD/CAE framework in the conceptual design phase that automatically generates 3D CAD designs and evaluates their engineering performance. The proposed framework comprises seven stages: (1) 2D generative design, (2) dimensionality reduction, (3) design of experiment in latent space, (4) CAD automation, (5) CAE automation, (6) transfer learning, and (7) visualization and analysis. The proposed framework is demonstrated through a road wheel design case study and indicates that AI can be practically incorporated into an end-use product design project. Engineers and industrial designers can jointly review a large number of generated 3D CAD models by using this framework along with the engineering performance results estimated by AI and find conceptual design candidates for the subsequent detailed design stage.
△ Less
Submitted 13 June, 2021; v1 submitted 25 May, 2020;
originally announced June 2020.
-
A Study on Anxiety about Using Robo-taxis: HMI Design for Anxiety Factor Analysis and Anxiety Relief Based on Field Tests
Authors:
Soyoung Yoo,
Sunghee Lee,
Seongsin Kim,
Eunji Kim,
Hwan Hwangbo,
Namwoo Kang
Abstract:
Despite the approaching commercialization of robo-taxis, various anxiety factors concerning the safety of autonomous vehicles are expected to form a large barrier against consumers' use of robo-taxi services. The purpose of this study is to derive the various internal and external factors that contribute to the anxieties of robo-taxi passengers, and to propose a human-machine interface (HMI) conce…
▽ More
Despite the approaching commercialization of robo-taxis, various anxiety factors concerning the safety of autonomous vehicles are expected to form a large barrier against consumers' use of robo-taxi services. The purpose of this study is to derive the various internal and external factors that contribute to the anxieties of robo-taxi passengers, and to propose a human-machine interface (HMI) concept to resolve such factors, by testing robo-taxi services on real, complex urban roads. In addition, a remote system for safely testing a robo-taxi in complex downtown areas was constructed, by adopting the Wizard of Oz (WOZ) methodology. From the results of our tests - conducted upon 28 subjects in the central area of Seoul - 19 major anxiety factors arising from autonomous driving were identified, and seven HMI functions to resolve such factors were designed. The functions were evaluated and their anxiety reduction effects verified. In addition, the various design insights required to increase the reliability of robo-taxis were provided through quantitative and qualitative analysis of the user experience surveys and interviews.
△ Less
Submitted 21 February, 2020;
originally announced February 2020.
-
Form + Function: Optimizing Aesthetic Product Design via Adaptive, Geometrized Preference Elicitation
Authors:
Namwoo Kang,
Yi Ren,
Fred Feinberg,
Panos Papalambros
Abstract:
Visual design is critical to product success, and the subject of intensive marketing research effort. Yet visual elements, due to their holistic and interactive nature, do not lend themselves well to optimization using extant decompositional methods for preference elicitation. Here we present a systematic methodology to incorporate interactive, 3D-rendered product configurations into a conjoint-li…
▽ More
Visual design is critical to product success, and the subject of intensive marketing research effort. Yet visual elements, due to their holistic and interactive nature, do not lend themselves well to optimization using extant decompositional methods for preference elicitation. Here we present a systematic methodology to incorporate interactive, 3D-rendered product configurations into a conjoint-like framework. The method relies on rapid, scalable machine learning algorithms to adaptively update product designs along with standard information-oriented product attributes. At its heart is a parametric account of a product's geometry, along with a novel, adaptive "bi-level" query task that can estimate individuals' visual design form preferences and their trade-offs against such traditional elements as price and product features. We illustrate the method's performance through extensive simulations and robustness checks, a formal proof of the bi-level query methodology's domain of superiority, and a field test for the design of a mid-priced sedan, using real-time 3D rendering for an online panel. Results indicate not only substantially enhanced predictive accuracy, but two quantities beyond the reach of standard conjoint methods: trade-offs between form and function overall, and willingness-to-pay for specific design elements. Moreover -- and most critically for applications -- the method provides "optimal" visual designs for both individuals and model-derived or analyst-supplied consumer grou**s, as well as their sensitivities to form and functional elements.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
AIM 2019 Challenge on Constrained Super-Resolution: Methods and Results
Authors:
Kai Zhang,
Shuhang Gu,
Radu Timofte,
Zheng Hui,
Xiumei Wang,
Xinbo Gao,
Dongliang Xiong,
Shuai Liu,
Ruipeng Gang,
Nan Nan,
Chenghua Li,
Xueyi Zou,
Ning Kang,
Zhan Wang,
Hang Xu,
Chaofeng Wang,
Zheng Li,
Linlin Wang,
Jun Shi,
Wenyu Sun,
Zhiqiang Lang,
Jiangtao Nie,
Wei Wei,
Lei Zhang,
Yazhe Niu
, et al. (4 additional authors not shown)
Abstract:
This paper reviews the AIM 2019 challenge on constrained example-based single image super-resolution with focus on proposed solutions and results. The challenge had 3 tracks. Taking the three main aspects (i.e., number of parameters, inference/running time, fidelity (PSNR)) of MSRResNet as the baseline, Track 1 aims to reduce the amount of parameters while being constrained to maintain or improve…
▽ More
This paper reviews the AIM 2019 challenge on constrained example-based single image super-resolution with focus on proposed solutions and results. The challenge had 3 tracks. Taking the three main aspects (i.e., number of parameters, inference/running time, fidelity (PSNR)) of MSRResNet as the baseline, Track 1 aims to reduce the amount of parameters while being constrained to maintain or improve the running time and the PSNR result, Tracks 2 and 3 aim to optimize running time and PSNR result with constrain of the other two aspects, respectively. Each track had an average of 64 registered participants, and 12 teams submitted the final results. They gauge the state-of-the-art in single image super-resolution.
△ Less
Submitted 4 November, 2019;
originally announced November 2019.
-
Best-of-Three Voting on Dense Graphs
Authors:
Nan Kang,
Nicolas Rivera
Abstract:
Given a graph $G$ of $n$ vertices, where each vertex is initially attached an opinion of either red or blue. We investigate a random process known as the Best-of-three voting. In this process, at each time step, every vertex chooses three neighbours at random and adopts the majority colour. We study this process for a class of graphs with minimum degree $d = n^α$\,, where…
▽ More
Given a graph $G$ of $n$ vertices, where each vertex is initially attached an opinion of either red or blue. We investigate a random process known as the Best-of-three voting. In this process, at each time step, every vertex chooses three neighbours at random and adopts the majority colour. We study this process for a class of graphs with minimum degree $d = n^α$\,, where $α= Ω\left( (\log \log n)^{-1} \right)$. We prove that if initially each vertex is red with probability greater than $1/2+δ$, and blue otherwise, where $δ\geq (\log d)^{-C}$ for some $C>0$, then with high probability this dynamic reaches a final state where all vertices are red within $O\left( \log \log n\right) + O\left( \log \left( δ^{-1} \right) \right)$ steps.
△ Less
Submitted 22 March, 2019;
originally announced March 2019.
-
Deep Generative Design: Integration of Topology Optimization and Generative Models
Authors:
Sangeun Oh,
Yongsu Jung,
Seongsin Kim,
Ik** Lee,
Namwoo Kang
Abstract:
Deep learning has recently been applied to various research areas of design optimization. This study presents the need and effectiveness of adopting deep learning for generative design (or design exploration) research area. This work proposes an artificial intelligent (AI)-based design automation framework that is capable of generating numerous design options which are not only aesthetic but also…
▽ More
Deep learning has recently been applied to various research areas of design optimization. This study presents the need and effectiveness of adopting deep learning for generative design (or design exploration) research area. This work proposes an artificial intelligent (AI)-based design automation framework that is capable of generating numerous design options which are not only aesthetic but also optimized for engineering performance. The proposed framework integrates topology optimization and deep generative models (e.g., generative adversarial networks (GANs)) in an iterative manner to explore new design options, thus generating a large number of designs starting from limited previous design data. In addition, anomaly detection can evaluate the novelty of generated designs, thus hel** designers choose among design options. The 2D wheel design problem is applied as a case study for validation of the proposed framework. The framework manifests better aesthetics, diversity, and robustness of generated designs than previous generative design methods.
△ Less
Submitted 20 May, 2019; v1 submitted 1 March, 2019;
originally announced March 2019.
-
Online Makespan Minimization: The Power of Restart
Authors:
Zhiyi Huang,
Ning Kang,
Zhihao Gavin Tang,
Xiaowei Wu,
Yuhao Zhang
Abstract:
We consider the online makespan minimization problem on identical machines. Chen and Vestjens (ORL 1997) show that the largest processing time first (LPT) algorithm is 1.5-competitive. For the special case of two machines, Noga and Seiden (TCS 2001) introduce the SLEEPY algorithm that achieves a competitive ratio of $(5 - \sqrt{5})/2 \approx 1.382$, matching the lower bound by Chen and Vestjens (O…
▽ More
We consider the online makespan minimization problem on identical machines. Chen and Vestjens (ORL 1997) show that the largest processing time first (LPT) algorithm is 1.5-competitive. For the special case of two machines, Noga and Seiden (TCS 2001) introduce the SLEEPY algorithm that achieves a competitive ratio of $(5 - \sqrt{5})/2 \approx 1.382$, matching the lower bound by Chen and Vestjens (ORL 1997). Furthermore, Noga and Seiden note that in many applications one can kill a job and restart it later, and they leave an open problem whether algorithms with restart can obtain better competitive ratios.
We resolve this long-standing open problem on the positive end. Our algorithm has a natural rule for killing a processing job: a newly-arrived job replaces the smallest processing job if 1) the new job is larger than other pending jobs, 2) the new job is much larger than the processing one, and 3) the processed portion is small relative to the size of the new job. With appropriate choice of parameters, we show that our algorithm improves the 1.5 competitive ratio for the general case, and the 1.382 competitive ratio for the two-machine case.
△ Less
Submitted 6 June, 2018;
originally announced June 2018.
-
How to Match when All Vertices Arrive Online
Authors:
Zhiyi Huang,
Ning Kang,
Zhihao Gavin Tang,
Xiaowei Wu,
Yuhao Zhang,
Xue Zhu
Abstract:
We introduce a fully online model of maximum cardinality matching in which all vertices arrive online. On the arrival of a vertex, its incident edges to previously-arrived vertices are revealed. Each vertex has a deadline that is after all its neighbors' arrivals. If a vertex remains unmatched until its deadline, the algorithm must then irrevocably either match it to an unmatched neighbor, or leav…
▽ More
We introduce a fully online model of maximum cardinality matching in which all vertices arrive online. On the arrival of a vertex, its incident edges to previously-arrived vertices are revealed. Each vertex has a deadline that is after all its neighbors' arrivals. If a vertex remains unmatched until its deadline, the algorithm must then irrevocably either match it to an unmatched neighbor, or leave it unmatched. The model generalizes the existing one-sided online model and is motivated by applications including ride-sharing platforms, real-estate agency, etc.
We show that the Ranking algorithm by Karp et al. (STOC 1990) is $0.5211$-competitive in our fully online model for general graphs. Our analysis brings a novel charging mechanic into the randomized primal dual technique by Devanur et al. (SODA 2013), allowing a vertex other than the two endpoints of a matched edge to share the gain. To our knowledge, this is the first analysis of Ranking that beats $0.5$ on general graphs in an online matching problem, a first step towards solving the open problem by Karp et al. (STOC 1990) about the optimality of Ranking on general graphs. If the graph is bipartite, we show that the competitive ratio of Ranking is between $0.5541$ and $0.5671$. Finally, we prove that the fully online model is strictly harder than the previous model as no online algorithm can be $0.6317 < 1-\frac{1}{e}$-competitive in our model even for bipartite graphs.
△ Less
Submitted 12 February, 2018;
originally announced February 2018.
-
Online Submodular Maximization with Free Disposal: Randomization Beats 0.25 for Partition Matroids
Authors:
T-H. Hubert Chan,
Zhiyi Huang,
Shaofeng H. -C. Jiang,
Ning Kang,
Zhihao Gavin Tang
Abstract:
We study the online submodular maximization problem with free disposal under a matroid constraint. Elements from some ground set arrive one by one in rounds, and the algorithm maintains a feasible set that is independent in the underlying matroid. In each round when a new element arrives, the algorithm may accept the new element into its feasible set and possibly remove elements from it, provided…
▽ More
We study the online submodular maximization problem with free disposal under a matroid constraint. Elements from some ground set arrive one by one in rounds, and the algorithm maintains a feasible set that is independent in the underlying matroid. In each round when a new element arrives, the algorithm may accept the new element into its feasible set and possibly remove elements from it, provided that the resulting set is still independent. The goal is to maximize the value of the final feasible set under some monotone submodular function, to which the algorithm has oracle access.
For $k$-uniform matroids, we give a deterministic algorithm with competitive ratio at least $0.2959$, and the ratio approaches $\frac{1}{α_\infty} \approx 0.3178$ as $k$ approaches infinity, improving the previous best ratio of $0.25$ by Chakrabarti and Kale (IPCO 2014), Buchbinder et al. (SODA 2015) and Chekuri et al. (ICALP 2015). We also show that our algorithm is optimal among a class of deterministic monotone algorithms that accept a new arriving element only if the objective is strictly increased.
Further, we prove that no deterministic monotone algorithm can be strictly better than $0.25$-competitive even for partition matroids, the most modest generalization of $k$-uniform matroids, matching the competitive ratio by Chakrabarti and Kale (IPCO 2014) and Chekuri et al. (ICALP 2015). Interestingly, we show that randomized algorithms are strictly more powerful by giving a (non-monotone) randomized algorithm for partition matroids with ratio $\frac{1}{α_\infty} \approx 0.3178$.
△ Less
Submitted 25 October, 2016;
originally announced October 2016.
-
Online Convex Covering and Packing Problems
Authors:
T-H. Hubert Chan,
Zhiyi Huang,
Ning Kang
Abstract:
We study the online convex covering problem and online convex packing problem. The (offline) convex covering problem is modeled by the following convex program: $\min_{x \in R_+^n} f(x) \ \text{s.t}\ A x \ge 1$, where $f : R_+^n \mapsto R_+$ is a monotone and convex cost function, and $A$ is an $m \times n$ matrix with non-negative entries. Each row of the constraint matrix $A$ corresponds to a co…
▽ More
We study the online convex covering problem and online convex packing problem. The (offline) convex covering problem is modeled by the following convex program: $\min_{x \in R_+^n} f(x) \ \text{s.t}\ A x \ge 1$, where $f : R_+^n \mapsto R_+$ is a monotone and convex cost function, and $A$ is an $m \times n$ matrix with non-negative entries. Each row of the constraint matrix $A$ corresponds to a covering constraint. In the online problem, each row of $A$ comes online and the algorithm must maintain a feasible assignment $x$ and may only increase $x$ over time. The (offline) convex packing problem is modeled by the following convex program: $\max_{y\in R_+^m} \sum_{j = 1}^m y_j - g(A^T y)$, where $g : R_+^n \mapsto R_+$ is a monotone and convex cost function. It is the Fenchel dual program of convex covering when $g$ is the convex conjugate of $f$. In the online problem, each variable $y_j$ arrives online and the algorithm must decide the value of $y_j$ on its arrival.
We propose simple online algorithms for both problems using the online primal dual technique, and obtain nearly optimal competitive ratios for both problems for the important special case of polynomial cost functions. For any convex polynomial cost functions with non-negative coefficients and maximum degree $τ$, we introduce an $O(τ\log n)^τ$-competitive online convex covering algorithm, and an $O(τ)$-competitive online convex packing algorithm, matching the known $Ω(τ\log n)^τ$ and $Ω(τ)$ lower bounds respectively.
There is a large family of online resource allocation problems that can be modeled under this online convex covering and packing framework, including online covering and packing problems (with linear objectives), online mixed covering and packing, and online combinatorial auction. Our framework allows us to study these problems using a unified approach.
△ Less
Submitted 14 April, 2015; v1 submitted 6 February, 2015;
originally announced February 2015.
-
A Concept Annotation System for Clinical Records
Authors:
Ning Kang,
Rogier Barendse,
Zubair Afzal,
Bharat Singh,
Martijn J. Schuemie,
Erik M. van Mulligen,
Jan A. Kors
Abstract:
Unstructured information comprises a valuable source of data in clinical records. For text mining in clinical records, concept extraction is the first step in finding assertions and relationships. This study presents a system developed for the annotation of medical concepts, including medical problems, tests, and treatments, mentioned in clinical records. The system combines six publicly available…
▽ More
Unstructured information comprises a valuable source of data in clinical records. For text mining in clinical records, concept extraction is the first step in finding assertions and relationships. This study presents a system developed for the annotation of medical concepts, including medical problems, tests, and treatments, mentioned in clinical records. The system combines six publicly available named entity recognition system into one framework, and uses a simple voting scheme that allows to tune precision and recall of the system to specific needs. The system provides both a web service interface and a UIMA interface which can be easily used by other systems. The system was tested in the fourth i2b2 challenge and achieved an F-score of 82.1% for the concept exact match task, a score which is among the top-ranking systems. To our knowledge, this is the first publicly available clinical record concept annotation system.
△ Less
Submitted 7 December, 2010;
originally announced December 2010.