Showing 1–19 of 19 results for author: Yao, C

Seed-TTS: A Family of High-Quality Versatile Speech Generation Models

Authors: Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, Jiaxin Li, Xiaoyang Li, Xingxing Li, Lin Liu, Shouda Liu, Sichao Liu , et al. (21 additional authors not shown)

Abstract: We introduce Seed-TTS, a family of large-scale autoregressive text-to-speech (TTS) models capable of generating speech that is virtually indistinguishable from human speech. Seed-TTS serves as a foundation model for speech generation and excels in speech in-context learning, achieving performance in speaker similarity and naturalness that matches ground truth human speech in both objective and sub… ▽ More We introduce Seed-TTS, a family of large-scale autoregressive text-to-speech (TTS) models capable of generating speech that is virtually indistinguishable from human speech. Seed-TTS serves as a foundation model for speech generation and excels in speech in-context learning, achieving performance in speaker similarity and naturalness that matches ground truth human speech in both objective and subjective evaluations. With fine-tuning, we achieve even higher subjective scores across these metrics. Seed-TTS offers superior controllability over various speech attributes such as emotion and is capable of generating highly expressive and diverse speech for speakers in the wild. Furthermore, we propose a self-distillation method for speech factorization, as well as a reinforcement learning approach to enhance model robustness, speaker similarity, and controllability. We additionally present a non-autoregressive (NAR) variant of the Seed-TTS model, named $\text{Seed-TTS}_\text{DiT}$, which utilizes a fully diffusion-based architecture. Unlike previous NAR-based TTS systems, $\text{Seed-TTS}_\text{DiT}$ does not depend on pre-estimated phoneme durations and performs speech generation through end-to-end processing. We demonstrate that this variant achieves comparable performance to the language model-based variant and showcase its effectiveness in speech editing. We encourage readers to listen to demos at \url{https://bytedancespeech.github.io/seedtts_tech_report}. △ Less

Submitted 4 June, 2024; originally announced June 2024.

I$^2$VC: A Unified Framework for Intra- & Inter-frame Video Compression

Authors: Meiqin Liu, Chenming Xu, Yukai Gu, Chao Yao, Yao Zhao

Abstract: Video compression aims to reconstruct seamless frames by encoding the motion and residual information from existing frames. Previous neural video compression methods necessitate distinct codecs for three types of frames (I-frame, P-frame and B-frame), which hinders a unified approach and generalization across different video contexts. Intra-codec techniques lack the advanced Motion Estimation and… ▽ More Video compression aims to reconstruct seamless frames by encoding the motion and residual information from existing frames. Previous neural video compression methods necessitate distinct codecs for three types of frames (I-frame, P-frame and B-frame), which hinders a unified approach and generalization across different video contexts. Intra-codec techniques lack the advanced Motion Estimation and Motion Compensation (MEMC) found in inter-codec, leading to fragmented frameworks lacking uniformity. Our proposed Intra- & Inter-frame Video Compression (I$^2$VC) framework employs a single spatio-temporal codec that guides feature compression rates according to content importance. This unified codec transforms the dependence across frames into a conditional coding scheme, thus integrating intra- and inter-frame compression into one cohesive strategy. Given the absence of explicit motion data, achieving competent inter-frame compression with only a conditional codec poses a challenge. To resolve this, our approach includes an implicit inter-frame alignment mechanism. With the pre-trained diffusion denoising process, the utilization of a diffusion-inverted reference feature rather than random noise supports the initial compression state. This process allows for selective denoising of motion-rich regions based on decoded features, facilitating accurate alignment without the need for MEMC. Our experimental findings, across various compression configurations (AI, LD and RA) and frame types, prove that I$^2$VC outperforms the state-of-the-art perceptual learned codecs. Impressively, it exhibits a 58.4% enhancement in perceptual reconstruction performance when benchmarked against the H.266/VVC standard (VTM). Official implementation can be found at https://github.com/GYukai/I2VC. △ Less

Submitted 1 June, 2024; v1 submitted 23 May, 2024; originally announced May 2024.

Comments: 19 pages, 10 figures

Spatial-Related Sensors Matters: 3D Human Motion Reconstruction Assisted with Textual Semantics

Authors: Xueyuan Yang, Chao Yao, Xiaojuan Ban

Abstract: Leveraging wearable devices for motion reconstruction has emerged as an economical and viable technique. Certain methodologies employ sparse Inertial Measurement Units (IMUs) on the human body and harness data-driven strategies to model human poses. However, the reconstruction of motion based solely on sparse IMUs data is inherently fraught with ambiguity, a consequence of numerous identical IMU r… ▽ More Leveraging wearable devices for motion reconstruction has emerged as an economical and viable technique. Certain methodologies employ sparse Inertial Measurement Units (IMUs) on the human body and harness data-driven strategies to model human poses. However, the reconstruction of motion based solely on sparse IMUs data is inherently fraught with ambiguity, a consequence of numerous identical IMU readings corresponding to different poses. In this paper, we explore the spatial importance of multiple sensors, supervised by text that describes specific actions. Specifically, uncertainty is introduced to derive weighted features for each IMU. We also design a Hierarchical Temporal Transformer (HTT) and apply contrastive learning to achieve precise temporal and feature alignment of sensor data with textual semantics. Experimental results demonstrate our proposed approach achieves significant improvements in multiple metrics compared to existing methods. Notably, with textual supervision, our method not only differentiates between ambiguous actions such as sitting and standing but also produces more precise and natural motion. △ Less

Submitted 26 December, 2023; originally announced January 2024.

Comments: Accepted by AAAI 2024

Probabilistic Constellation Sha** for OFDM-Based ISAC Signaling

Authors: Zhen Du, Fan Liu, Yifeng Xiong, Tony Xiao Han, Weijie Yuan, Yuanhao Cui, Changhua Yao, Yonina C. Eldar

Abstract: Integrated Sensing and Communications (ISAC) has garnered significant attention as a promising technology for the upcoming sixth-generation wireless communication systems (6G). In pursuit of this goal, a common strategy is that a unified waveform, such as Orthogonal Frequency Division Multiplexing (OFDM), should serve dual-functional roles by enabling simultaneous sensing and communications (S&C)… ▽ More Integrated Sensing and Communications (ISAC) has garnered significant attention as a promising technology for the upcoming sixth-generation wireless communication systems (6G). In pursuit of this goal, a common strategy is that a unified waveform, such as Orthogonal Frequency Division Multiplexing (OFDM), should serve dual-functional roles by enabling simultaneous sensing and communications (S&C) operations. However, the sensing performance of an OFDM communication signal is substantially affected by the randomness of the data symbols mapped from bit streams. Therefore, achieving a balance between preserving communication capability (i.e., the randomness) while improving sensing performance remains a challenging task. To cope with this issue, in this paper we analyze the ambiguity function of the OFDM communication signal modulated by random data. Subsequently, a probabilistic constellation sha** (PCS) method is proposed to devise the probability distributions of constellation points, which is able to strike a scalable S&C tradeoff of the random transmitted signal. Finally, the superiority of the proposed PCS method over conventional uniformly distributed constellations is validated through numerical simulations. △ Less

Submitted 27 October, 2023; originally announced October 2023.

IBVC: Interpolation-driven B-frame Video Compression

Authors: Chenming Xu, Meiqin Liu, Chao Yao, Weisi Lin, Yao Zhao

Abstract: Learned B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction. However, previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation or video frame interpolation. They suffer from inaccurate quantized motions and inefficient motion compens… ▽ More Learned B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction. However, previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation or video frame interpolation. They suffer from inaccurate quantized motions and inefficient motion compensation. To address these issues, we propose a simple yet effective structure called Interpolation-driven B-frame Video Compression (IBVC). Our approach only involves two major operations: video frame interpolation and artifact reduction compression. IBVC introduces a bit-rate free MEMC based on interpolation, which avoids optical-flow quantization and additional compression distortions. Later, to reduce duplicate bit-rate consumption and focus on unaligned artifacts, a residual guided masking encoder is deployed to adaptively select the meaningful contexts with interpolated multi-scale dependencies. In addition, a conditional spatio-temporal decoder is proposed to eliminate location errors and artifacts instead of using MEMC coding in other methods. The experimental results on B-frame coding demonstrate that IBVC has significant improvements compared to the relevant state-of-the-art methods. Meanwhile, our approach can save bit rates compared with the random access (RA) configuration of H.266 (VTM). The code will be available at https://github.com/ruhig6/IBVC. △ Less

Submitted 14 March, 2024; v1 submitted 24 September, 2023; originally announced September 2023.

Comments: Submitted to Pattern Recognition

Direct Power Flow Controller with Continuous Full Regulation Range

Authors: Chong Yao, Youjun Zhang

Abstract: For enhancing power flow control in power transmission, a simplified new structure of direct power flow controller with continuous full regulation range (F-DPFC) was proposed. It has only one-stage power conversion and comprises of a three-phase transformer in parallel and a three-phase trans-former in series with grid, three single-phase full-bridge ac units, and a three-phase filter. Compared wi… ▽ More For enhancing power flow control in power transmission, a simplified new structure of direct power flow controller with continuous full regulation range (F-DPFC) was proposed. It has only one-stage power conversion and comprises of a three-phase transformer in parallel and a three-phase trans-former in series with grid, three single-phase full-bridge ac units, and a three-phase filter. Compared with previous DPFC, the proposed one dispenses with two complex three-phase se-lection switches which connect with high-voltage grid directly, and has a continuous 360° adjustment range of compensation voltage by taking place of buck-type ac unit with full-bridge type ac unit, and then expanding the limit of its duty cycle from [0,1] to [-1,1]. Within a large smooth zone replacing six separate zones, the proposed F-DPFC can regulate the ampli-tude and phase angle of grid node voltage respectively and simultaneously, and then the active and reactive power flow in grid can be controlled smoothly and effectively. The new structure is easy to achieve modular expansion and enables it to operate under high voltage and power conditions. Its struc-ture and operational principle were analyzed in detail, and a prototype was developed. The experimental results verified the feasibility and the correctness of the theoretical analysis. △ Less

Submitted 27 July, 2023; originally announced July 2023.

Comments: 9 pages,20 figures

Screentone-Aware Manga Super-Resolution Using DeepLearning

Authors: Chih-Yuan Yao, Husan-Ting Chou, Yu-Sheng Lin, Kuo-wei Chen

Abstract: Manga, as a widely beloved form of entertainment around the world, have shifted from paper to electronic screens with the proliferation of handheld devices. However, as the demand for image quality increases with screen development, high-quality images can hinder transmission and affect the viewing experience. Traditional vectorization methods require a significant amount of manual parameter adjus… ▽ More Manga, as a widely beloved form of entertainment around the world, have shifted from paper to electronic screens with the proliferation of handheld devices. However, as the demand for image quality increases with screen development, high-quality images can hinder transmission and affect the viewing experience. Traditional vectorization methods require a significant amount of manual parameter adjustment to process screentone. Using deep learning, lines and screentone can be automatically extracted and image resolution can be enhanced. Super-resolution can convert low-resolution images to high-resolution images while maintaining low transmission rates and providing high-quality results. However, traditional Super Resolution methods for improving manga resolution do not consider the meaning of screentone density, resulting in changes to screentone density and loss of meaning. In this paper, we aims to address this issue by first classifying the regions and lines of different screentone in the manga using deep learning algorithm, then using corresponding super-resolution models for quality enhancement based on the different classifications of each block, and finally combining them to obtain images that maintain the meaning of screentone and lines in the manga while improving image resolution. △ Less

Submitted 14 May, 2023; originally announced May 2023.

SigVIC: Spatial Importance Guided Variable-Rate Image Compression

Authors: Jiaming Liang, Meiqin Liu, Chao Yao, Chunyu Lin, Yao Zhao

Abstract: Variable-rate mechanism has improved the flexibility and efficiency of learning-based image compression that trains multiple models for different rate-distortion tradeoffs. One of the most common approaches for variable-rate is to channel-wisely or spatial-uniformly scale the internal features. However, the diversity of spatial importance is instructive for bit allocation of image compression. In… ▽ More Variable-rate mechanism has improved the flexibility and efficiency of learning-based image compression that trains multiple models for different rate-distortion tradeoffs. One of the most common approaches for variable-rate is to channel-wisely or spatial-uniformly scale the internal features. However, the diversity of spatial importance is instructive for bit allocation of image compression. In this paper, we introduce a Spatial Importance Guided Variable-rate Image Compression (SigVIC), in which a spatial gating unit (SGU) is designed for adaptively learning a spatial importance mask. Then, a spatial scaling network (SSN) takes the spatial importance mask to guide the feature scaling and bit allocation for variable-rate. Moreover, to improve the quality of decoded image, Top-K shallow features are selected to refine the decoded features through a shallow feature fusion module (SFFM). Experiments show that our method outperforms other learning-based methods (whether variable-rate or not) and traditional codecs, with storage saving and high flexibility. △ Less

Submitted 16 March, 2023; originally announced March 2023.

Comments: Accepted by IEEE ICASSP2023 (Camera Ready)

An Anchor-Free Detector for Continuous Speech Keyword Spotting

Authors: Zhiyuan Zhao, Chuanxin Tang, Chengdong Yao, Chong Luo

Abstract: Continuous Speech Keyword Spotting (CSKWS) is a task to detect predefined keywords in a continuous speech. In this paper, we regard CSKWS as a one-dimensional object detection task and propose a novel anchor-free detector, named AF-KWS, to solve the problem. AF-KWS directly regresses the center locations and lengths of the keywords through a single-stage deep neural network. In particular, AF-KWS… ▽ More Continuous Speech Keyword Spotting (CSKWS) is a task to detect predefined keywords in a continuous speech. In this paper, we regard CSKWS as a one-dimensional object detection task and propose a novel anchor-free detector, named AF-KWS, to solve the problem. AF-KWS directly regresses the center locations and lengths of the keywords through a single-stage deep neural network. In particular, AF-KWS is tailored for this speech task as we introduce an auxiliary unknown class to exclude other words from non-speech or silent background. We have built two benchmark datasets named LibriTop-20 and continuous meeting analysis keywords (CMAK) dataset for CSKWS. Evaluations on these two datasets show that our proposed AF-KWS outperforms reference schemes by a large margin, and therefore provides a decent baseline for future research. △ Less

Submitted 9 August, 2022; originally announced August 2022.

Comments: Accepted by Interspeech 2022

Revisiting Document Image Dewar** by Grid Regularization

Authors: Xiangwei Jiang, Rujiao Long, Nan Xue, Zhibo Yang, Cong Yao, Gui-Song Xia

Abstract: This paper addresses the problem of document image dewar**, which aims at eliminating the geometric distortion in document images for document digitization. Instead of designing a better neural network to approximate the optical flow fields between the inputs and outputs, we pursue the best readability by taking the text lines and the document boundaries into account from a constrained optimizat… ▽ More This paper addresses the problem of document image dewar**, which aims at eliminating the geometric distortion in document images for document digitization. Instead of designing a better neural network to approximate the optical flow fields between the inputs and outputs, we pursue the best readability by taking the text lines and the document boundaries into account from a constrained optimization perspective. Specifically, our proposed method first learns the boundary points and the pixels in the text lines and then follows the most simple observation that the boundaries and text lines in both horizontal and vertical directions should be kept after dewar** to introduce a novel grid regularization scheme. To obtain the final forward map** for dewar**, we solve an optimization problem with our proposed grid regularization. The experiments comprehensively demonstrate that our proposed approach outperforms the prior arts by large margins in terms of readability (with the metrics of Character Errors Rate and the Edit Distance) while maintaining the best image quality on the publicly-available DocUNet benchmark. △ Less

Submitted 31 March, 2022; originally announced March 2022.

Joint Routing and Charging Problem of Electric Vehicles with Incentive-aware Customers Considering Spatio-temporal Charging Prices

Authors: Canqi Yao, Shibo Chen, Mauro Salazar, Zaiyue Yang

Abstract: This paper investigates the scheduling problem of a fleet of electric vehicles, providing mobility as a service to a set of time-specified customers, where the operator needs to solve the routing and charging problem jointly for each EV. Hereby we consider incentive-aware customers and propose that the operator offers monetary incentives to customers in exchange for time flexibility. In this way,… ▽ More This paper investigates the scheduling problem of a fleet of electric vehicles, providing mobility as a service to a set of time-specified customers, where the operator needs to solve the routing and charging problem jointly for each EV. Hereby we consider incentive-aware customers and propose that the operator offers monetary incentives to customers in exchange for time flexibility. In this way, the fleet operator can achieve a routing and charging schedule with lower costs, whilst the customers receive monetary compensation for their flexibility. Specifically, we first propose a bi-level optimization model whereby the fleet operator optimizes the routing and charging schedule accounting for the spatio-temporal varying charging price, jointly with a monetary incentive to reimburse the delivery time flexibility experienced by the customers. Concurrently the customers choose their own time flexibility by minimizing their own cost. Second, we cope with the computational burden coming from this nonlinear bi-level optimization model with an accurate reformulation approach consisting of the KKT optimality conditions, a Big-M-based linearization method, and the zero duality gap of convex optimization problems. This way, we convert the proposed problem into a single-level optimization problem, which can be solved by a strengthened generalized Benders decomposition method holding a faster convergence rate than the generalized Benders decomposition method. To evaluate the effectiveness of the proposed mathematical model, we carry out numerous simulation experiments by using the VRP-REP data of Belgium. The numerical results showcase that the proposed mathematical model can reduce the delivery fees for the customers together with the cost of operation incurred by the fleet operator. △ Less

Submitted 26 May, 2022; v1 submitted 6 January, 2022; originally announced January 2022.

Comments: Submitted to TRC

Incentive-aware Electric Vehicle Routing Problem: a Bi-level Model and a Joint Solution Algorithm

Authors: Canqi Yao, Shibo Chen, Mauro Salazar, Zaiyue Yang

Abstract: Fixed pickup and delivery times can strongly limit the performance of freight transportation. Against this backdrop, fleet operators can use compensation mechanisms such as monetary incentives to buy delay time from their customers, in order to improve the fleet efficiency and ultimately minimize the costs of operation. To make the most of such an operational model, the fleet activities and the in… ▽ More Fixed pickup and delivery times can strongly limit the performance of freight transportation. Against this backdrop, fleet operators can use compensation mechanisms such as monetary incentives to buy delay time from their customers, in order to improve the fleet efficiency and ultimately minimize the costs of operation. To make the most of such an operational model, the fleet activities and the incentives should be jointly optimized accounting for the customers' reactions. Against this backdrop, this paper presents an incentive-aware electric vehicle routing scheme in which the fleet operator actively provides incentives to the customers in exchange of pickup or delivery time flexibility. Specifically, we first devise a bi-level model whereby the fleet operator optimizes the routes and charging schedules of the fleet jointly with an incentive rate to reimburse the delivery delays experienced by the customers. At the same time, the customers choose the admissible delays by minimizing a monetarily-weighted combination of the delays minus the reimbursement offered by the operator. Second, we tackle the complexity resulting from the bi-level and nonlinear problem structure with an equivalent transformation method, reformulating the problem as a single-level optimization problem that can be solved with standard mixed-integer linear programming algorithms. We demonstrate the effectiveness of our framework via extensive numerical experiments using VRP-REP data from Belgium. Our results show that by jointly optimizing routes and incentives subject to the customers' preferences, the operational costs can be reduced by up to 5%, whilst customers can save more than 30% in total delivery fees. △ Less

Submitted 24 March, 2022; v1 submitted 12 October, 2021; originally announced October 2021.

Comments: Accepted by ACC2022

Cooperative Operation of the Fleet Operator and Incentive-aware Customers in an On-demand Delivery System: A Bi-level Approach

Authors: Canqi Yao, Shibo Chen, Zaiyue Yang

Abstract: In this paper, we study the cooperative operation problem between the fleet operator and incentive-aware customers in an on-demand delivery system. Specifically, the fleet operator offers discounts on transportation costs in exchange of the delivery time flexibility of customers. In order to capture the interaction between the fleet operator and customers, a novel bi-level optimization framework i… ▽ More In this paper, we study the cooperative operation problem between the fleet operator and incentive-aware customers in an on-demand delivery system. Specifically, the fleet operator offers discounts on transportation costs in exchange of the delivery time flexibility of customers. In order to capture the interaction between the fleet operator and customers, a novel bi-level optimization framework is proposed. By exploiting the strong duality, and the KKT optimality condition of customer optimization problems, we can reformulate the bi-level optimization problem as a mixed integer nonlinear programming problem. Considering the inherent difficulties of MINLP, a computationally efficient algorithm, which combines the merits of Lagrangian dual decomposition and Benders decomposition, is devised to solve the resulting MINLP problem in a distributed manner. Finally, extensive numerical experiments demonstrate that the proposed cooperation scheme can decrease the delivery fees for the customers, and reduce the operation cost of the fleet operator at the same time, thus leading to a win-win situation for both sides. △ Less

Submitted 12 October, 2023; v1 submitted 8 September, 2021; originally announced September 2021.

Comments: Accepted for publication in IEEE Internet of Things Journal

Journal ref: IEEE Internet of Things Journal, 2023

Evacuation Problem Under the Nuclear Leakage Accident

Authors: Canqi Yao, Shibo Chen, Zaiyue Yang

Abstract: To handle the detrimental effects brought by leakage of radioactive gases at nuclear power station, we propose a bus based evacuation optimization problem. The proposed model incorporates the following four constraints, 1) the maximum dose of radiation per evacuee, 2) the limitation of bus capacity, 3) the number of evacuees at demand node (bus pickup stop), 4) evacuees balance at demand and shelt… ▽ More To handle the detrimental effects brought by leakage of radioactive gases at nuclear power station, we propose a bus based evacuation optimization problem. The proposed model incorporates the following four constraints, 1) the maximum dose of radiation per evacuee, 2) the limitation of bus capacity, 3) the number of evacuees at demand node (bus pickup stop), 4) evacuees balance at demand and shelter nodes, which is formulated as a mixed integer nonlinear programming (MINLP) problem. Then, to eliminate the difficulties of choosing a proper M value in Big-M method, a Big-M free method is employed to linearize the nonlinear terms of the MINLP problem. Finally, the resultant mixed integer linear program (MILP) problem is solvable with efficient commercial solvers such as CPLEX or Gurobi, which guarantees the optimal evacuation plan obtained. To evaluate the effectiveness of proposed evacuation model, we test our model on two different scenarios (a random one and a practical scenario). For both scenarios, our model attains executable evacuation plan within given 3600 seconds computation time. △ Less

Submitted 19 May, 2021; originally announced May 2021.

Comments: Accepted by 2021 40th Chinese Control Conference (CCC). IEEE

Joint Routing and Charging Problem of Multiple Electric Vehicles: A Fast Optimization Algorithm

Authors: Canqi Yao, Shibo Chen, Zaiyue Yang

Abstract: Logistics has gained great attentions with the prosperous development of commerce, which is often seen as the classic optimal vehicle routing problem. Meanwhile, electric vehicle (EV) has been widely used in logistic fleet to curb the emission of green house gases in recent years. Solving the optimization problem of joint routing and charging of multiple EVs is in a urgent need, whose objective fu… ▽ More Logistics has gained great attentions with the prosperous development of commerce, which is often seen as the classic optimal vehicle routing problem. Meanwhile, electric vehicle (EV) has been widely used in logistic fleet to curb the emission of green house gases in recent years. Solving the optimization problem of joint routing and charging of multiple EVs is in a urgent need, whose objective function includes charging time, charging cost, EVs travel time, usage fees of EV and revenue from serving customers. This joint problem is formulated as a mixed integer programming (MIP) problem, which, however, is NP-hard due to integer restrictions and bilinear terms from the coupling between routing and charging decisions. The main contribution of this paper lies at proposing an efficient two stage algorithm that can decompose the original MIP problem into two linear programming (LP) problems, by exploiting the exactness of LP relaxation and eliminating the coupled term. This algorithm can achieve a nearoptimal solution in polynomial time. In addition, another variant algorithm is proposed based on the two stage one, to further improve the quality of solution. △ Less

Submitted 12 May, 2021; originally announced May 2021.

Comments: Accepted by IEEE Transactions on Intelligent Transportation Systems, DOI: 10.1109/TITS.2021.3076601

Journal ref: IEEE Transactions on Intelligent Transportation Systems, 2021

Claw U-Net: A Unet-based Network with Deep Feature Concatenation for Scleral Blood Vessel Segmentation

Authors: Chang Yao, **gyu Tang, Menghan Hu, Yue Wu, Wenyi Guo, Qingli Li, Xiao-** Zhang

Abstract: Sturge-Weber syndrome (SWS) is a vascular malformation disease, and it may cause blindness if the patient's condition is severe. Clinical results show that SWS can be divided into two types based on the characteristics of scleral blood vessels. Therefore, how to accurately segment scleral blood vessels has become a significant problem in computer-aided diagnosis. In this research, we propose to co… ▽ More Sturge-Weber syndrome (SWS) is a vascular malformation disease, and it may cause blindness if the patient's condition is severe. Clinical results show that SWS can be divided into two types based on the characteristics of scleral blood vessels. Therefore, how to accurately segment scleral blood vessels has become a significant problem in computer-aided diagnosis. In this research, we propose to continuously upsample the bottom layer's feature maps to preserve image details, and design a novel Claw UNet based on UNet for scleral blood vessel segmentation. Specifically, the residual structure is used to increase the number of network layers in the feature extraction stage to learn deeper features. In the decoding stage, by fusing the features of the encoding, upsampling, and decoding parts, Claw UNet can achieve effective segmentation in the fine-grained regions of scleral blood vessels. To effectively extract small blood vessels, we use the attention mechanism to calculate the attention coefficient of each position in images. Claw UNet outperforms other UNet-based networks on scleral blood vessel image dataset. △ Less

Submitted 20 October, 2020; originally announced October 2020.

Comments: 5 pages,4 figures

Deep Learning for Post-Processing Ensemble Weather Forecasts

Authors: Peter Grönquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Shigang Li, Torsten Hoefler

Abstract: Quantifying uncertainty in weather forecasts is critical, especially for predicting extreme weather events. This is typically accomplished with ensemble prediction systems, which consist of many perturbed numerical weather simulations, or trajectories, run in parallel. These systems are associated with a high computational cost and often involve statistical post-processing steps to inexpensively i… ▽ More Quantifying uncertainty in weather forecasts is critical, especially for predicting extreme weather events. This is typically accomplished with ensemble prediction systems, which consist of many perturbed numerical weather simulations, or trajectories, run in parallel. These systems are associated with a high computational cost and often involve statistical post-processing steps to inexpensively improve their raw prediction qualities. We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. These enable the model to account for non-linear relationships that are not captured by current numerical models or post-processing methods. Applied to global data, our mixed models achieve a relative improvement in ensemble forecast skill (CRPS) of over 14%. Furthermore, we demonstrate that the improvement is larger for extreme weather events on select case studies. We also show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble. By using fewer trajectories, the computational costs of an ensemble prediction system can be reduced, allowing it to run at higher resolution and produce more accurate forecasts. △ Less

Submitted 21 September, 2020; v1 submitted 18 May, 2020; originally announced May 2020.

Transfer Learning for Automated OCTA Detection of Diabetic Retinopathy

Authors: David Le, Minhaj Alam, Cham Yao, Jennifer I. Lim, R. V. P. Chan, Devrim Toslak, Xincheng Yao

Abstract: Purpose: To test the feasibility of using deep learning for optical coherence tomography angiography (OCTA) detection of diabetic retinopathy (DR). Methods: A deep learning convolutional neural network (CNN) architecture VGG16 was employed for this study. A transfer learning process was implemented to re-train the CNN for robust OCTA classification. In order to demonstrate the feasibility of using… ▽ More Purpose: To test the feasibility of using deep learning for optical coherence tomography angiography (OCTA) detection of diabetic retinopathy (DR). Methods: A deep learning convolutional neural network (CNN) architecture VGG16 was employed for this study. A transfer learning process was implemented to re-train the CNN for robust OCTA classification. In order to demonstrate the feasibility of using this method for artificial intelligence (AI) screening of DR in clinical environments, the re-trained CNN was incorporated into a custom developed GUI platform which can be readily operated by ophthalmic personnel. Results: With last nine layers re-trained, CNN architecture achieved the best performance for automated OCTA classification. The overall accuracy of the re-trained classifier for differentiating healthy, NoDR, and NPDR was 87.27%, with 83.76% sensitivity and 90.82% specificity. The AUC metrics for binary classification of healthy, NoDR and DR were 0.97, 0.98 and 0.97, respectively. The GUI platform enabled easy validation of the method for AI screening of DR in a clinical environment. Conclusion: With a transfer leaning process to adopt the early layers for simple feature analysis and to re-train the upper layers for fine feature analysis, the CNN architecture VGG16 can be used for robust OCTA classification of healthy, NoDR, and NPDR eyes. Translational Relevance: OCTA can capture microvascular changes in early DR. A transfer learning process enables robust implementation of convolutional neural network (CNN) for automated OCTA classification of DR. △ Less

Submitted 4 October, 2019; originally announced October 2019.

Comments: 20 pages, 4 figures, 6 tables

Rethinking Irregular Scene Text Recognition

Authors: Shangbang Long, Yushuo Guan, Bingxuan Wang, Kaigui Bian, Cong Yao

Abstract: Reading text from natural images is challenging due to the great variety in text font, color, size, complex background and etc.. The perspective distortion and non-linear spatial arrangement of characters make it further difficult. While rectification based method is intuitively grounded and has pushed the envelope by far, its potential is far from being well exploited. In this paper, we present a… ▽ More Reading text from natural images is challenging due to the great variety in text font, color, size, complex background and etc.. The perspective distortion and non-linear spatial arrangement of characters make it further difficult. While rectification based method is intuitively grounded and has pushed the envelope by far, its potential is far from being well exploited. In this paper, we present a bag of tricks that prove to significantly improve the performance of rectification based method. On curved text dataset, our method achieves an accuracy of 89.6% on CUTE-80 and 76.3% on Total-Text, an improvement over previous state-of-the-art by 6.3% and 14.7% respectively. Furthermore, our combination of tricks helps us win the ICDAR 2019 Arbitrary-Shaped Text Challenge (Latin script), achieving an accuracy of 74.3% on the held-out test set. We release our code as well as data samples for further exploration at https://github.com/Jyouhou/ICDAR2019-ArT-Recognition-Alchemy △ Less

Submitted 11 November, 2019; v1 submitted 30 August, 2019; originally announced August 2019.

Comments: Technical report for participation in ICDAR2019-ArT recognition track