-
Joint State Estimation and Noise Identification Based on Variational Optimization
Authors:
Hua Lan,
Shijie Zhao,
**jie Hu,
Zengfu Wang,
**g Fu
Abstract:
In this article, the state estimation problems with unknown process noise and measurement noise covariances for both linear and nonlinear systems are considered. By formulating the joint estimation of system state and noise parameters into an optimization problem, a novel adaptive Kalman filter method based on conjugate-computation variational inference, referred to as CVIAKF, is proposed to appro…
▽ More
In this article, the state estimation problems with unknown process noise and measurement noise covariances for both linear and nonlinear systems are considered. By formulating the joint estimation of system state and noise parameters into an optimization problem, a novel adaptive Kalman filter method based on conjugate-computation variational inference, referred to as CVIAKF, is proposed to approximate the joint posterior probability density function of the latent variables. Unlike the existing adaptive Kalman filter methods utilizing variational inference in natural-parameter space, CVIAKF performs optimization in expectation-parameter space, resulting in a faster and simpler solution. Meanwhile, CVIAKF divides optimization objectives into conjugate and non-conjugate parts of nonlinear dynamical models, whereas conjugate computations and stochastic mirror-descent are applied, respectively. Remarkably, the reparameterization trick is used to reduce the variance of stochastic gradients of the non-conjugate parts. The effectiveness of CVIAKF is validated through synthetic and real-world datasets of maneuvering target tracking.
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
Classification-Aided Robust Multiple Target Tracking Using Neural Enhanced Message Passing
Authors:
Xianglong Bai,
Zengfu Wang,
Quan Pan,
Tao Yun,
Hua Lan
Abstract:
We address the challenge of tracking an unknown number of targets in strong clutter environments using measurements from a radar sensor. Leveraging the range-Doppler spectra information, we identify the measurement classes, which serve as additional information to enhance clutter rejection and data association, thus bolstering the robustness of target tracking. We first introduce a novel neural en…
▽ More
We address the challenge of tracking an unknown number of targets in strong clutter environments using measurements from a radar sensor. Leveraging the range-Doppler spectra information, we identify the measurement classes, which serve as additional information to enhance clutter rejection and data association, thus bolstering the robustness of target tracking. We first introduce a novel neural enhanced message passing approach, where the beliefs obtained by the unified message passing are fed into the neural network as additional information. The output beliefs are then utilized to refine the original beliefs. Then, we propose a classification-aided robust multiple target tracking algorithm, employing the neural enhanced message passing technique. This algorithm is comprised of three modules: a message-passing module, a neural network module, and a Dempster-Shafer module. The message-passing module is used to represent the statistical model by the factor graph and infers target kinematic states, visibility states, and data associations based on the spatial measurement information. The neural network module is employed to extract features from range-Doppler spectra and derive beliefs on whether a measurement is target-generated or clutter-generated. The Dempster-Shafer module is used to fuse the beliefs obtained from both the factor graph and the neural network. As a result, our proposed algorithm adopts a model-and-data-driven framework, effectively enhancing clutter suppression and data association, leading to significant improvements in multiple target tracking performance. We validate the effectiveness of our approach using both simulated and real data scenarios, demonstrating its capability to handle challenging tracking scenarios in practical radar applications.
△ Less
Submitted 18 October, 2023;
originally announced October 2023.
-
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
Authors:
Shilong Wu,
Chenxi Wang,
Hang Chen,
Yusheng Dai,
Chenyue Zhang,
Ruoyu Wang,
Hongbo Lan,
Jun Du,
Chin-Hui Lee,
**gdong Chen,
Shinji Watanabe,
Sabato Marco Siniscalchi,
Odette Scharenborg,
Zhong-Qiu Wang,
Jia Pan,
Jianqing Gao
Abstract:
Previous Multimodal Information based Speech Processing (MISP) challenges mainly focused on audio-visual speech recognition (AVSR) with commendable success. However, the most advanced back-end recognition systems often hit performance limits due to the complex acoustic environments. This has prompted a shift in focus towards the Audio-Visual Target Speaker Extraction (AVTSE) task for the MISP 2023…
▽ More
Previous Multimodal Information based Speech Processing (MISP) challenges mainly focused on audio-visual speech recognition (AVSR) with commendable success. However, the most advanced back-end recognition systems often hit performance limits due to the complex acoustic environments. This has prompted a shift in focus towards the Audio-Visual Target Speaker Extraction (AVTSE) task for the MISP 2023 challenge in ICASSP 2024 Signal Processing Grand Challenges. Unlike existing audio-visual speech enhance-ment challenges primarily focused on simulation data, the MISP 2023 challenge uniquely explores how front-end speech processing, combined with visual clues, impacts back-end tasks in real-world scenarios. This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments. This paper delivers a thorough overview of the task setting, dataset, and baseline system of the MISP 2023 challenge. It also includes an in-depth analysis of the challenges participants may encounter. The experimental results highlight the demanding nature of this task, and we look forward to the innovative solutions participants will bring forward.
△ Less
Submitted 15 September, 2023;
originally announced September 2023.
-
Hierarchical Metadata Information Constrained Self-Supervised Learning for Anomalous Sound Detection Under Domain Shift
Authors:
Haiyan Lan,
Qiaoxi Zhu,
Jian Guan,
Yuming Wei,
Wenwu Wang
Abstract:
Self-supervised learning methods have achieved promising performance for anomalous sound detection (ASD) under domain shift, where the type of domain shift is considered in feature learning by incorporating section IDs. However, the attributes accompanying audio files under each section, such as machine operating conditions and noise types, have not been considered, although they are also crucial…
▽ More
Self-supervised learning methods have achieved promising performance for anomalous sound detection (ASD) under domain shift, where the type of domain shift is considered in feature learning by incorporating section IDs. However, the attributes accompanying audio files under each section, such as machine operating conditions and noise types, have not been considered, although they are also crucial for characterizing domain shifts. In this paper, we present a hierarchical metadata information constrained self-supervised (HMIC) ASD method, where the hierarchical relation between section IDs and attributes is constructed, and used as constraints to obtain finer feature representation. In addition, we propose an attribute-group-center (AGC)-based method for calculating the anomaly score under the domain shift condition. Experiments are performed to demonstrate its improved performance over the state-of-the-art self-supervised methods in DCASE 2022 challenge Task 2.
△ Less
Submitted 18 December, 2023; v1 submitted 14 September, 2023;
originally announced September 2023.
-
Score-based Generative Models for Photoacoustic Image Reconstruction with Rotation Consistency Constraints
Authors:
Shangqing Tong,
Hengrong Lan,
Liming Nie,
Jianwen Luo,
Fei Gao
Abstract:
Photoacoustic tomography (PAT) is a newly emerged imaging modality which enables both high optical contrast and acoustic depth of penetration. Reconstructing images of photoacoustic tomography from limited amount of senser data is among one of the major challenges in photoacoustic imaging. Previous works based on deep learning were trained in supervised fashion, which directly map the input partia…
▽ More
Photoacoustic tomography (PAT) is a newly emerged imaging modality which enables both high optical contrast and acoustic depth of penetration. Reconstructing images of photoacoustic tomography from limited amount of senser data is among one of the major challenges in photoacoustic imaging. Previous works based on deep learning were trained in supervised fashion, which directly map the input partially known sensor data to the ground truth reconstructed from full field of view. Recently, score-based generative models played an increasingly significant role in generative modeling. Leveraging this probabilistic model, we proposed Rotation Consistency Constrained Score-based Generative Model (RCC-SGM), which recovers the PAT images by iterative sampling between Langevin dynamics and a constraint term utilizing the rotation consistency between the images and the measurements. Our proposed method can generalize to different measurement processes (32.29 PSNR with 16 measurements under random sampling, whereas 28.50 for supervised counterpart), while supervised methods need to train on specific inverse map**s.
△ Less
Submitted 23 June, 2023;
originally announced June 2023.
-
Variational Nonlinear Kalman Filtering with Unknown Process Noise Covariance
Authors:
Hua Lan,
**jie Hu,
Zengfu Wang,
Qiang Cheng
Abstract:
Motivated by the maneuvering target tracking with sensors such as radar and sonar, this paper considers the joint and recursive estimation of the dynamic state and the time-varying process noise covariance in nonlinear state space models. Due to the nonlinearity of the models and the non-conjugate prior, the state estimation problem is generally intractable as it involves integrals of general nonl…
▽ More
Motivated by the maneuvering target tracking with sensors such as radar and sonar, this paper considers the joint and recursive estimation of the dynamic state and the time-varying process noise covariance in nonlinear state space models. Due to the nonlinearity of the models and the non-conjugate prior, the state estimation problem is generally intractable as it involves integrals of general nonlinear functions and unknown process noise covariance, resulting in the posterior probability distribution functions lacking closed-form solutions. This paper presents a recursive solution for joint nonlinear state estimation and model parameters identification based on the approximate Bayesian inference principle. The stochastic search variational inference is adopted to offer a flexible, accurate, and effective approximation of the posterior distributions. We make two contributions compared to existing variational inference-based noise adaptive filtering methods. First, we introduce an auxiliary latent variable to decouple the latent variables of dynamic state and process noise covariance, thereby improving the flexibility of the posterior inference. Second, we split the variational lower bound optimization into conjugate and non-conjugate parts, whereas the conjugate terms are directly optimized that admit a closed-form solution and the non-conjugate terms are optimized by natural gradients, achieving the trade-off between inference speed and accuracy. The performance of the proposed method is verified on radar target tracking applications by both simulated and real-world data.
△ Less
Submitted 5 May, 2023;
originally announced May 2023.
-
Cross-domain Self-supervised Framework for Photoacoustic Computed Tomography Image Reconstruction
Authors:
Hengrong Lan,
Lijie Huang,
Zhiqiang Li,
**g Lv,
Jianwen Luo
Abstract:
Accurate image reconstruction is crucial for photoacoustic (PA) computed tomography (PACT). Recently, deep learning has been used to reconstruct the PA image with a supervised scheme, which requires high-quality images as ground truth labels. In practice, there are inevitable trade-offs between cost and performance since the use of more channels is an expensive strategy to access more measurements…
▽ More
Accurate image reconstruction is crucial for photoacoustic (PA) computed tomography (PACT). Recently, deep learning has been used to reconstruct the PA image with a supervised scheme, which requires high-quality images as ground truth labels. In practice, there are inevitable trade-offs between cost and performance since the use of more channels is an expensive strategy to access more measurements. Here, we propose a cross-domain unsupervised reconstruction (CDUR) strategy with a pure transformer model, which overcomes the lack of ground truth labels from limited PA measurements. The proposed approach exploits the equivariance of PACT to achieve high performance with a smaller number of channels. We implement a self-supervised reconstruction in a model-based form. Meanwhile, we also leverage the self-supervision to enforce the measurement and image consistency on three partitions of measured PA data, by randomly masking different channels. We find that dynamically masking a high proportion of the channels, e.g., 80%, yields nontrivial self-supervisors in both image and signal domains, which decrease the multiplicity of the pseudo solution to efficiently reconstruct the image from fewer PA measurements with minimum error of the image. Experimental results on in-vivo PACT dataset of mice demonstrate the potential of our unsupervised framework. In addition, our method shows a high performance (0.83 structural similarity index (SSIM) in the extreme sparse case with 13 channels), which is close to that of supervised scheme (0.77 SSIM with 16 channels). On top of all the advantages, our method may be deployed on different trainable models in an end-to-end manner.
△ Less
Submitted 20 September, 2023; v1 submitted 16 January, 2023;
originally announced January 2023.
-
Robust Multitarget Tracking in Interference Environments: A Message-Passing Approach
Authors:
Xianglong Bai,
Hua Lan,
Zengfu Wang,
Quan Pan,
Yuhang Hao,
Can Li
Abstract:
Multitarget tracking in the interference environments suffers from the nonuniform, unknown and time-varying clutter, resulting in dramatic performance deterioration. We address this challenge by proposing a robust multitarget tracking algorithm, which estimates the states of clutter and targets simultaneously by the message-passing (MP) approach. We define the non-homogeneous clutter with a finite…
▽ More
Multitarget tracking in the interference environments suffers from the nonuniform, unknown and time-varying clutter, resulting in dramatic performance deterioration. We address this challenge by proposing a robust multitarget tracking algorithm, which estimates the states of clutter and targets simultaneously by the message-passing (MP) approach. We define the non-homogeneous clutter with a finite mixture model containing a uniform component and multiple nonuniform components. The measured signal strength is utilized to estimate the mean signal-to-noise ratio (SNR) of targets and the mean clutter-to-noise ratio (CNR) of clutter, which are then used as additional feature information of targets and clutter to improve the performance of discrimination of targets from clutter. We also present a hybrid data association which can reason over correspondence between targets, clutter, and measurements. Then, a unified MP algorithm is used to infer the marginal posterior probability distributions of targets, clutter, and data association by splitting the joint probability distribution into a mean-field approximate part and a belief propagation part. As a result, a closed-loop iterative optimization of the posterior probability distribution can be obtained, which can effectively deal with the coupling between target tracking, clutter estimation and data association. Simulation results demonstrate the performance superiority and robustness of the proposed multitarget tracking algorithm compared with the probability hypothesis density (PHD) filter and the cardinalized PHD (CPHD) filter.
△ Less
Submitted 14 December, 2022;
originally announced December 2022.
-
Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search
Authors:
Zihan Wang,
Qi Meng,
HaiFeng Lan,
XinRui Zhang,
KeHao Guo,
Akshat Gupta
Abstract:
Speech emotion recognition (SER) classifies audio into emotion categories such as Happy, Angry, Fear, Disgust and Neutral. While Speech Emotion Recognition (SER) is a common application for popular languages, it continues to be a problem for low-resourced languages, i.e., languages with no pretrained speech-to-text recognition models. This paper firstly proposes a language-specific model that extr…
▽ More
Speech emotion recognition (SER) classifies audio into emotion categories such as Happy, Angry, Fear, Disgust and Neutral. While Speech Emotion Recognition (SER) is a common application for popular languages, it continues to be a problem for low-resourced languages, i.e., languages with no pretrained speech-to-text recognition models. This paper firstly proposes a language-specific model that extract emotional information from multiple pre-trained speech models, and then designs a multi-domain model that simultaneously performs SER for various languages. Our multidomain model employs a multi-gating mechanism to generate unique weighted feature combination for each language, and also searches for specific neural network structure for each language through a neural architecture search module. In addition, we introduce a contrastive auxiliary loss to build more separable representations for audio data. Our experiments show that our model raises the state-of-the-art accuracy by 3% for German and 14.3% for French.
△ Less
Submitted 15 November, 2022; v1 submitted 31 October, 2022;
originally announced November 2022.
-
Local Information Assisted Attention-free Decoder for Audio Captioning
Authors:
Feiyang Xiao,
Jian Guan,
Haiyan Lan,
Qiaoxi Zhu,
Wenwu Wang
Abstract:
Automated audio captioning aims to describe audio data with captions using natural language. Existing methods often employ an encoder-decoder structure, where the attention-based decoder (e.g., Transformer decoder) is widely used and achieves state-of-the-art performance. Although this method effectively captures global information within audio data via the self-attention mechanism, it may ignore…
▽ More
Automated audio captioning aims to describe audio data with captions using natural language. Existing methods often employ an encoder-decoder structure, where the attention-based decoder (e.g., Transformer decoder) is widely used and achieves state-of-the-art performance. Although this method effectively captures global information within audio data via the self-attention mechanism, it may ignore the event with short time duration, due to its limitation in capturing local information in an audio signal, leading to inaccurate prediction of captions. To address this issue, we propose a method using the pretrained audio neural networks (PANNs) as the encoder and local information assisted attention-free Transformer (LocalAFT) as the decoder. The novelty of our method is in the proposal of the LocalAFT decoder, which allows local information within an audio signal to be captured while retaining the global information. This enables the events of different duration, including short duration, to be captured for more precise caption generation. Experiments show that our method outperforms the state-of-the-art methods in Task 6 of the DCASE 2021 Challenge with the standard attention-based decoder for caption generation.
△ Less
Submitted 3 July, 2022; v1 submitted 10 January, 2022;
originally announced January 2022.
-
Deep Learning Adapted Acceleration for Limited-view Photoacoustic Computed Tomography
Authors:
Hengrong Lan,
Jiali Gong,
Fei Gao
Abstract:
Photoacoustic imaging (PAI) is a non-invasive imaging modality that detects the ultrasound signal generated from tissue with light excitation. Photoacoustic computed tomography (PACT) uses unfocused large-area light to illuminate the target with ultrasound transducer array for PA signal detection. Limited-view issue could cause a low-quality image in PACT due to the limitation of geometric conditi…
▽ More
Photoacoustic imaging (PAI) is a non-invasive imaging modality that detects the ultrasound signal generated from tissue with light excitation. Photoacoustic computed tomography (PACT) uses unfocused large-area light to illuminate the target with ultrasound transducer array for PA signal detection. Limited-view issue could cause a low-quality image in PACT due to the limitation of geometric condition. The model-based method is used to resolve this problem, which contains different regularization. To adapt fast and high-quality reconstruction of limited-view PA data, in this paper, a model-based method that combines the mathematical variational model with deep learning is proposed to speed up and regularize the unrolled procedure of reconstruction. A deep neural network is designed to adapt the step of the gradient updated term of data consistency in the gradient descent procedure, which can obtain a high-quality PA image only with a few iterations. Note that all parameters and priors are automatically learned during the offline training stage. In experiments, we show that this method outperforms the other methods with half-view (180 degrees) simulation and real data. The comparison of different model-based methods show that our proposed scheme has superior performances (over 0.05 for SSIM) with same iteration (3 times) steps. Furthermore, an unseen data is used to validate the generalization of different methods. Finally, we find that our method obtains superior results (0.94 value of SSIM for in vivo) with a high robustness and accelerated reconstruction.
△ Less
Submitted 7 November, 2021;
originally announced November 2021.
-
Photoacoustic-monitored laser treatment for tattoo removal: a feasibility study
Authors:
Yiyun Wang,
Daohuai Jiang,
Hengrong Lan,
Feng Gao,
Fei Gao
Abstract:
Skin blemishes and diseases have attracted increasing research interest in recent decades, due to their growing frequency of occurrence and the severity of related diseases. Various laser treatment approaches have been introduced for the alleviation and removal of skin pigmentation. The treatments' effects highly depend on the experience and prognosis of the relevant operators. But, the operation…
▽ More
Skin blemishes and diseases have attracted increasing research interest in recent decades, due to their growing frequency of occurrence and the severity of related diseases. Various laser treatment approaches have been introduced for the alleviation and removal of skin pigmentation. The treatments' effects highly depend on the experience and prognosis of the relevant operators. But, the operation process lacks real-time feedback, which may directly reflect the extent of the treatment. In this manuscript, we report a photoacoustic-guided laser treatment method with a feasibility study, specifically for laser treatment targeting the tattoo's removal. The results well validated the feasibility of the proposed method through the experiments on phantoms and ex vivo pig skin samples.
△ Less
Submitted 25 May, 2021;
originally announced May 2021.
-
AS-Net: Fast Photoacoustic Reconstruction with Multi-feature Fusion from Sparse Data
Authors:
Mengjie Guo,
Hengrong Lan,
Changchun Yang,
Fei Gao
Abstract:
Photoacoustic (PA) imaging is a biomedical imaging modality capable of acquiring high-contrast images of optical absorption at depths much greater than traditional optical imaging techniques. However, practical instrumentation and geometry limit the number of available acoustic sensors surrounding the imaging target, which results in the sparsity of sensor data. Conventional PA image reconstructio…
▽ More
Photoacoustic (PA) imaging is a biomedical imaging modality capable of acquiring high-contrast images of optical absorption at depths much greater than traditional optical imaging techniques. However, practical instrumentation and geometry limit the number of available acoustic sensors surrounding the imaging target, which results in the sparsity of sensor data. Conventional PA image reconstruction methods give severe artifacts when they are applied directly to the sparse PA data. In this paper, we firstly propose to employ a novel signal processing method to make sparse PA raw data more suitable for the neural network, concurrently speeding up image reconstruction. Then we propose Attention Steered Network (AS-Net) for PA reconstruction with multi-feature fusion. AS-Net is validated on different datasets, including simulated photoacoustic data from fundus vasculature phantoms and experimental data from in vivo fish and mice. Notably, the method is also able to eliminate some artifacts present in the ground truth for in vivo data. Results demonstrated that our method provides superior reconstructions at a faster speed.
△ Less
Submitted 31 May, 2021; v1 submitted 21 January, 2021;
originally announced January 2021.
-
Limited-view Photoacoustic Imaging Reconstruction With Dual Domain Inputs Under Mutual Information Constraint
Authors:
Jiadong Zhang,
Hengrong Lan,
Changchun Yang,
Shanshan Guo,
Feng Gao,
Fei Gao
Abstract:
Based on photoacoustic effect, photoacoustic tomography is develo** very fast in recent years, and becoming an important imaging tool for both preclinical and clinical studies. With enough ultrasound transducers placed around the biological tissue, PAT can provide both deep penetration and high image contrast by hybrid usage of light and sound. However, considering space and measurement environm…
▽ More
Based on photoacoustic effect, photoacoustic tomography is develo** very fast in recent years, and becoming an important imaging tool for both preclinical and clinical studies. With enough ultrasound transducers placed around the biological tissue, PAT can provide both deep penetration and high image contrast by hybrid usage of light and sound. However, considering space and measurement environmental limitations, transducers are always placed in a limited-angle way, which means that the other side without transducer coverage suffers severe information loss. With conventional image reconstruction algorithms, the limited-view tissue induces artifacts and information loss, which may cause doctors misdiagnosis or missed diagnosis. In order to solve limited-view PA imaging reconstruction problem, we propose to use both time domain and frequency domain reconstruction algorithms to get delay-and-sum (DAS) image inputs and k-space image inputs. These dual domain images share nearly same texture information but different artifact information, which can teach network how to distinguish these two kinds of information at input level. In this paper, we propose Dual Domain Unet (DuDoUnet) with specially designed Information Sharing Block (ISB), which can further share two domains' information and distinguish artifacts. Besides, we use mutual information (MI) with an auxiliary network, whose inputs and outputs are both ground truth, to compensate prior knowledge of limited-view PA inputs. The proposed method is verified with a public clinical database, and shows superior results with SSIM = 93.5622% and PSNR = 20.8859.
△ Less
Submitted 11 November, 2020;
originally announced November 2020.
-
Deep Learning Enables Robust and Precise Light Focusing on Treatment Needs
Authors:
Changchun Yang,
Hengrong Lan,
Fei Gao
Abstract:
If light passes through the body tissues, focusing only on areas where treatment needs, such as tumors, will revolutionize many biomedical imaging and therapy technologies. So how to focus light through deep inhomogeneous tissues overcoming scattering is Holy Grail in biomedical areas. In this paper, we use deep learning to learn and accelerate the process of phase pre-compensation using wavefront…
▽ More
If light passes through the body tissues, focusing only on areas where treatment needs, such as tumors, will revolutionize many biomedical imaging and therapy technologies. So how to focus light through deep inhomogeneous tissues overcoming scattering is Holy Grail in biomedical areas. In this paper, we use deep learning to learn and accelerate the process of phase pre-compensation using wavefront sha**. We present an approach (LoftGAN, light only focuses on treatment needs) for learning the relationship between phase domain X and speckle domain Y . Our goal is not just to learn an inverse map** F:Y->X such that we can know the corresponding X needed for imaging Y like most work, but also to make focusing that is susceptible to disturbances more robust and precise by ensuring that the phase obtained can be forward mapped back to speckle. So we introduce different constraints to enforce F(Y)=X and H(F(Y))=Y with the transmission map** H:X->Y. Both simulation and physical experiments are performed to investigate the effects of light focusing to demonstrate the effectiveness of our method and comparative experiments prove the crucial improvement of robustness and precision. Codes are available at https://github.com/ChangchunYang/LoftGAN.
△ Less
Submitted 16 August, 2020;
originally announced August 2020.
-
Deep learning for photoacoustic imaging: a survey
Authors:
Changchun Yang,
Hengrong Lan,
Feng Gao,
Fei Gao
Abstract:
Machine learning has been developed dramatically and witnessed a lot of applications in various fields over the past few years. This boom originated in 2009, when a new model emerged, that is, the deep artificial neural network, which began to surpass other established mature models on some important benchmarks. Later, it was widely used in academia and industry. Ranging from image analysis to nat…
▽ More
Machine learning has been developed dramatically and witnessed a lot of applications in various fields over the past few years. This boom originated in 2009, when a new model emerged, that is, the deep artificial neural network, which began to surpass other established mature models on some important benchmarks. Later, it was widely used in academia and industry. Ranging from image analysis to natural language processing, it fully exerted its magic and now become the state-of-the-art machine learning models. Deep neural networks have great potential in medical imaging technology, medical data analysis, medical diagnosis and other healthcare issues, and is promoted in both pre-clinical and even clinical stages. In this review, we performed an overview of some new developments and challenges in the application of machine learning to medical image analysis, with a special focus on deep learning in photoacoustic imaging. The aim of this review is threefold: (i) introducing deep learning with some important basics, (ii) reviewing recent works that apply deep learning in the entire ecological chain of photoacoustic imaging, from image reconstruction to disease diagnosis, (iii) providing some open source materials and other resources for researchers interested in applying deep learning to photoacoustic imaging.
△ Less
Submitted 1 December, 2020; v1 submitted 10 August, 2020;
originally announced August 2020.
-
OTHR multitarget tracking with a GMRF model of ionospheric parameters
Authors:
Zhen Guo,
Zengfu Wang,
Hua Lan,
Quan Pan,
Kun Lu
Abstract:
The ionosphere is the propagation medium for radio waves transmitted by an over-the-horizon radar (OTHR). Ionospheric parameters, typically, virtual ionospheric heights (VIHs), are required to perform coordinate registration for OTHR multitarget tracking and localization. The inaccuracy of ionospheric parameters has a significant deleterious effect on the target localization of OTHR. Therefore, to…
▽ More
The ionosphere is the propagation medium for radio waves transmitted by an over-the-horizon radar (OTHR). Ionospheric parameters, typically, virtual ionospheric heights (VIHs), are required to perform coordinate registration for OTHR multitarget tracking and localization. The inaccuracy of ionospheric parameters has a significant deleterious effect on the target localization of OTHR. Therefore, to improve the localization accuracy of OTHR, it is important to develop accurate models and estimation methods of ionospheric parameters and the corresponding target tracking algorithms. In this paper, we consider the variation of the ionosphere with location and the spatial correlation of the ionosphere in OTHR target tracking. We use a Gaussian Markov random field (GMRF) to model the VIHs, providing a more accurate representation of the VIHs for OTHR target tracking. Based on expectation-conditional maximization and GMRF modeling of the VIHs, we propose a novel joint optimization solution, called ECM-GMRF, to perform target state estimation, multipath data association and VIHs estimation simultaneously. In ECM-GMRF, the measurements from both ionosondes and OTHR are exploited to estimate the VIHs, leading to a better estimation of the VIHs which improves the accuracy of data association and target state estimation, and vice versa. The simulation indicates the effectiveness of the proposed algorithm.
△ Less
Submitted 5 May, 2020;
originally announced May 2020.
-
Measurement-Level Fusion for OTHR Network Using Message Passing
Authors:
Hua Lan,
Zengfu Wang,
Xianglong Bai,
Quan Pan,
Kun Lu
Abstract:
Tracking an unknown number of targets based on multipath measurements provided by an over-the-horizon radar (OTHR) network with a statistical ionospheric model is complicated, which requires solving four subproblems: target detection, target tracking, multipath data association and ionospheric height identification. A joint solution is desired since the four subproblems are highly correlated, but…
▽ More
Tracking an unknown number of targets based on multipath measurements provided by an over-the-horizon radar (OTHR) network with a statistical ionospheric model is complicated, which requires solving four subproblems: target detection, target tracking, multipath data association and ionospheric height identification. A joint solution is desired since the four subproblems are highly correlated, but suffering from the intractable inference problem of high-dimensional latent variables. In this paper, a unified message passing approach, combining belief propagation (BP) and mean-field (MF) approximation, is developed for simplifying the intractable inference. Based upon the factor graph corresponding to a factorization of the joint probability distribution function (PDF) of the latent variables and a choice for a separation of this factorization into BP region and MF region, the posterior PDFs of continuous latent variables including target kinematic state, target visibility state, and ionospheric height, are approximated by MF due to its simple MP update rules for conjugate-exponential models. With regard to discrete multipath data association which contains one-to-one frame (hard) constraints, its PDF is approximated by loopy BP. Finally, the approximated posterior PDFs are updated iteratively in a closed-loop manner, which is effective for dealing with the coupling issue among target detection, target tracking, multipath data association, and ionospheric height identification. Meanwhile, the proposed approach has the measurement-level fusion architecture due to the direct processing of the raw multipath measurements from an OTHR network, which is benefit to improving target tracking performance. Its performance is demonstrated on a simulated OTHR network multitarget tracking scenario.
△ Less
Submitted 3 April, 2020; v1 submitted 22 March, 2020;
originally announced March 2020.
-
Portable probe design for photoacoustic imaging in vivo
Authors:
Yongjian Zhao,
Shaohui Yu,
Luyao Zhu,
Hengrong Lan,
**wei Li,
Jianfeng Li,
Fei Gao
Abstract:
A low-cost adjustable illumination scheme for hand-held photoacoustic imaging probe is presented, manufactured and tested in this paper. Compared with traditional photoacoustic probe design, it has the following advantages: (1) Different excitation modes can be selected as needed. By tuning control parameters, it can achieve bright-field, dark-field, and hybrid field light illumination schemes. (2…
▽ More
A low-cost adjustable illumination scheme for hand-held photoacoustic imaging probe is presented, manufactured and tested in this paper. Compared with traditional photoacoustic probe design, it has the following advantages: (1) Different excitation modes can be selected as needed. By tuning control parameters, it can achieve bright-field, dark-field, and hybrid field light illumination schemes. (2) The spot-adjustable unit (SAU) specifically designed for beam expansion, together with a water tank for transmitting ultrasonic waves, enable the device to break through the constraints of the transfer medium and is more widely used. The beam-expansion experiment is conducted to verify the function of SAU. After that, we built a PAT system based on our newly designed apparatus. Phantom and in vivo experimental results show different performance in different illumination schemes
△ Less
Submitted 5 February, 2020;
originally announced February 2020.
-
Deep Learning Enabled Real-Time Photoacoustic Tomography System via Single Data Acquisition Channel
Authors:
Hengrong Lan,
Daohuai Jiang,
Feng Gao,
Fei Gao
Abstract:
Photoacoustic computed tomography (PACT) combines the optical contrast of optical imaging and the penetrability of sonography. In this work, we develop a novel PACT system to provide real-time imaging, which is achieved by a 120-elements ultrasound array only using a single data acquisition (DAQ) channel. To reduce the channel number of DAQ, we superimpose 30 nearby channels' signals together in t…
▽ More
Photoacoustic computed tomography (PACT) combines the optical contrast of optical imaging and the penetrability of sonography. In this work, we develop a novel PACT system to provide real-time imaging, which is achieved by a 120-elements ultrasound array only using a single data acquisition (DAQ) channel. To reduce the channel number of DAQ, we superimpose 30 nearby channels' signals together in the analog domain, and shrinking to 4 channels of data (120/30=4). Furthermore, a four-to-one delay-line module is designed to combine these four channels' data into one channel before entering the single-channel DAQ, followed by decoupling the signals after data acquisition. To reconstruct the image from four superimposed 30-channels'PA signals, we train a dedicated deep learning model to reconstruct the final PA image. In this paper, we present the preliminary results of phantom and in-vivo experiments, which manifests its robust real-time imaging performance. The significance of this novel PACT system is that it dramatically reduces the cost of multi-channel DAQ module (from 120 channels to 1 channel), paving the way to a portable, low-cost and real-time PACT system.
△ Less
Submitted 6 May, 2021; v1 submitted 21 January, 2020;
originally announced January 2020.
-
A Message Passing Approach for Multiple Maneuvering Target Tracking
Authors:
Hua Lan,
Jirong Ma,
Zengfu Wang,
Quan Pan,
Xiong Xu
Abstract:
This paper considers the problem of detecting and tracking multiple maneuvering targets, which suffers from the intractable inference of high-dimensional latent variables that include target kinematic state, target visibility state, motion mode-model association, and data association. A unified message passing algorithm that combines belief propagation (BP) and mean-field (MF) approximation is pro…
▽ More
This paper considers the problem of detecting and tracking multiple maneuvering targets, which suffers from the intractable inference of high-dimensional latent variables that include target kinematic state, target visibility state, motion mode-model association, and data association. A unified message passing algorithm that combines belief propagation (BP) and mean-field (MF) approximation is proposed for simplifying the intractable inference. By assuming conjugate-exponential priors for target kinematic state, target visibility state, and motion mode-model association, the MF approximation decouples the joint inference of target kinematic state, target visibility state, motion mode-model association into individual low-dimensional inference, yielding simple message passing update equations. The BP is exploited to approximate the probabilities of data association events since it is compatible with hard constraints. Finally, the approximate posterior probability distributions are updated iteratively in a closed-loop manner, which is effective for dealing with the coupling issue between the estimations of target kinematic state and target visibility state and decisions on motion mode-model association and data association. The performance of the proposed algorithm is demonstrated by comparing with the well-known multiple maneuvering target tracking algorithms, including interacting multiple model joint probabilistic data association, interacting multiple model hypothesis-oriented multiple hypothesis tracker and multiple model generalized labeled multi-Bernoulli.
△ Less
Submitted 18 November, 2019;
originally announced November 2019.
-
Verification of infinite-step and K-step opacity Using Petri Nets
Authors:
Hao Lan,
Yin Tong,
** Guo,
Carla Seatzu
Abstract:
This paper addresses the problem of infinite-step opacity and K-step opacity of discrete event systems modeled with Petri nets. A Petri net system is said to be infinite-step/K-step opaque if all its secret states remains opaque to an intruder for any instant within infinite/K steps. In other words, the intruder is never able to ascertain that the system used to be in a secrete state within infini…
▽ More
This paper addresses the problem of infinite-step opacity and K-step opacity of discrete event systems modeled with Petri nets. A Petri net system is said to be infinite-step/K-step opaque if all its secret states remains opaque to an intruder for any instant within infinite/K steps. In other words, the intruder is never able to ascertain that the system used to be in a secrete state within infinite/K steps based on its observation of the systems evolution. Based on the notion of basis reachability and the twoway observer, an efficient approach to verify infinite-step opacity and K-step opacity is proposed.
△ Less
Submitted 9 September, 2019;
originally announced September 2019.
-
Verification of Detectability Using Petri Nets and Detector
Authors:
Hao Lan,
Yin Tong,
** Guo,
Carla Seatzu
Abstract:
Detectability describes the property of a system to uniquely determine, after a finite number of observations, the current and subsequent states. In this paper, to reduce the complexity of checking the detectability properties in the framework of bounded labeled Petri nets, we use a new tool, which is called detector, to verifying the strong detectability and periodically strong detectability. Fir…
▽ More
Detectability describes the property of a system to uniquely determine, after a finite number of observations, the current and subsequent states. In this paper, to reduce the complexity of checking the detectability properties in the framework of bounded labeled Petri nets, we use a new tool, which is called detector, to verifying the strong detectability and periodically strong detectability. First, an approach, which is based on the reachable graph and its detector, is proposed. Then, we develop a novel approach which is based on the analysis of the detector of the basis reachability graph. Without computing the whole reachability space, and without building the observer, the proposed approaches are more efficient.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Y-Net: A Hybrid Deep Learning Reconstruction Framework for Photoacoustic Imaging in vivo
Authors:
Hengrong Lan,
Daohuai Jiang,
Changchun Yang,
Fei Gao
Abstract:
Photoacoustic imaging (PAI) is an emerging non-invasive imaging modality combining the advantages of deep ultrasound penetration and high optical contrast. Image reconstruction is an essential topic in PAI, which is unfortunately an ill-posed problem due to the complex and unknown optical/acoustic parameters in tissue. Conventional algorithms used in PAI (e.g., delay-and-sum) provide a fast soluti…
▽ More
Photoacoustic imaging (PAI) is an emerging non-invasive imaging modality combining the advantages of deep ultrasound penetration and high optical contrast. Image reconstruction is an essential topic in PAI, which is unfortunately an ill-posed problem due to the complex and unknown optical/acoustic parameters in tissue. Conventional algorithms used in PAI (e.g., delay-and-sum) provide a fast solution while many artifacts remain, especially for linear array probe with limited-view issue. Convolutional neural network (CNN) has shown state-of-the-art results in computer vision, and more and more work based on CNN has been studied in medical image processing recently. In this paper, we present a non-iterative scheme filling the gap between existing direct-processing and post-processing methods, and propose a new framework Y-Net: a CNN architecture to reconstruct the PA image by optimizing both raw data and beamformed images once. The network connected two encoders with one decoder path, which optimally utilizes more information from raw data and beamformed image. The results of the test set showed good performance compared with conventional reconstruction algorithms and other deep learning methods. Our method is also validated with experiments both in-vitro and in vivo, which still performs better than other existing methods. The proposed Y-Net architecture also has high potential in medical image reconstruction for other imaging modalities beyond PAI.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
-
Verification of Detectability in Petri Nets Using Verifier Nets
Authors:
Hao Lan,
Yin Tong,
Carla Seatzu,
** Guo
Abstract:
Detectability describes the property of a system whose current and the subsequent states can be uniquely determined after a finite number of observations. In this paper, we developed a novel approach to verifying strong detectability and periodically strong detectability of bounded labeled Petri nets. Our approach is based on the analysis of the basis reachability graph of a special Petri net, cal…
▽ More
Detectability describes the property of a system whose current and the subsequent states can be uniquely determined after a finite number of observations. In this paper, we developed a novel approach to verifying strong detectability and periodically strong detectability of bounded labeled Petri nets. Our approach is based on the analysis of the basis reachability graph of a special Petri net, called Verifier Net, that is built from the Petri net model of the given system. Without computing the whole reachability space and without enumerating all the markings, the proposed approaches are more efficient.
△ Less
Submitted 21 March, 2019;
originally announced March 2019.
-
Verification of C-detectability Using Petri Nets
Authors:
Hao Lan,
Yin Tong,
** Guo,
Carla Seatzu
Abstract:
Detectability describes the property of an system whose current and the subsequent states can be uniquely determined after a finite number of observations. In this paper, we relax detectability to C-detectability that only requires a given set of crucial states can be distinguished from other states. Four types of C-detectability: strong C-detectability, weak C-detectability, periodically strong C…
▽ More
Detectability describes the property of an system whose current and the subsequent states can be uniquely determined after a finite number of observations. In this paper, we relax detectability to C-detectability that only requires a given set of crucial states can be distinguished from other states. Four types of C-detectability: strong C-detectability, weak C-detectability, periodically strong C-detectability, and periodically weak C-detectability are defined in the framework of labeled Petri nets, which have larger modeling power than finite automata. Moreover, based on the notion of basis markings, the approaches are developed to verify the four C-detectability of a bounded labeled Petri net system. Without computing the whole reachability space and without enumerating all the markings consistent with an observation, the proposed approaches are more efficient.
△ Less
Submitted 19 March, 2019;
originally announced March 2019.
