Showing 1–16 of 16 results for author: Do, T T

Interpretable Lightweight Transformer via Unrolling of Learned Graph Smoothness Priors

Authors: Tam Thuc Do, Parham Eftekhar, Seyed Alireza Hosseini, Gene Cheung, Philip Chou

Abstract: We build interpretable and lightweight transformer-like neural networks by unrolling iterative optimization algorithms that minimize graph smoothness priors -- the quadratic graph Laplacian regularizer (GLR) and the $\ell_1$-norm graph total variation (GTV) -- subject to an interpolation constraint. The crucial insight is that a normalized signal-dependent graph learning module amounts to a varian… ▽ More We build interpretable and lightweight transformer-like neural networks by unrolling iterative optimization algorithms that minimize graph smoothness priors -- the quadratic graph Laplacian regularizer (GLR) and the $\ell_1$-norm graph total variation (GTV) -- subject to an interpolation constraint. The crucial insight is that a normalized signal-dependent graph learning module amounts to a variant of the basic self-attention mechanism in conventional transformers. Unlike "black-box" transformers that require learning of large key, query and value matrices to compute scaled dot products as affinities and subsequent output embeddings, resulting in huge parameter sets, our unrolled networks employ shallow CNNs to learn low-dimensional features per node to establish pairwise Mahalanobis distances and construct sparse similarity graphs. At each layer, given a learned graph, the target interpolated signal is simply a low-pass filtered output derived from the minimization of an assumed graph smoothness prior, leading to a dramatic reduction in parameter count. Experiments for two image interpolation applications verify the restoration performance, parameter efficiency and robustness to covariate shift of our graph-based unrolled networks compared to conventional transformers. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Learned Nonlinear Predictor for Critically Sampled 3D Point Cloud Attribute Compression

Authors: Tam Thuc Do, Philip A. Chou, Gene Cheung

Abstract: We study 3D point cloud attribute compression via a volumetric approach: assuming point cloud geometry is known at both encoder and decoder, parameters $θ$ of a continuous attribute function $f: \mathbb{R}^3 \mapsto \mathbb{R}$ are quantized to $\hatθ$ and encoded, so that discrete samples $f_{\hatθ}(\mathbf{x}_i)$ can be recovered at known 3D points $\mathbf{x}_i \in \mathbb{R}^3$ at the decoder.… ▽ More We study 3D point cloud attribute compression via a volumetric approach: assuming point cloud geometry is known at both encoder and decoder, parameters $θ$ of a continuous attribute function $f: \mathbb{R}^3 \mapsto \mathbb{R}$ are quantized to $\hatθ$ and encoded, so that discrete samples $f_{\hatθ}(\mathbf{x}_i)$ can be recovered at known 3D points $\mathbf{x}_i \in \mathbb{R}^3$ at the decoder. Specifically, we consider a nested sequences of function subspaces $\mathcal{F}^{(p)}_{l_0} \subseteq \cdots \subseteq \mathcal{F}^{(p)}_L$, where $\mathcal{F}_l^{(p)}$ is a family of functions spanned by B-spline basis functions of order $p$, $f_l^*$ is the projection of $f$ on $\mathcal{F}_l^{(p)}$ and encoded as low-pass coefficients $F_l^*$, and $g_l^*$ is the residual function in orthogonal subspace $\mathcal{G}_l^{(p)}$ (where $\mathcal{G}_l^{(p)} \oplus \mathcal{F}_l^{(p)} = \mathcal{F}_{l+1}^{(p)}$) and encoded as high-pass coefficients $G_l^*$. In this paper, to improve coding performance over [1], we study predicting $f_{l+1}^*$ at level $l+1$ given $f_l^*$ at level $l$ and encoding of $G_l^*$ for the $p=1$ case (RAHT($1$)). For the prediction, we formalize RAHT(1) linear prediction in MPEG-PCC in a theoretical framework, and propose a new nonlinear predictor using a polynomial of bilateral filter. We derive equations to efficiently compute the critically sampled high-pass coefficients $G_l^*$ amenable to encoding. We optimize parameters in our resulting feed-forward network on a large training set of point clouds by minimizing a rate-distortion Lagrangian. Experimental results show that our improved framework outperformed the MPEG G-PCC predictor by $11$ to $12\%$ in bit rate reduction. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Blockwise Principal Component Analysis for monotone missing data imputation and dimensionality reduction

Authors: Tu T. Do, Mai Anh Vu, Tuan L. Vo, Hoang Thien Ly, Thu Nguyen, Steven A. Hicks, Michael A. Riegler, Pål Halvorsen, Binh T. Nguyen

Abstract: Monotone missing data is a common problem in data analysis. However, imputation combined with dimensionality reduction can be computationally expensive, especially with the increasing size of datasets. To address this issue, we propose a Blockwise principal component analysis Imputation (BPI) framework for dimensionality reduction and imputation of monotone missing data. The framework conducts Pri… ▽ More Monotone missing data is a common problem in data analysis. However, imputation combined with dimensionality reduction can be computationally expensive, especially with the increasing size of datasets. To address this issue, we propose a Blockwise principal component analysis Imputation (BPI) framework for dimensionality reduction and imputation of monotone missing data. The framework conducts Principal Component Analysis (PCA) on the observed part of each monotone block of the data and then imputes on merging the obtained principal components using a chosen imputation technique. BPI can work with various imputation techniques and can significantly reduce imputation time compared to conducting dimensionality reduction after imputation. This makes it a practical and efficient approach for large datasets with monotone missing data. Our experiments validate the improvement in speed. In addition, our experiments also show that while applying MICE imputation directly on missing data may not yield convergence, applying BPI with MICE for the data may lead to convergence. △ Less

Submitted 10 January, 2024; v1 submitted 10 May, 2023; originally announced May 2023.

Volumetric Attribute Compression for 3D Point Clouds using Feedforward Network with Geometric Attention

Authors: Tam Thuc Do, Philip A. Chou, Gene Cheung

Abstract: We study 3D point cloud attribute compression using a volumetric approach: given a target volumetric attribute function $f : \mathbb{R}^3 \rightarrow \mathbb{R}$, we quantize and encode parameter vector $θ$ that characterizes $f$ at the encoder, for reconstruction $f_{\hatθ}(\mathbf{x})$ at known 3D points $\mathbf{x}$'s at the decoder. Extending a previous work Region Adaptive Hierarchical Transf… ▽ More We study 3D point cloud attribute compression using a volumetric approach: given a target volumetric attribute function $f : \mathbb{R}^3 \rightarrow \mathbb{R}$, we quantize and encode parameter vector $θ$ that characterizes $f$ at the encoder, for reconstruction $f_{\hatθ}(\mathbf{x})$ at known 3D points $\mathbf{x}$'s at the decoder. Extending a previous work Region Adaptive Hierarchical Transform (RAHT) that employs piecewise constant functions to span a nested sequence of function spaces, we propose a feedforward linear network that implements higher-order B-spline bases spanning function spaces without eigen-decomposition. Feedforward network architecture means that the system is amenable to end-to-end neural learning. The key to our network is space-varying convolution, similar to a graph operator, whose weights are computed from the known 3D geometry for normalization. We show that the number of layers in the normalization at the encoder is equivalent to the number of terms in a matrix inverse Taylor series. Experimental results on real-world 3D point clouds show up to 2-3 dB gain over RAHT in energy compaction and 20-30% bitrate reduction. △ Less

Submitted 1 April, 2023; originally announced April 2023.

Conditional expectation with regularization for missing data imputation

Authors: Mai Anh Vu, Thu Nguyen, Tu T. Do, Nhan Phan, Nitesh V. Chawla, Pål Halvorsen, Michael A. Riegler, Binh T. Nguyen

Abstract: Missing data frequently occurs in datasets across various domains, such as medicine, sports, and finance. In many cases, to enable proper and reliable analyses of such data, the missing values are often imputed, and it is necessary that the method used has a low root mean square error (RMSE) between the imputed and the true values. In addition, for some critical applications, it is also often a re… ▽ More Missing data frequently occurs in datasets across various domains, such as medicine, sports, and finance. In many cases, to enable proper and reliable analyses of such data, the missing values are often imputed, and it is necessary that the method used has a low root mean square error (RMSE) between the imputed and the true values. In addition, for some critical applications, it is also often a requirement that the imputation method is scalable and the logic behind the imputation is explainable, which is especially difficult for complex methods that are, for example, based on deep learning. Based on these considerations, we propose a new algorithm named "conditional Distribution-based Imputation of Missing Values with Regularization" (DIMV). DIMV operates by determining the conditional distribution of a feature that has missing entries, using the information from the fully observed features as a basis. As will be illustrated via experiments in the paper, DIMV (i) gives a low RMSE for the imputed values compared to state-of-the-art methods; (ii) fast and scalable; (iii) is explainable as coefficients in a regression model, allowing reliable and trustable analysis, makes it a suitable choice for critical domains where understanding is important such as in medical fields, finance, etc; (iv) can provide an approximated confidence region for the missing values in a given sample; (v) suitable for both small and large scale data; (vi) in many scenarios, does not require a huge number of parameters as deep learning approaches; (vii) handle multicollinearity in imputation effectively; and (viii) is robust to the normally distributed assumption that its theoretical grounds rely on. △ Less

Submitted 11 September, 2023; v1 submitted 2 February, 2023; originally announced February 2023.

Hybrid Model-based / Data-driven Graph Transform for Image Coding

Authors: Saghar Bagheri, Tam Thuc Do, Gene Cheung, Antonio Ortega

Abstract: Transform coding to sparsify signal representations remains crucial in an image compression pipeline. While the Karhunen-Loève transform (KLT) computed from an empirical covariance matrix $\bar{C}$ is theoretically optimal for a stationary process, in practice, collecting sufficient statistics from a non-stationary image to reliably estimate $\bar{C}$ can be difficult. In this paper, to encode an… ▽ More Transform coding to sparsify signal representations remains crucial in an image compression pipeline. While the Karhunen-Loève transform (KLT) computed from an empirical covariance matrix $\bar{C}$ is theoretically optimal for a stationary process, in practice, collecting sufficient statistics from a non-stationary image to reliably estimate $\bar{C}$ can be difficult. In this paper, to encode an intra-prediction residual block, we pursue a hybrid model-based / data-driven approach: the first $K$ eigenvectors of a transform matrix are derived from a statistical model, e.g., the asymmetric discrete sine transform (ADST), for stability, while the remaining $N-K$ are computed from $\bar{C}$ for performance. The transform computation is posed as a graph learning problem, where we seek a graph Laplacian matrix minimizing a graphical lasso objective inside a convex cone sharing the first $K$ eigenvectors in a Hilbert space of real symmetric matrices. We efficiently solve the problem via augmented Lagrangian relaxation and proximal gradient (PG). Using WebP as a baseline image codec, experimental results show that our hybrid graph transform achieved better energy compaction than default discrete cosine transform (DCT) and better stability than KLT. △ Less

Submitted 2 March, 2022; originally announced March 2022.

Understanding Public Opinion on Using Hydroxychloroquine for COVID-19 Treatment via Social Media

Authors: Thuy T. Do, Du Nguyen, Anh Le, Anh Nguyen, Dong Nguyen, Nga Hoang, Uyen Le, Tuan Tran

Abstract: Hydroxychloroquine (HCQ) is used to prevent or treat malaria caused by mosquito bites. Recently, the drug has been suggested to treat COVID-19, but that has not been supported by scientific evidence. The information regarding the drug efficacy has flooded social networks, posting potential threats to the community by perverting their perceptions of the drug efficacy. This paper studies the reactio… ▽ More Hydroxychloroquine (HCQ) is used to prevent or treat malaria caused by mosquito bites. Recently, the drug has been suggested to treat COVID-19, but that has not been supported by scientific evidence. The information regarding the drug efficacy has flooded social networks, posting potential threats to the community by perverting their perceptions of the drug efficacy. This paper studies the reactions of social network users on the recommendation of using HCQ for COVID-19 treatment by analyzing the reaction patterns and sentiment of the tweets. We collected 164,016 tweets from February to December 2020 and used a text mining approach to identify social reaction patterns and opinion change over time. Our descriptive analysis identified an irregularity of the users' reaction patterns associated tightly with the social and news feeds on the development of HCQ and COVID-19 treatment. The study linked the tweets and Google search frequencies to reveal the viewpoints of local communities on the use of HCQ for COVID-19 treatment across different states. Further, our tweet sentiment analysis reveals that public opinion changed significantly over time regarding the recommendation of using HCQ for COVID-19 treatment. The data showed that high support in the early dates but it significantly declined in October. Finally, using the manual classification of 4,850 tweets by humans as our benchmark, our sentiment analysis showed that the Google Cloud Natural Language algorithm outperformed the Valence Aware Dictionary and sEntiment Reasoner in classifying tweets, especially in the sarcastic tweet group. △ Less

Submitted 1 January, 2022; originally announced January 2022.

Comments: This paper will be presented at HEALTHINF 2022 - Conference

What to Prioritize? Natural Language Processing for the Development of a Modern Bug Tracking Solution in Hardware Development

Authors: Thi Thu Hang Do, Markus Dobler, Niklas Kühl

Abstract: Managing large numbers of incoming bug reports and finding the most critical issues in hardware development is time consuming, but crucial in order to reduce development costs. In this paper, we present an approach to predict the time to fix, the risk and the complexity of debugging and resolution of a bug report using different supervised machine learning algorithms, namely Random Forest, Naive B… ▽ More Managing large numbers of incoming bug reports and finding the most critical issues in hardware development is time consuming, but crucial in order to reduce development costs. In this paper, we present an approach to predict the time to fix, the risk and the complexity of debugging and resolution of a bug report using different supervised machine learning algorithms, namely Random Forest, Naive Bayes, SVM, MLP and XGBoost. Further, we investigate the effect of the application of active learning and we evaluate the impact of different text representation techniques, namely TF-IDF, Word2Vec, Universal Sentence Encoder and XLNet on the model's performance. The evaluation shows that a combination of text embeddings generated through the Universal Sentence Encoder and MLP as classifier outperforms all other methods, and is well suited to predict the risk and complexity of bug tickets. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Comments: Hawaii International Conference on System Sciences 2022 (HICSS-55)

Jamming Detection in Massive MIMO Systems

Authors: Hossein Akhlaghpasand, S. Mohammad Razavizadeh, Emil Björnson, Tan Tai Do

Abstract: This paper considers the physical layer security of a pilot-based massive multiple-input multiple-output (MaMIMO) system in presence of a multi-antenna jammer. To improve security of the network, we propose a new jamming detection method that makes use of a generalized likelihood ratio test over some coherence blocks. Our proposed method utilizes intentionally unused pilots in the network. The per… ▽ More This paper considers the physical layer security of a pilot-based massive multiple-input multiple-output (MaMIMO) system in presence of a multi-antenna jammer. To improve security of the network, we propose a new jamming detection method that makes use of a generalized likelihood ratio test over some coherence blocks. Our proposed method utilizes intentionally unused pilots in the network. The performance of the proposed detector improves by increasing the number of antennas at the base station, the number of unused pilots and also by the number of the coherence blocks that are utilized. Simulation results confirm our analyses and show that in the MaMIMO regime, perfect detection (i.e., correct detection probability is one) is achievable even with a small number of unused pilots. △ Less

Submitted 30 November, 2017; v1 submitted 28 November, 2017; originally announced November 2017.

Comments: 5 pages, 2 figures, to appear in IEEE Wireless Communications Letters

Jamming-Resistant Receivers for the Massive MIMO Uplink

Authors: Tan Tai Do, Emil Björnson, Erik G. Larsson, S. Mohammad Razavizadeh

Abstract: We design a jamming-resistant receiver scheme to enhance the robustness of a massive MIMO uplink system against jamming. We assume that a jammer attacks the system both in the pilot and data transmission phases. The key feature of the proposed scheme is that, in the pilot phase, we estimate not only the legitimate channel, but also the jamming channel by exploiting a purposely unused pilot sequenc… ▽ More We design a jamming-resistant receiver scheme to enhance the robustness of a massive MIMO uplink system against jamming. We assume that a jammer attacks the system both in the pilot and data transmission phases. The key feature of the proposed scheme is that, in the pilot phase, we estimate not only the legitimate channel, but also the jamming channel by exploiting a purposely unused pilot sequence. The jamming channel estimate is used to constructed linear receive filters that reject the impact of the jamming signal. The performance of the proposed scheme is analytically evaluated using asymptotic properties of massive MIMO. The optimal regularized zero-forcing receiver and the optimal power allocation are also studied. Numerical results are provided to verify our analysis and show that the proposed scheme greatly improves the achievable rates, as compared to conventional receivers. Interestingly, the proposed scheme works particularly well under strong jamming attacks, since the improved estimate of the jamming channel outweighs the extra jamming power. △ Less

Submitted 28 February, 2017; originally announced February 2017.

Comments: submitted to IEEE Trans. Inf. Forensics and Security

Jamming Resistant Receivers for Massive MIMO

Authors: Tan Tai Do, Emil Björnson, Erik G. Larsson

Abstract: We design jamming resistant receivers to enhance the robustness of a massive MIMO uplink channel against jamming. In the pilot phase, we estimate not only the desired channel, but also the jamming channel by exploiting purposely unused pilot sequences. The jamming channel estimate is used to construct the linear receive filter to reduce impact that jamming has on the achievable rates. The performa… ▽ More We design jamming resistant receivers to enhance the robustness of a massive MIMO uplink channel against jamming. In the pilot phase, we estimate not only the desired channel, but also the jamming channel by exploiting purposely unused pilot sequences. The jamming channel estimate is used to construct the linear receive filter to reduce impact that jamming has on the achievable rates. The performance of the proposed scheme is analytically and numerically evaluated. These results show that the proposed scheme greatly improves the rates, as compared to conventional receivers. Moreover, the proposed schemes still work well with stronger jamming power. △ Less

Submitted 15 February, 2017; originally announced February 2017.

Comments: Accepted in the 42nd IEEE Int. Conf. Acoust., Speech, and Signal Process. (ICASSP2017)

Massive MIMO Pilot Retransmission Strategies for Robustification against Jamming

Authors: Tan Tai Do, Hien Quoc Ngo, Trung Q. Duong, Tobias J. Oechtering, Mikael Skoglund

Abstract: This letter proposes anti-jamming strategies based on pilot retransmission for a single user uplink massive MIMO under jamming attack. A jammer is assumed to attack the system both in the training and data transmission phases. We first derive an achievable rate which enables us to analyze the effect of jamming attacks on the system performance. Counter-attack strategies are then proposed to mitiga… ▽ More This letter proposes anti-jamming strategies based on pilot retransmission for a single user uplink massive MIMO under jamming attack. A jammer is assumed to attack the system both in the training and data transmission phases. We first derive an achievable rate which enables us to analyze the effect of jamming attacks on the system performance. Counter-attack strategies are then proposed to mitigate this effect under two different scenarios: random and deterministic jamming attacks. Numerical results illustrate our analysis and benefit of the proposed schemes. △ Less

Submitted 24 November, 2016; originally announced November 2016.

Comments: Accepted for publication in the IEEE Wireless Communications Letters

On the Entropy Computation of Large Complex Gaussian Mixture Distributions

Authors: Su Min Kim, Tan Tai Do, Tobias J. Oechtering, Gunnar Peters

Abstract: The entropy computation of Gaussian mixture distributions with a large number of components has a prohibitive computational complexity. In this paper, we propose a novel approach exploiting the sphere decoding concept to bound and approximate such entropy terms with reduced complexity and good accuracy. Moreover, we propose an SNR region based enhancement of the approximation method to reduce the… ▽ More The entropy computation of Gaussian mixture distributions with a large number of components has a prohibitive computational complexity. In this paper, we propose a novel approach exploiting the sphere decoding concept to bound and approximate such entropy terms with reduced complexity and good accuracy. Moreover, we propose an SNR region based enhancement of the approximation method to reduce the complexity even further. Using Monte-Carlo simulations, the proposed methods are numerically demonstrated for the computation of the mutual information including the entropy term of various channels with finite constellation modulations such as binary and quadratic amplitude modulation (QAM) inputs for communication applications. △ Less

Submitted 27 May, 2015; v1 submitted 12 February, 2015; originally announced February 2015.

Comments: 14 pages, Accepted to IEEE Transactions on Signal Processing

Discriminative Local Sparse Representations for Robust Face Recognition

Authors: Yi Chen, Umamahesh Srinivas, Thong T. Do, Vishal Monga, Trac D. Tran

Abstract: A key recent advance in face recognition models a test face image as a sparse linear combination of a set of training face images. The resulting sparse representations have been shown to possess robustness against a variety of distortions like random pixel corruption, occlusion and disguise. This approach however makes the restrictive (in many scenarios) assumption that test faces must be perfectl… ▽ More A key recent advance in face recognition models a test face image as a sparse linear combination of a set of training face images. The resulting sparse representations have been shown to possess robustness against a variety of distortions like random pixel corruption, occlusion and disguise. This approach however makes the restrictive (in many scenarios) assumption that test faces must be perfectly aligned (or registered) to the training data prior to classification. In this paper, we propose a simple yet robust local block-based sparsity model, using adaptively-constructed dictionaries from local features in the training data, to overcome this misalignment problem. Our approach is inspired by human perception: we analyze a series of local discriminative features and combine them to arrive at the final classification decision. We propose a probabilistic graphical model framework to explicitly mine the conditional dependencies between these distinct sparse local features. In particular, we learn discriminative graphs on sparse representations obtained from distinct local slices of a face. Conditional correlations between these sparse features are first discovered (in the training phase), and subsequently exploited to bring about significant improvements in recognition rates. Experimental results obtained on benchmark face databases demonstrate the effectiveness of the proposed algorithms in the presence of multiple registration errors (such as translation, rotation, and scaling) as well as under variations of pose and illumination. △ Less

Submitted 8 November, 2011; originally announced November 2011.

Fast and Efficient Compressive Sensing using Structurally Random Matrices

Authors: Thong T. Do, Lu Gan, Nam H. Nguyen, Trac D. Tran

Abstract: This paper introduces a new framework of fast and efficient sensing matrices for practical compressive sensing, called Structurally Random Matrix (SRM). In the proposed framework, we pre-randomize a sensing signal by scrambling its samples or flip** its sample signs and then fast-transform the randomized samples and finally, subsample the transform coefficients as the final sensing measurements.… ▽ More This paper introduces a new framework of fast and efficient sensing matrices for practical compressive sensing, called Structurally Random Matrix (SRM). In the proposed framework, we pre-randomize a sensing signal by scrambling its samples or flip** its sample signs and then fast-transform the randomized samples and finally, subsample the transform coefficients as the final sensing measurements. SRM is highly relevant for large-scale, real-time compressive sensing applications as it has fast computation and supports block-based processing. In addition, we can show that SRM has theoretical sensing performance comparable with that of completely random sensing matrices. Numerical simulation results verify the validity of the theory as well as illustrate the promising potentials of the proposed sensing framework. △ Less

Submitted 24 June, 2011; originally announced June 2011.

Building a Vietnamese Language Query Processing Framework for ELibrary Searching Systems

Authors: Dang Tuan Nguyen, Ha Quy-Tinh Luong, Tuyen Thi-Thanh Do

Abstract: In the objective of building intelligent searching systems for Elibraries or online bookstores, we have proposed a searching system model based on a Vietnamese language query processing component. Such document searching systems based on this model can allow users to use Vietnamese queries that represent content information as input, instead of entering keywords for searching in specific fields… ▽ More In the objective of building intelligent searching systems for Elibraries or online bookstores, we have proposed a searching system model based on a Vietnamese language query processing component. Such document searching systems based on this model can allow users to use Vietnamese queries that represent content information as input, instead of entering keywords for searching in specific fields in database. To simplify the realization process of system based on this searching system model, we set a target of building a framework to support the rapid development of Vietnamese language query processing components. Such framework let the implementation of Vietnamese language query processing component in similar systems in this domain to be done more easily. △ Less

Submitted 2 November, 2009; originally announced November 2009.

Comments: 5 pages IEEE format, International Journal of Computer Science and Information Security, IJCSIS 2009, ISSN 1947 5500, Impact Factor 0.423, http://sites.google.com/site/ijcsis/

Report number: ISSN 1947 5500

Journal ref: International Journal of Computer Science and Information Security, IJCSIS, Vol. 6, No. 1, pp. 092-096, October 2009, USA