Showing 1–50 of 64 results for author: Wadayama, T

Ordinary Differential Equation-based MIMO Signal Detection

Authors: Ayano Nakai-Kasai, Tadashi Wadayama

Abstract: The required signal processing rate in future wireless communication systems exceeds the performance of the latest electronics-based processors. Introduction of analog optical computation is one promising direction for energy-efficient processing. This paper considers a continuous-time minimum mean squared error detection for multiple-input multiple-output systems to realize signal detection using… ▽ More The required signal processing rate in future wireless communication systems exceeds the performance of the latest electronics-based processors. Introduction of analog optical computation is one promising direction for energy-efficient processing. This paper considers a continuous-time minimum mean squared error detection for multiple-input multiple-output systems to realize signal detection using analog optical devices. The proposed method is formulated by an ordinary differential equation (ODE) and its performance at any continuous time can be theoretically analyzed. Deriving and analyzing the continuous-time system is a meaningful step to verifying the feasibility of analog-domain signal processing in the future systems. In addition, considering such an ODE brings byproducts to discrete-time detection algorithms, which can be a novel methodology of algorithm construction and analysis. △ Less

Submitted 16 March, 2024; v1 submitted 27 April, 2023; originally announced April 2023.

Comments: arXiv admin note: text overlap with arXiv:2205.01338

Stochastic Dynamics of Noisy Average Consensus: Analysis and Optimization

Authors: Tadashi Wadayama, Ayano Nakai-Kasai

Abstract: A continuous-time average consensus system is a linear dynamical system defined over a graph, where each node has its own state value that evolves according to a simultaneous linear differential equation. A node is allowed to interact with neighboring nodes. Average consensus is a phenomenon that the all the state values converge to the average of the initial state values. In this paper, we assume… ▽ More A continuous-time average consensus system is a linear dynamical system defined over a graph, where each node has its own state value that evolves according to a simultaneous linear differential equation. A node is allowed to interact with neighboring nodes. Average consensus is a phenomenon that the all the state values converge to the average of the initial state values. In this paper, we assume that a node can communicate with neighboring nodes through an additive white Gaussian noise channel. We first formulate the noisy average consensus system by using a stochastic differential equation (SDE), which allows us to use the Euler-Maruyama method, a numerical technique for solving SDEs. By studying the stochastic behavior of the residual error of the Euler-Maruyama method, we arrive at the covariance evolution equation. The analysis of the residual error leads to a compact formula for mean squared error (MSE), which shows that the sum of the inverse eigenvalues of the Laplacian matrix is the most dominant factor influencing the MSE. Furthermore, we propose optimization problems aimed at minimizing the MSE at a given target time, and introduce a deep unfolding-based optimization method to solve these problems. The quality of the solution is validated by numerical experiments. △ Less

Submitted 29 March, 2023; originally announced March 2023.

Comments: 12 pages

Ordinary Differential Equation-based Sparse Signal Recovery

Authors: Tadashi Wadayama, Ayano Nakai-Kasai

Abstract: This study investigates the use of continuous-time dynamical systems for sparse signal recovery. The proposed dynamical system is in the form of a nonlinear ordinary differential equation (ODE) derived from the gradient flow of the Lasso objective function. The sparse signal recovery process of this ODE-based approach is demonstrated by numerical simulations using the Euler method. The state of th… ▽ More This study investigates the use of continuous-time dynamical systems for sparse signal recovery. The proposed dynamical system is in the form of a nonlinear ordinary differential equation (ODE) derived from the gradient flow of the Lasso objective function. The sparse signal recovery process of this ODE-based approach is demonstrated by numerical simulations using the Euler method. The state of the continuous-time dynamical system eventually converges to the equilibrium point corresponding to the minimum of the objective function. To gain insight into the local convergence properties of the system, a linear approximation around the equilibrium point is applied, yielding a closed-form error evolution ODE. This analysis shows the behavior of convergence to the equilibrium point. In addition, a variational optimization problem is proposed to optimize a time-dependent regularization parameter in order to improve both convergence speed and solution quality. The deep unfolded-variational optimization method is introduced as a means of solving this optimization problem, and its effectiveness is validated through numerical experiments. △ Less

Submitted 28 March, 2023; originally announced March 2023.

Comments: 13 pages

Gradient Flow Decoding for LDPC Codes

Authors: Tadashi Wadayama, Kensho Nakajima, Ayano Nakai-Kasai

Abstract: The power consumption of the integrated circuit is becoming a significant burden, particularly for large-scale signal processing tasks requiring high throughput. The decoding process of LDPC codes is such a heavy signal processing task that demands power efficiency and higher decoding throughput. A promising approach to reducing both power and latency of a decoding process is to use an analog circ… ▽ More The power consumption of the integrated circuit is becoming a significant burden, particularly for large-scale signal processing tasks requiring high throughput. The decoding process of LDPC codes is such a heavy signal processing task that demands power efficiency and higher decoding throughput. A promising approach to reducing both power and latency of a decoding process is to use an analog circuit instead of a digital circuit. This paper investigates a continuous-time gradient flow-based approach for decoding LDPC codes, which employs a potential energy function similar to the objective function used in the gradient descent bit flip** (GDBF) algorithm. We experimentally demonstrate that the decoding performance of the gradient flow decoding is comparable to that of the multi-bit mode GDBF algorithm. Since an analog circuit of the gradient flow decoding requires only analog arithmetic operations and an integrator, future advancements in programmable analog integrated circuits may make practical implementation feasible. △ Less

Submitted 28 March, 2023; originally announced March 2023.

Comments: 6 pages

Deep Unfolding-based Weighted Averaging for Federated Learning in Heterogeneous Environments

Authors: Ayano Nakai-Kasai, Tadashi Wadayama

Abstract: Federated learning is a collaborative model training method that iterates model updates by multiple clients and aggregation of the updates by a central server. Device and statistical heterogeneity of participating clients cause significant performance degradation so that an appropriate aggregation weight should be assigned to each client in the aggregation phase of the server. To adjust the aggreg… ▽ More Federated learning is a collaborative model training method that iterates model updates by multiple clients and aggregation of the updates by a central server. Device and statistical heterogeneity of participating clients cause significant performance degradation so that an appropriate aggregation weight should be assigned to each client in the aggregation phase of the server. To adjust the aggregation weights, this paper employs deep unfolding, which is known as the parameter tuning method that leverages both learning capability using training data like deep learning and domain knowledge. This enables us to directly incorporate the heterogeneity of the environment of interest into the tuning of the aggregation weights. The proposed approach can be combined with various federated learning algorithms. The results of numerical experiments indicate that a higher test accuracy for unknown class-balanced data can be obtained with the proposed method than that with conventional heuristic weighting methods. The proposed method can handle large-scale learning models with the aid of pretrained models such that it can perform practical real-world tasks. Convergence rate of federated learning algorithms with the proposed method is also provided in this paper. △ Less

Submitted 28 August, 2023; v1 submitted 23 December, 2022; originally announced December 2022.

Deterministic Fault-Tolerant Connectivity Labeling Scheme

Authors: Taisuke Izumi, Yuval Emek, Tadashi Wadayama, Toshimitsu Masuzawa

Abstract: The \emph{$f$-fault-tolerant connectivity labeling} ($f$-FTC labeling) is a scheme of assigning each vertex and edge with a small-size label such that one can determine the connectivity of two vertices $s$ and $t$ under the presence of at most $f$ faulty edges only from the labels of $s$, $t$, and the faulty edges. This paper presents a new deterministic $f$-FTC labeling scheme attaining… ▽ More The \emph{$f$-fault-tolerant connectivity labeling} ($f$-FTC labeling) is a scheme of assigning each vertex and edge with a small-size label such that one can determine the connectivity of two vertices $s$ and $t$ under the presence of at most $f$ faulty edges only from the labels of $s$, $t$, and the faulty edges. This paper presents a new deterministic $f$-FTC labeling scheme attaining $O(f^2 \mathrm{polylog}(n))$-bit label size and a polynomial construction time, which settles the open problem left by Dory and Parter [PODC'21]. The key ingredient of our construction is to develop a deterministic counterpart of the graph sketch technique by Ahn, Guha, and McGreger [SODA'12], via some natural connection with the theory of error-correcting codes. This technique removes one major obstacle in de-randomizing the Dory-Parter scheme. The whole scheme is obtained by combining this technique with a new deterministic graph sparsification algorithm derived from the seminal $ε$-net theory, which is also of independent interest. As byproducts, our result deduces the first deterministic fault-tolerant approximate distance labeling scheme with a non-trivial performance guarantee and an improved deterministic fault-tolerant compact routing. The authors believe that our new technique is potentially useful in the future exploration of more efficient FTC labeling schemes and other related applications based on graph sketches. △ Less

Submitted 16 November, 2023; v1 submitted 24 August, 2022; originally announced August 2022.

Precoder Design for Correlated Data Aggregation via Over-the-Air Computation in Sensor Networks

Authors: Ayano Nakai-Kasai, Tadashi Wadayama

Abstract: Over-the-air computation (AirComp) enables efficient wireless data aggregation in sensor networks by simultaneous processing of calculation and communication. This paper proposes a novel precoder design for AirComp that incorporates statistical properties of sensing data, spatial correlation and heterogeneous data correlation. The proposed design of the precoder requires no iterative processes so… ▽ More Over-the-air computation (AirComp) enables efficient wireless data aggregation in sensor networks by simultaneous processing of calculation and communication. This paper proposes a novel precoder design for AirComp that incorporates statistical properties of sensing data, spatial correlation and heterogeneous data correlation. The proposed design of the precoder requires no iterative processes so that it can be realized with low computational costs. Moreover, this method provides dimensionality reduction of sensing data to reduce communication costs per sensor. We evaluate performance of the proposed method in terms of various system parameters. The results show the superiority of the proposed method to conventional non-iterative methods in cases where there are a large number of sensors and where the number of receive antennas at the aggregator is less than that of the total transmit antennas at the sensors. △ Less

Submitted 2 September, 2022; v1 submitted 6 May, 2022; originally announced May 2022.

MMSE Signal Detection for MIMO Systems based on Ordinary Differential Equation

Authors: Ayano Nakai-Kasai, Tadashi Wadayama

Abstract: Motivated by emerging technologies for energy efficient analog computing and continuous-time processing, this paper proposes continuous-time minimum mean squared error estimation for multiple-input multiple-output (MIMO) systems based on an ordinary differential equation. Mean squared error (MSE) is a principal detection performance measure of estimation methods for MIMO systems. We derive an anal… ▽ More Motivated by emerging technologies for energy efficient analog computing and continuous-time processing, this paper proposes continuous-time minimum mean squared error estimation for multiple-input multiple-output (MIMO) systems based on an ordinary differential equation. Mean squared error (MSE) is a principal detection performance measure of estimation methods for MIMO systems. We derive an analytical MSE formula that indicates the MSE at any time. The MSE of the proposed method depends on a regularization parameter which affects the convergence property of the MSE. Furthermore, we extend the proposed method by using a time-dependent regularization parameter to achieve better convergence performance. Numerical experiments indicated excellent agreement with the theoretical values and improvement in the convergence performance owing to the use of the time-dependent parameter. △ Less

Submitted 2 September, 2022; v1 submitted 3 May, 2022; originally announced May 2022.

Proximal Decoding for LDPC-coded Massive MIMO Channels

Authors: Tadashi Wadayama, Satoshi Takabe

Abstract: We propose a novel optimization-based decoding algorithm for LDPC-coded massive MIMO channels. The proposed decoding algorithm is based on a proximal gradient method for solving an approximate maximum a posteriori (MAP) decoding problem. The key idea is the use of a code-constraint polynomial penalizing a vector far from a codeword as a regularizer in the approximate MAP objective function. The co… ▽ More We propose a novel optimization-based decoding algorithm for LDPC-coded massive MIMO channels. The proposed decoding algorithm is based on a proximal gradient method for solving an approximate maximum a posteriori (MAP) decoding problem. The key idea is the use of a code-constraint polynomial penalizing a vector far from a codeword as a regularizer in the approximate MAP objective function. The code proximal operator is naturally derived from code-constraint polynomials. The proposed algorithm, called proximal decoding, can be described by a simple recursion consisting of the gradient descent step for a negative log-likelihood function and the code proximal operation. Several numerical experiments show that the proposed algorithm outperforms known massive MIMO detection algorithms, such as an MMSE detector with belief propagation decoding. △ Less

Submitted 9 February, 2021; originally announced February 2021.

Acceleration of Cooperative Least Mean Square via Chebyshev Periodical Successive Over-Relaxation

Authors: Tadashi Wadayama, Satoshi Takabe

Abstract: A distributed algorithm for least mean square (LMS) can be used in distributed signal estimation and in distributed training for multivariate regression models. The convergence speed of an algorithm is a critical factor because a faster algorithm requires less communications overhead and it results in a narrower network bandwidth. The goal of this paper is to present that use of Chebyshev periodic… ▽ More A distributed algorithm for least mean square (LMS) can be used in distributed signal estimation and in distributed training for multivariate regression models. The convergence speed of an algorithm is a critical factor because a faster algorithm requires less communications overhead and it results in a narrower network bandwidth. The goal of this paper is to present that use of Chebyshev periodical successive over-relaxation (PSOR) can accelerate distributed LMS algorithms in a naturally manner. The basic idea of Chbyshev PSOR is to introduce index-dependent PSOR factors that control the spectral radius of a matrix governing the convergence behavior of the modified fixed-point iteration. Accelerations of convergence speed are empirically confirmed in a wide range of networks, such as known small graphs (e.g., Karate graph), and random graphs, such as Erdos-Renyi (ER) random graphs and Barabasi-Albert random graphs. △ Less

Submitted 24 November, 2020; originally announced November 2020.

Convergence Acceleration via Chebyshev Step: Plausible Interpretation of Deep-Unfolded Gradient Descent

Authors: Satoshi Takabe, Tadashi Wadayama

Abstract: Deep unfolding is a promising deep-learning technique, whose network architecture is based on expanding the recursive structure of existing iterative algorithms. Although convergence acceleration is a remarkable advantage of deep unfolding, its theoretical aspects have not been revealed yet. The first half of this study details the theoretical analysis of the convergence acceleration in deep-unfol… ▽ More Deep unfolding is a promising deep-learning technique, whose network architecture is based on expanding the recursive structure of existing iterative algorithms. Although convergence acceleration is a remarkable advantage of deep unfolding, its theoretical aspects have not been revealed yet. The first half of this study details the theoretical analysis of the convergence acceleration in deep-unfolded gradient descent (DUGD) whose trainable parameters are step sizes. We propose a plausible interpretation of the learned step-size parameters in DUGD by introducing the principle of Chebyshev steps derived from Chebyshev polynomials. The use of Chebyshev steps in gradient descent (GD) enables us to bound the spectral radius of a matrix governing the convergence speed of GD, leading to a tight upper bound on the convergence rate. The convergence rate of GD using Chebyshev steps is shown to be asymptotically optimal, although it has no momentum terms. We also show that Chebyshev steps numerically explain the learned step-size parameters in DUGD well. In the second half of the study, %we apply the theory of Chebyshev steps and Chebyshev-periodical successive over-relaxation (Chebyshev-PSOR) is proposed for accelerating linear/nonlinear fixed-point iterations. Theoretical analysis and numerical experiments indicate that Chebyshev-PSOR exhibits significantly faster convergence for various examples such as Jacobi method and proximal gradient methods. △ Less

Submitted 26 October, 2020; originally announced October 2020.

Comments: 17 pages, 22 figures, This manuscript is the revised and updated version of arXiv:2001.03280 and arXiv:2001.05142

Deep Unfolded Multicast Beamforming

Authors: Satoshi Takabe, Tadashi Wadayama

Abstract: Multicast beamforming is a promising technique for multicast communication. Providing an efficient and powerful beamforming design algorithm is a crucial issue because multicast beamforming problems such as a max-min-fair problem are NP-hard in general. Recently, deep learning-based approaches have been proposed for beamforming design. Although these approaches using deep neural networks exhibit r… ▽ More Multicast beamforming is a promising technique for multicast communication. Providing an efficient and powerful beamforming design algorithm is a crucial issue because multicast beamforming problems such as a max-min-fair problem are NP-hard in general. Recently, deep learning-based approaches have been proposed for beamforming design. Although these approaches using deep neural networks exhibit reasonable performance gain compared with conventional optimization-based algorithms, their scalability is an emerging problem for large systems in which beamforming design becomes a more demanding task. In this paper, we propose a novel deep unfolded trainable beamforming design with high scalability and efficiency. The algorithm is designed by expanding the recursive structure of an existing algorithm based on projections onto convex sets and embedding a constant number of trainable parameters to the expanded network, which leads to a scalable and stable training process. Numerical results show that the proposed algorithm can accelerate its convergence speed by using unsupervised learning, which is a challenging training process for deep unfolding. △ Less

Submitted 20 April, 2020; originally announced April 2020.

Comments: 6 pages, 5 figures

Chebyshev Inertial Landweber Algorithm for Linear Inverse Problems

Authors: Tadashi Wadayama, Satoshi Takabe

Abstract: The Landweber algorithm defined on complex/real Hilbert spaces is a gradient descent algorithm for linear inverse problems. Our contribution is to present a novel method for accelerating convergence of the Landweber algorithm. In this paper, we first extend the theory of the Chebyshev inertial iteration to the Landweber algorithm on Hilbert spaces. An upper bound on the convergence rate clarifies… ▽ More The Landweber algorithm defined on complex/real Hilbert spaces is a gradient descent algorithm for linear inverse problems. Our contribution is to present a novel method for accelerating convergence of the Landweber algorithm. In this paper, we first extend the theory of the Chebyshev inertial iteration to the Landweber algorithm on Hilbert spaces. An upper bound on the convergence rate clarifies the speed of global convergence of the proposed method. The Chebyshev inertial Landweber algorithm can be applied to wide class of signal recovery problems on a Hilbert space including deconvolution for continuous signals. The theoretical discussion developed in this paper naturally leads to a novel practical signal recovery algorithm. As a demonstration, a MIMO detection algorithm based on the projected Landweber algorithm is derived. The proposed MIMO detection algorithm achieves much smaller symbol error rate compared with the MMSE detector. △ Less

Submitted 16 January, 2020; originally announced January 2020.

Comments: 5 pages

Theoretical Interpretation of Learned Step Size in Deep-Unfolded Gradient Descent

Authors: Satoshi Takabe, Tadashi Wadayama

Abstract: Deep unfolding is a promising deep-learning technique in which an iterative algorithm is unrolled to a deep network architecture with trainable parameters. In the case of gradient descent algorithms, as a result of the training process, one often observes the acceleration of the convergence speed with learned non-constant step size parameters whose behavior is not intuitive nor interpretable from… ▽ More Deep unfolding is a promising deep-learning technique in which an iterative algorithm is unrolled to a deep network architecture with trainable parameters. In the case of gradient descent algorithms, as a result of the training process, one often observes the acceleration of the convergence speed with learned non-constant step size parameters whose behavior is not intuitive nor interpretable from conventional theory. In this paper, we provide a theoretical interpretation of the learned step size of deep-unfolded gradient descent (DUGD). We first prove that the training process of DUGD reduces not only the mean squared error loss but also the spectral radius related to the convergence rate. Next, we show that minimizing the upper bound of the spectral radius naturally leads to the Chebyshev step which is a sequence of the step size based on Chebyshev polynomials. The numerical experiments confirm that the Chebyshev steps qualitatively reproduce the learned step size parameters in DUGD, which provides a plausible interpretation of the learned parameters. Additionally, we show that the Chebyshev steps achieve the lower bound of the convergence rate for the first-order method in a specific limit without learning parameters or momentum terms. △ Less

Submitted 30 January, 2020; v1 submitted 15 January, 2020; originally announced January 2020.

Comments: 12 pages, 12 figures, typos are fixed

Chebyshev Inertial Iteration for Accelerating Fixed-Point Iterations

Authors: Tadashi Wadayama, Satoshi Takabe

Abstract: A novel method which is called the Chebyshev inertial iteration for accelerating the convergence speed of fixed-point iterations is presented. The Chebyshev inertial iteration can be regarded as a valiant of the successive over relaxation or Krasnosel'skiǐ-Mann iteration utilizing the inverse of roots of a Chebyshev polynomial as iteration dependent inertial factors. One of the most notable featur… ▽ More A novel method which is called the Chebyshev inertial iteration for accelerating the convergence speed of fixed-point iterations is presented. The Chebyshev inertial iteration can be regarded as a valiant of the successive over relaxation or Krasnosel'skiǐ-Mann iteration utilizing the inverse of roots of a Chebyshev polynomial as iteration dependent inertial factors. One of the most notable features of the proposed method is that it can be applied to nonlinear fixed-point iterations in addition to linear fixed-point iterations. Linearization around the fixed point is the key for the analysis on the local convergence rate of the proposed method. The proposed method appears effective in particular for accelerating the proximal gradient methods such as ISTA. It is also proved that the proposed method can successfully accelerate almost any fixed-point iterations if all the eigenvalues of the Jacobian at the fixed point are real. △ Less

Submitted 9 January, 2020; originally announced January 2020.

Comments: 9 pages

Trainable Projected Gradient Detector for Sparsely Spread Code Division Multiple Access

Authors: Satoshi Takabe, Yuki Yamauchi, Tadashi Wadayama

Abstract: Sparsely spread code division multiple access (SCDMA) is a promising non-orthogonal multiple access technique for future wireless communications. In this paper, we propose a novel trainable multiuser detector called sparse trainable projected gradient (STPG) detector, which is based on the notion of deep unfolding. In the STPG detector, trainable parameters are embedded to a projected gradient des… ▽ More Sparsely spread code division multiple access (SCDMA) is a promising non-orthogonal multiple access technique for future wireless communications. In this paper, we propose a novel trainable multiuser detector called sparse trainable projected gradient (STPG) detector, which is based on the notion of deep unfolding. In the STPG detector, trainable parameters are embedded to a projected gradient descent algorithm, which can be trained by standard deep learning techniques such as back propagation and stochastic gradient descent. Advantages of the detector are its low computational cost and small number of trainable parameters, which enables us to treat massive SCDMA systems. In particular, its computational cost is smaller than a conventional belief propagation (BP) detector while the STPG detector exhibits nearly same detection performance with a BP detector. We also propose a scalable joint learning of signature sequences and the STPG detector for signature design. Numerical results show that the joint learning improves multiuser detection performance particular in the low SNR regime. △ Less

Submitted 23 October, 2019; originally announced October 2019.

Comments: 6 pages, 5 figures

Deep Learning-Based Average Consensus

Authors: Masako Kishida, Masaki Ogura, Yuichi Yoshida, Tadashi Wadayama

Abstract: In this study, we analyzed the problem of accelerating the linear average consensus algorithm for complex networks. We propose a data-driven approach to tuning the weights of temporal (i.e., time-varying) networks using deep learning techniques. Given a finite-time window, the proposed approach first unfolds the linear average consensus protocol to obtain a feedforward signal-flow graph, which is… ▽ More In this study, we analyzed the problem of accelerating the linear average consensus algorithm for complex networks. We propose a data-driven approach to tuning the weights of temporal (i.e., time-varying) networks using deep learning techniques. Given a finite-time window, the proposed approach first unfolds the linear average consensus protocol to obtain a feedforward signal-flow graph, which is regarded as a neural network. The edge weights of the obtained neural network are then trained using standard deep learning techniques to minimize consensus error over a given finite-time window. Through this training process, we obtain a set of optimized time-varying weights, which yield faster consensus for a complex network. We also demonstrate that the proposed approach can be extended for infinite-time window problems. Numerical experiments revealed that our approach can achieve a significantly smaller consensus error compared to baseline strategies. △ Less

Submitted 6 August, 2020; v1 submitted 26 August, 2019; originally announced August 2019.

Journal ref: IEEE Access, volume 8, 2020, pp.142404 - 142412

Proceedings of the 11th Asia-Europe Workshop on Concepts in Information Theory

Authors: A. J. Han Vinck, Kees A. Schouhamer Immink, Tadashi Wadayama, Van Khu Vu, Akiko Manada, Kui Cai, Shunsuke Horii, Yoshiki Abe, Mitsugu Iwamoto, Kazuo Ohta, Xingwei Zhong, Zhen Mei, Renfei Bu, J. H. Weber, Vitaly Skachek, Hiroyoshi Morita, N. Hovhannisyan, Hiroshi Kamabe, Shan Lu, Hirosuke Yamamoto, Kengo Hasimoto, O. Ytrehus, Shigeaki Kuzuoaka, Mikihiko Nishiara, Han Mao Kiah , et al. (2 additional authors not shown)

Abstract: This year, 2019 we celebrate 30 years of our friendship between Asian and European scientists at the AEW11 in Rotterdam, the Netherlands. Many of the 1989 participants are also present at the 2019 event. This year we have many participants from different parts of Asia and Europe. It shows the importance of this event. It is a good tradition to pay a tribute to a special lecturer in our community.… ▽ More This year, 2019 we celebrate 30 years of our friendship between Asian and European scientists at the AEW11 in Rotterdam, the Netherlands. Many of the 1989 participants are also present at the 2019 event. This year we have many participants from different parts of Asia and Europe. It shows the importance of this event. It is a good tradition to pay a tribute to a special lecturer in our community. This year we selected Hiroyoshi Morita, who is a well known information theorist with many original contributions. △ Less

Submitted 26 June, 2019; originally announced July 2019.

Compute-and-forward relaying with LDPC codes over QPSK scheme

Authors: Satoshi Takabe, Tadashi Wadayama, Ángeles Vazquez-Castro, Masahito Hayashi

Abstract: In this paper, we study a compute-and-forward (CAF) relaying scheme with low-density parity-check (LDPC) codes, a special case of physical layer network coding, under the quadrature phase shift keying (QPSK) modulation. The novelty of this paper is the theoretical analysis of decoding performance of the CAF scheme and traditional separation decoding (SD) scheme with joint decoding or with successi… ▽ More In this paper, we study a compute-and-forward (CAF) relaying scheme with low-density parity-check (LDPC) codes, a special case of physical layer network coding, under the quadrature phase shift keying (QPSK) modulation. The novelty of this paper is the theoretical analysis of decoding performance of the CAF scheme and traditional separation decoding (SD) scheme with joint decoding or with successive interference cancellation (SIC) decoding when the reception powers from both senders are not equal but close to each other. First, we study the decoding performance of linear random coding (LRC) in the CAF scheme whose decoder is based on the degraded channel. When rotation angles of constellations of two senders are controlled, we show that they can achieve rates well beyond the multiple access channel (MAC) with LRC with optimal rotation angles. Second, we analyze the practical feasibility of CAF using LDPC codes in terms of computational costs and decoding performance of belief propagation (BP) decoders. The calculation complexity of the BP decoder for the CAF scheme is almost equal to that of the SIC BP decoder, and smaller than the joint BP decoder in the SD scheme. Decoding performance is evaluated by the asymptotic decodable region using density evolution. The results show that, with code rate fixed, the CAF scheme has better performance than the SD scheme with joint BP decoding and SIC BP decoding in the high rate region. △ Less

Submitted 17 April, 2019; originally announced April 2019.

Comments: 6 pages, 5 figures

Complex Trainable ISTA for Linear and Nonlinear Inverse Problems

Authors: Satoshi Takabe, Tadashi Wadayama, Yonina C. Eldar

Abstract: Complex-field signal recovery problems from noisy linear/nonlinear measurements appear in many areas of signal processing and wireless communications. In this paper, we propose a trainable iterative signal recovery algorithm named complex-field TISTA (C-TISTA) which treats complex-field nonlinear inverse problems. C-TISTA is based on the concept of deep unfolding and consists of a gradient descent… ▽ More Complex-field signal recovery problems from noisy linear/nonlinear measurements appear in many areas of signal processing and wireless communications. In this paper, we propose a trainable iterative signal recovery algorithm named complex-field TISTA (C-TISTA) which treats complex-field nonlinear inverse problems. C-TISTA is based on the concept of deep unfolding and consists of a gradient descent step with the Wirtinger derivatives followed by a shrinkage step with a trainable complex-valued shrinkage function. Importantly, it contains a small number of trainable parameters so that its training process can be executed efficiently. Numerical results indicate that C-TISTA shows remarkable signal recovery performance compared with existing algorithms. △ Less

Submitted 17 October, 2019; v1 submitted 15 April, 2019; originally announced April 2019.

Comments: 6 pages, 3 figures

Asymptotic Analysis on LDPC-BICM Scheme for Compute-and-Forward Relaying

Authors: Satoshi Takabe, Tadashi Wadayama, Masahito Hayashi

Abstract: The compute-and-forward (CAF) scheme has attracted great interests due to its high band-width efficiency on two-way relay channels. In the CAF scheme, a relay attempts to decode a linear combination of transmitted messages from other terminals or relays. It is a crucial issue to study practical error-correcting codes in order to realize the CAF scheme with low computational complexity. In this pap… ▽ More The compute-and-forward (CAF) scheme has attracted great interests due to its high band-width efficiency on two-way relay channels. In the CAF scheme, a relay attempts to decode a linear combination of transmitted messages from other terminals or relays. It is a crucial issue to study practical error-correcting codes in order to realize the CAF scheme with low computational complexity. In this paper, we present an efficient bit-interleaved coded modulation (BICM) scheme for the CAF scheme with phase shift keying (PSK) modulations. In particular, we examine the asymptotic decoding performance of the BICM scheme with low-density parity-check (LDPC) codes by using the density evolution (DE) method. Based on the asymmetric nature of the channel model, we utilize the population dynamics method for the DE equations without the all-zero codeword assumption. The results show that, for two-way relay channels with QPSK and 8PSK modulations, the LDPC-BICM scheme provides higher achievable rate compared with an alternative separation decoding scheme. △ Less

Submitted 16 January, 2019; originally announced January 2019.

Comments: 5 pages, 7 figures

Deep Learning-Aided Trainable Projected Gradient Decoding for LDPC Codes

Authors: Tadashi Wadayama, Satoshi Takabe

Abstract: We present a novel optimization-based decoding algorithm for LDPC codes that is suitable for hardware architectures specialized to feed-forward neural networks. The algorithm is based on the projected gradient descent algorithm with a penalty function for solving a non-convex minimization problem. The proposed algorithm has several internal parameters such as step size parameters, a softness param… ▽ More We present a novel optimization-based decoding algorithm for LDPC codes that is suitable for hardware architectures specialized to feed-forward neural networks. The algorithm is based on the projected gradient descent algorithm with a penalty function for solving a non-convex minimization problem. The proposed algorithm has several internal parameters such as step size parameters, a softness parameter, and the penalty coefficients. We use a standard tool set of deep learning, i.e., back propagation and stochastic gradient descent (SGD) type algorithms, to optimize these parameters. Several numerical experiments show that the proposed algorithm outperforms the belief propagation decoding in some cases. △ Less

Submitted 14 January, 2019; originally announced January 2019.

Comments: 8 pages

Trainable Projected Gradient Detector for Massive Overloaded MIMO Channels: Data-driven Tuning Approach

Authors: Satoshi Takabe, Masayuki Imanishi, Tadashi Wadayama, Ryo Hayakawa, Kazunori Hayashi

Abstract: This paper presents a deep learning-aided iterative detection algorithm for massive overloaded multiple-input multiple-output (MIMO) systems where the number of transmit antennas $n$ is larger than that of receive antennas $m$. Since the proposed algorithm is based on the projected gradient descent method with trainable parameters, it is named the trainable projected gradient-detector (TPG-detecto… ▽ More This paper presents a deep learning-aided iterative detection algorithm for massive overloaded multiple-input multiple-output (MIMO) systems where the number of transmit antennas $n$ is larger than that of receive antennas $m$. Since the proposed algorithm is based on the projected gradient descent method with trainable parameters, it is named the trainable projected gradient-detector (TPG-detector). The trainable internal parameters, such as the step-size parameter, can be optimized with standard deep learning techniques, i.e., the back propagation and stochastic gradient descent algorithms. This approach is referred to as data-driven tuning, and ensures fast convergence during parameter estimation in the proposed scheme. The TPG-detector mainly consists of matrix-vector product operations whose computational cost is proportional to $m n$ for each iteration. In addition, the number of trainable parameters in the TPG-detector is independent of the number of antennas. These features of the TPG-detector result in a fast and stable training process and reasonable scalability for large systems. Numerical simulations show that the proposed detector achieves a comparable detection performance to those of existing algorithms for massive overloaded MIMO channels, e.g., the state-of-the-art IW-SOAV detector, with a lower computation cost. △ Less

Submitted 9 July, 2019; v1 submitted 25 December, 2018; originally announced December 2018.

Comments: 12 pages, 16 figures, accepted to IEEE Access, this is a long version of arXiv:1806.10827

Asymptotic Analysis of Spatial Coupling Coding for Compute-and-Forward Relaying

Authors: Satoshi Takabe, Tadashi Wadayama, Masahito Hayashi

Abstract: Compute-and-forward (CAF) relaying is effective to increase bandwidth efficiency of wireless two-way relay channels. In a CAF scheme, a relay is designed to decode a linear combination composed of transmitted messages from other terminals or relays. Design for error-correcting codes and its decoding algorithms suitable for CAF relaying schemes remain as an important issue to be studied. As describ… ▽ More Compute-and-forward (CAF) relaying is effective to increase bandwidth efficiency of wireless two-way relay channels. In a CAF scheme, a relay is designed to decode a linear combination composed of transmitted messages from other terminals or relays. Design for error-correcting codes and its decoding algorithms suitable for CAF relaying schemes remain as an important issue to be studied. As described in this paper, we will present an asymptotic performance analysis of LDPC codes over two-way relay channels based on density evolution (DE). Because of the asymmetric characteristics of the channel, we use the population dynamics DE combined with DE formulas for asymmetric channels to obtain BP thresholds. Additionally, we also evaluate the asymptotic performance of spatially coupled LDPC codes for two-way relay channels. The results indicate that the spatial coupling codes yield improvements in the BP threshold compared with corresponding uncoupled codes for two-way relay channels. Finally, we will compare the mutual information rate and rate achievability between the CAF scheme and the MAC separation decoding scheme. We demonstrate the possibility that the CAF scheme has higher reliability in the high-rate region. △ Less

Submitted 3 July, 2018; originally announced July 2018.

Comments: 8 pages, 6 figures. arXiv admin note: substantial text overlap with arXiv:1801.06328

Deep Learning-Aided Projected Gradient Detector for Massive Overloaded MIMO Channels

Authors: Satoshi Takabe, Masayuki Imanishi, Tadashi Wadayama, Kazunori Hayashi

Abstract: The paper presents a deep learning-aided iterative detection algorithm for massive overloaded MIMO systems. Since the proposed algorithm is based on the projected gradient descent method with trainable parameters, it is named as trainable projected descent-detector (TPG-detector). The trainable internal parameters can be optimized with standard deep learning techniques such as back propagation and… ▽ More The paper presents a deep learning-aided iterative detection algorithm for massive overloaded MIMO systems. Since the proposed algorithm is based on the projected gradient descent method with trainable parameters, it is named as trainable projected descent-detector (TPG-detector). The trainable internal parameters can be optimized with standard deep learning techniques such as back propagation and stochastic gradient descent algorithms. This approach referred to as data-driven tuning brings notable advantages of the proposed scheme such as fast convergence. The numerical experiments show that TPG-detector achieves comparable detection performance to those of the known algorithms for massive overloaded MIMO channels with lower computation cost. △ Less

Submitted 25 December, 2018; v1 submitted 28 June, 2018; originally announced June 2018.

Comments: 6 pages, 8 figures, metadata is updated

Secure Computation-and-Forward with Linear Codes

Authors: Masahito Hayashi, Tadashi Wadayama, Angeles Vazquez-Castro

Abstract: We discuss secure transmission via an untrusted relay when we have a multiple access phase from two nodes to the relay and broadcast phase from the relay to the two nodes. To realize the security, we construct a code that securely transmits the modulo sum of the messages of two nodes via a multiple access channel. In this code, the relay cannot obtain any information for the message of each node,… ▽ More We discuss secure transmission via an untrusted relay when we have a multiple access phase from two nodes to the relay and broadcast phase from the relay to the two nodes. To realize the security, we construct a code that securely transmits the modulo sum of the messages of two nodes via a multiple access channel. In this code, the relay cannot obtain any information for the message of each node, and can decode only the messages of the two nodes. Our code is constructed by simple combination of an existing liner code and universal2 hash function. △ Less

Submitted 27 April, 2018; originally announced April 2018.

Connectivity of Ad Hoc Wireless Networks with Node Faults

Authors: Satoshi Takabe, Tadashi Wadayama

Abstract: Connectivity of wireless sensor networks (WSNs) is a fundamental global property expected to be maintained even though some sensor nodes are at fault. In this paper, we investigate the connectivity of random geometric graphs (RGGs) in the node fault model as an abstract model of ad hoc WSNs with unreliable nodes. In the model, each node is assumed to be stochastically at fault, i.e., removed from… ▽ More Connectivity of wireless sensor networks (WSNs) is a fundamental global property expected to be maintained even though some sensor nodes are at fault. In this paper, we investigate the connectivity of random geometric graphs (RGGs) in the node fault model as an abstract model of ad hoc WSNs with unreliable nodes. In the model, each node is assumed to be stochastically at fault, i.e., removed from a graph. As a measure of reliability, the network breakdown probability is then defined as the average probability that a resulting survival graph is disconnected over RGGs. We examine RGGs with general connection functions as an extension of a conventional RGG model and provide two mathematical analyses: the asymptotic analysis for infinite RGGs that reveals the phase transition thresholds of connectivity, and the non-asymptotic analysis for finite RGGs that provides a useful approximation formula. Those analyses are supported by numerical simulations in the Rayleigh SISO model reflecting a practical wireless channel. △ Less

Submitted 19 April, 2018; originally announced April 2018.

Comments: 6pages, 3 figures

Joint Quantizer Optimization based on Neural Quantizer for Sum-Product Decoder

Authors: Tadashi Wadayama, Satoshi Takabe

Abstract: A low-precision analog-to-digital converter (ADC) is required to implement a frontend device of wideband digital communication systems in order to reduce its power consumption. The goal of this paper is to present a novel joint quantizer optimization method for minimizing lower-precision quantizers matched to the sum-product algorithms. The principal idea is to introduce a quantizer that includes… ▽ More A low-precision analog-to-digital converter (ADC) is required to implement a frontend device of wideband digital communication systems in order to reduce its power consumption. The goal of this paper is to present a novel joint quantizer optimization method for minimizing lower-precision quantizers matched to the sum-product algorithms. The principal idea is to introduce a quantizer that includes a feed-forward neural network and the soft staircase function. Since the soft staircase function is differentiable and has non-zero gradient values everywhere, we can exploit backpropagation and a stochastic gradient descent method to train the feed-forward neural network in the quantizer. The expected loss regarding the channel input and the decoder output is minimized in a supervised training phase. The experimental results indicate that the joint quantizer optimization method successfully provides an 8-level quantizer for a low-density parity-check (LDPC) code that achieves only a 0.1-dB performance loss compared to the unquantized system. △ Less

Submitted 16 April, 2018; originally announced April 2018.

Comments: 6 pages

Asymptotic Achievable Rate of Two-Dimensional Constraint Codes based on Column by Column Encoding

Authors: Kazuya Hirata, Tadashi Wadayama

Abstract: In this paper, we propose a column by column encoding scheme suitable for two-dimensional (2D) constraint codes and derive a lower bound of its maximum achievable rate. It is shown that the maximum achievable rate is equal to the largest minimum degree of a subgraph of the maximal valid pair graph. A graph theoretical analysis to provide a lower bound of the maximum achievable rate is presented. F… ▽ More In this paper, we propose a column by column encoding scheme suitable for two-dimensional (2D) constraint codes and derive a lower bound of its maximum achievable rate. It is shown that the maximum achievable rate is equal to the largest minimum degree of a subgraph of the maximal valid pair graph. A graph theoretical analysis to provide a lower bound of the maximum achievable rate is presented. For several 2D-constraints such as the asymmetric and symmetric non-isolated bit constraints, the values of the lower bound are evaluated. △ Less

Submitted 16 April, 2018; originally announced April 2018.

Comments: 5 pages

Asymptotic Analysis on Spatial Coupling Coding for Two-Way Relay Channels

Authors: Satoshi Takabe, Yuta Ishimatsu, Tadashi Wadayama, Masahito Hayashi

Abstract: Compute-and-forward relaying is effective to increase bandwidth efficiency of wireless two-way relay channels. In a compute-and-forward scheme, a relay tries to decode a linear combination composed of transmitted messages from other terminals or relays. Design for error correcting codes and its decoding algorithms suitable for compute-and-forward relaying schemes are still important issue to be st… ▽ More Compute-and-forward relaying is effective to increase bandwidth efficiency of wireless two-way relay channels. In a compute-and-forward scheme, a relay tries to decode a linear combination composed of transmitted messages from other terminals or relays. Design for error correcting codes and its decoding algorithms suitable for compute-and-forward relaying schemes are still important issue to be studied. In this paper, we will present an asymptotic performance analysis on LDPC codes over two-way relay channels based on density evolution (DE). Because of the asymmetric nature of the channel, we employ the population dynamics DE combined with DE formulas for asymmetric channels to obtain BP thresholds. In addition, we also evaluate the asymptotic performance of spatially coupled LDPC codes for two-way relay channels. The results indicate that the spatial coupling codes yield improvements in the BP threshold compared with corresponding uncoupled codes for two-way relay channels. △ Less

Submitted 19 January, 2018; originally announced January 2018.

Comments: 5 pages

k-connectivity of Random Graphs and Random Geometric Graphs in Node Fault Model

Authors: Satoshi Takabe, Tadashi Wadayama

Abstract: k-connectivity of random graphs is a fundamental property indicating reliability of multi-hop wireless sensor networks (WSN). WSNs comprising of sensor nodes with limited power resources are modeled by random graphs with unreliable nodes, which is known as the node fault model. In this paper, we investigate k-connectivity of random graphs in the node fault model by evaluating the network breakdown… ▽ More k-connectivity of random graphs is a fundamental property indicating reliability of multi-hop wireless sensor networks (WSN). WSNs comprising of sensor nodes with limited power resources are modeled by random graphs with unreliable nodes, which is known as the node fault model. In this paper, we investigate k-connectivity of random graphs in the node fault model by evaluating the network breakdown probability, i.e., the disconnectivity probability of random graphs after stochastic node removals. Using the notion of a strongly typical set, we obtain universal asymptotic upper and lower bounds of the network breakdown probability. The bounds are applicable both to random graphs and to random geometric graphs. We then consider three representative random graph ensembles: the Erdos-Renyi random graph as the simplest case, the random intersection graph for WSNs with random key predistribution schemes, and the random geometric graph as a model of WSNs generated by random sensor node deployment. The bounds unveil the existence of the phase transition of the network breakdown probability for those ensembles. △ Less

Submitted 9 January, 2018; originally announced January 2018.

Comments: 6 pages

Trainable ISTA for Sparse Signal Recovery

Authors: Daisuke Ito, Satoshi Takabe, Tadashi Wadayama

Abstract: In this paper, we propose a novel sparse signal recovery algorithm called Trainable ISTA (TISTA). The proposed algorithm consists of two estimation units such as a linear estimation unit and a minimum mean squared error (MMSE) estimator-based shrinkage unit. The estimated error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The… ▽ More In this paper, we propose a novel sparse signal recovery algorithm called Trainable ISTA (TISTA). The proposed algorithm consists of two estimation units such as a linear estimation unit and a minimum mean squared error (MMSE) estimator-based shrinkage unit. The estimated error variance required in the MMSE shrinkage unit is precisely estimated from a tentative estimate of the original signal. The remarkable feature of the proposed scheme is that TISTA includes adjustable variables controlling a step size and the error variance for the MMSE shrinkage. The variables are adjusted by standard deep learning techniques. The number of trainable variables of TISTA is equal to the number of iteration rounds and it is much smaller than that of known learnable sparse signal recovery algorithms. This feature leads to highly stable and fast training processes of TISTA. Computer experiments show that TISTA is applicable to various classes of sensing matrices such as Gaussian matrices, binary matrices and matrices with large condition numbers. Numerical results also demonstrate that TISTA shows significantly faster convergence than those of AMP and LISTA in many cases. △ Less

Submitted 21 November, 2018; v1 submitted 6 January, 2018; originally announced January 2018.

Comments: 10 pages

Fault Tolerance of Random Graphs with respect to Connectivity: Mean-field Approximation for Semi-dense Random Graphs

Authors: Satoshi Takabe, Takafumi Nakano, Tadashi Wadayama

Abstract: The fault tolerance of random graphs with unbounded degrees with respect to connectivity is investigated, which relates to the reliability of wireless sensor networks with unreliable relay nodes. The model evaluates the network breakdown probability that a graph is disconnected after stochastic node removal. To establish a mean-field approximation for the model, we propose the cavity method for fi… ▽ More The fault tolerance of random graphs with unbounded degrees with respect to connectivity is investigated, which relates to the reliability of wireless sensor networks with unreliable relay nodes. The model evaluates the network breakdown probability that a graph is disconnected after stochastic node removal. To establish a mean-field approximation for the model, we propose the cavity method for finite systems. The analysis enables us to obtain an approximation formula for random graphs with any number of nodes and an arbitrary degree distribution. In addition, its asymptotic analysis reveals that the phase transition occurs in semi-dense random graphs whose average degree grows logarithmically. These results, which are supported by numerical simulations, coincide with the mathematical results, indicating successful predictions by mean-field approximation for unbounded but not dense random graphs. △ Less

Submitted 9 May, 2019; v1 submitted 21 December, 2017; originally announced December 2017.

Comments: 5 pages, 3 figures

Journal ref: Phys. Rev. E 99, 050304 (2019)

Proceedings of Workshop AEW10: Concepts in Information Theory and Communications

Authors: Kees A. Schouhamer Immink, Stan Baggen, Ferdaous Chaabane, Yanling Chen, Peter H. N. de With, Hela Gassara, Hamed Gharbi, Adel Ghazel, Khaled Grati, Naira M. Grigoryan, Ashot Harutyunyan, Masayuki Imanishi, Mitsugu Iwamoto, Ken-ichi Iwata, Hiroshi Kamabe, Brian M. Kurkoski, Shigeaki Kuzuoka, Patrick Langenhuizen, Jan Lewandowsky, Akiko Manada, Shigeki Miyake, Hiroyoshi Morita, Jun Muramatsu, Safa Najjar, Arnak V. Poghosyan , et al. (9 additional authors not shown)

Abstract: The 10th Asia-Europe workshop in "Concepts in Information Theory and Communications" AEW10 was held in Boppard, Germany on June 21-23, 2017. It is based on a longstanding cooperation between Asian and European scientists. The first workshop was held in Eindhoven, the Netherlands in 1989. The idea of the workshop is threefold: 1) to improve the communication between the scientist in the different p… ▽ More The 10th Asia-Europe workshop in "Concepts in Information Theory and Communications" AEW10 was held in Boppard, Germany on June 21-23, 2017. It is based on a longstanding cooperation between Asian and European scientists. The first workshop was held in Eindhoven, the Netherlands in 1989. The idea of the workshop is threefold: 1) to improve the communication between the scientist in the different parts of the world; 2) to exchange knowledge and ideas; and 3) to pay a tribute to a well respected and special scientist. △ Less

Submitted 27 July, 2017; originally announced July 2017.

Comments: 44 pages, editors for the proceedings: Yanling Chen and A. J. Han Vinck

MSC Class: 68P30; 94A05

On Zero Error Capacity of Nearest Neighbor Error Channels with Multilevel Alphabet

Authors: Takafumi Nakano, Tadashi Wadayama

Abstract: This paper studies the zero error capacity of the Nearest Neighbor Error (NNE) channels with a multilevel alphabet. In the NNE channels, a transmitted symbol is a $d$-tuple of elements in $\{0,1,2,\dots, n-1 \}$. It is assumed that only one element error to a nearest neighbor element in a transmitted symbol can occur. The NNE channels can be considered as a special type of limited magnitude error… ▽ More This paper studies the zero error capacity of the Nearest Neighbor Error (NNE) channels with a multilevel alphabet. In the NNE channels, a transmitted symbol is a $d$-tuple of elements in $\{0,1,2,\dots, n-1 \}$. It is assumed that only one element error to a nearest neighbor element in a transmitted symbol can occur. The NNE channels can be considered as a special type of limited magnitude error channels, and it is closely related to error models for flash memories. In this paper, we derive a lower bound of the zero error capacity of the NNE channels based on a result of the perfect Lee codes. An upper bound of the zero error capacity of the NNE channels is also derived from a feasible solution of a linear programming problem defined based on the confusion graphs of the NNE channels. As a result, a concise formula of the zero error capacity is obtained using the lower and upper bounds. △ Less

Submitted 30 January, 2017; originally announced January 2017.

Construction of Fixed Rate Non-Binary WOM Codes based on Integer Programming

Authors: Yoju Fu**o, Tadashi Wadayama

Abstract: In this paper, we propose a construction of non-binary WOM (Write-Once-Memory) codes for WOM storages such as flash memories. The WOM codes discussed in this paper are fixed rate WOM codes where messages in a fixed alphabet of size $M$ can be sequentially written in the WOM storage at least $t^*$-times. In this paper, a WOM storage is modeled by a state transition graph. The proposed construction… ▽ More In this paper, we propose a construction of non-binary WOM (Write-Once-Memory) codes for WOM storages such as flash memories. The WOM codes discussed in this paper are fixed rate WOM codes where messages in a fixed alphabet of size $M$ can be sequentially written in the WOM storage at least $t^*$-times. In this paper, a WOM storage is modeled by a state transition graph. The proposed construction has the following two features. First, it includes a systematic method to determine the encoding regions in the state transition graph. Second, the proposed construction includes a labeling method for states by using integer programming. Several novel WOM codes for $q$ level flash memories with 2 cells are constructed by the proposed construction. They achieve the worst numbers of writes $t^*$ that meet the known upper bound in many cases. In addition, we constructed fixed rate non-binary WOM codes with the capability to reduce ICI (inter cell interference) of flash cells. One of the advantages of the proposed construction is its flexibility. It can be applied to various storage devices, to various dimensions (i.e, number of cells), and various kind of additional constraints. △ Less

Submitted 28 January, 2017; originally announced January 2017.

Analysis of Breakdown Probability of Wireless Sensor Networks with Unreliable Relay Nodes

Authors: Takayuki Nozaki, Takafumi Nakano, Tadashi Wadayama

Abstract: In the present paper, we derive an upper bound of the average network breakdown probability of packet networks with unreliable relay nodes. We here assume that relay nodes get independently broken with a given node breakdown probability. A survivor graph is the induced subgraph obtained by removing the broken relay nodes and their connecting edges from the original graph. If the survivor network i… ▽ More In the present paper, we derive an upper bound of the average network breakdown probability of packet networks with unreliable relay nodes. We here assume that relay nodes get independently broken with a given node breakdown probability. A survivor graph is the induced subgraph obtained by removing the broken relay nodes and their connecting edges from the original graph. If the survivor network is disconnected, we consider a network breakdown happens. The primal contribution of the paper is to derive an upper bound of the average network breakdown probability, where the expectation is taken over a regular graph ensemble. The proof of the bound is based on a natural one-to-one correspondence between a regular graph and a regular bipartite graph, and also on enumeration of bipartite graphs satisfying certain conditions. This proof argument is inspired by the analysis of weight distribution for low-density parity-check codes. Compared with estimates of the average network breakdown probability obtained by computer experiments, it is observed that the upper bound provides the values which are not only upper bounds but also precise estimates of the network breakdown probability when the node breakdown probability is small. △ Less

Submitted 24 January, 2017; originally announced January 2017.

Comments: 5 pages, 4 figures, submitted to ISIT2017

Sparse Signal Recovery for Binary Compressed Sensing by Majority Voting Neural Networks

Authors: Daisuke Ito, Tadashi Wadayama

Abstract: In this paper, we propose majority voting neural networks for sparse signal recovery in binary compressed sensing. The majority voting neural network is composed of several independently trained feedforward neural networks employing the sigmoid function as an activation function. Our empirical study shows that a choice of a loss function used in training processes for the network is of prime impor… ▽ More In this paper, we propose majority voting neural networks for sparse signal recovery in binary compressed sensing. The majority voting neural network is composed of several independently trained feedforward neural networks employing the sigmoid function as an activation function. Our empirical study shows that a choice of a loss function used in training processes for the network is of prime importance. We found a loss function suitable for sparse signal recovery, which includes a cross entropy-like term and an $L_1$ regularized term. From the experimental results, we observed that the majority voting neural network achieves excellent recovery performance, which is approaching the optimal performance as the number of component nets grows. The simple architecture of the majority voting neural networks would be beneficial for both software and hardware implementations. △ Less

Submitted 29 October, 2016; originally announced October 2016.

Comments: 6 pages

Performance Analysis based on Density Evolution on Fault Erasure Belief Propagation Decoder

Authors: Hiroki Mori, Tadashi Wadayama

Abstract: In this paper, we will present an analysis on the fault erasure BP decoders based on the density evolution. In the fault BP decoder, messages exchanged in a BP process are stochastically corrupted due to unreliable logic gates and flip-flops; i.e., we assume circuit components with transient faults. We derived a set of the density evolution equations for the fault erasure BP processes. Our density… ▽ More In this paper, we will present an analysis on the fault erasure BP decoders based on the density evolution. In the fault BP decoder, messages exchanged in a BP process are stochastically corrupted due to unreliable logic gates and flip-flops; i.e., we assume circuit components with transient faults. We derived a set of the density evolution equations for the fault erasure BP processes. Our density evolution analysis reveals the asymptotic behaviors of the estimation error probability of the fault erasure BP decoders. In contrast to the fault free cases, it is observed that the error probabilities of the fault erasure BP decoder converge to positive values, and that there exists a discontinuity in an error curve corresponding to the fault BP threshold. It is also shown that an message encoding technique provides higher fault BP thresholds than those of the original decoders at the cost of increased circuit size. △ Less

Submitted 26 January, 2016; originally announced January 2016.

Comments: 12 pages, submitted to ISIT 2016

Bounds on Asymptotic Rate of Capacitive Crosstalk Avoidance Codes for On-chip Buses

Authors: Tadashi Wadayama, Taisuke Izumi

Abstract: In order to prevent the capacitive crosstalk in on-chip buses, several types of capacitive crosstalk avoidance codes have been devised. These codes are designed to prohibit transition patterns prone to the capacity crosstalk from any consecutive two words transmitted to on-chip buses. This paper provides a rigorous analysis on the asymptotic rate of (p,q)-transition free word sequences under the a… ▽ More In order to prevent the capacitive crosstalk in on-chip buses, several types of capacitive crosstalk avoidance codes have been devised. These codes are designed to prohibit transition patterns prone to the capacity crosstalk from any consecutive two words transmitted to on-chip buses. This paper provides a rigorous analysis on the asymptotic rate of (p,q)-transition free word sequences under the assumption that coding is based on a pair of a stateful encoder and a stateless decoder. The symbols p and q represent k-bit transition patterns that should not be appeared in any consecutive two words at the same adjacent k-bit positions. It is proved that the maximum rate of the sequences equals to the subgraph domatic number of (p,q)-transition free graph. Based on the theoretical results on the subgraph domatic partition problem, a pair of lower and upper bounds on the asymptotic rate is derived. We also present that the asymptotic rate 0.8325 is achievable for the (10,01)-transition free word sequences. △ Less

Submitted 25 January, 2016; originally announced January 2016.

Comments: 10 pages, 2 figures, submitted to ISIT 2016

Evaluation of Symmetric Mutual Information of the Simplified TDMR Channel Model

Authors: Tadashi Wadayama

Abstract: In the present paper, a simplified two-dimensional magnetic recording (TDMR) channel model is proposed in order to capture the qualitative features of writing and read-back processes of TDMR systems.The proposed channel model incorporates the effects of both linear interference from adjacent bit-cells and signal-dependent noise due to irregular grain boundaries between adjacent bit-cells. The simp… ▽ More In the present paper, a simplified two-dimensional magnetic recording (TDMR) channel model is proposed in order to capture the qualitative features of writing and read-back processes of TDMR systems.The proposed channel model incorporates the effects of both linear interference from adjacent bit-cells and signal-dependent noise due to irregular grain boundaries between adjacent bit-cells. The simplicity of the proposed model enables us to derive the closed form of the conditional PDF representing the probabilistic nature of the channel. The conditional PDF is Gaussian distributed and is parameterized by a signal-dependent covariance matrix. Based on this conditional PDF, a Monte Carlo method for approximating the symmetric mutual information of this channel is developed. The symmetric mutual information is closely related to the areal density limit for TDMR systems. The numerical results suggest that we may need low-rate coding, e.g., 2/3 or 1/2, when the jitter-like noise becomes dominant. △ Less

Submitted 31 March, 2015; originally announced March 2015.

Index ARQ Protocol for Reliable Contents Distribution over Broadcast Channels

Authors: Takahiro Oshima, Tadashi Wadayama

Abstract: In the present paper, we propose a broadcast ARQ protocol based on the concept of index coding. In the proposed scenario, a server wishes to transmit a finite sequence of packets to multiple receivers via a broadcast channel with packet erasures until all of the receivers successfully receive all of the packets. In the retransmission phase, the server produces a coded packet as a retransmitted pac… ▽ More In the present paper, we propose a broadcast ARQ protocol based on the concept of index coding. In the proposed scenario, a server wishes to transmit a finite sequence of packets to multiple receivers via a broadcast channel with packet erasures until all of the receivers successfully receive all of the packets. In the retransmission phase, the server produces a coded packet as a retransmitted packet based on the side-information sent from the receivers via feedback channels. A notable feature of the proposed protocol is that the decoding process at the receiver side has low decoding complexity because only a small number of addition operations are needed in order to recover an intended packet. This feature may be preferable for reducing the power consumption of receivers. The throughput performance of the proposed protocol is close to that of the ideal FEC throughput performance when the erasure probability is less than $0.1$. This implies that the proposed protocol provides almost optimal throughput performance in such a regime. △ Less

Submitted 31 March, 2015; originally announced March 2015.

Subgraph Domatic Problem and Writing Capacity of Memory Devises with Restricted State Transitions

Authors: Tadashi Wadayama, Taizuke Izumi, Hirotaka Ono

Abstract: A code design problem for memory devises with restricted state transitions is formulated as a combinatorial optimization problem that is called a subgraph domatic partition (subDP) problem. If any neighbor set of a given state transition graph contains all the colors, then the coloring is said to be valid. The goal of a subDP problem is to find a valid coloring with the largest number of colors fo… ▽ More A code design problem for memory devises with restricted state transitions is formulated as a combinatorial optimization problem that is called a subgraph domatic partition (subDP) problem. If any neighbor set of a given state transition graph contains all the colors, then the coloring is said to be valid. The goal of a subDP problem is to find a valid coloring with the largest number of colors for a subgraph of a given directed graph. The number of colors in an optimal valid coloring gives the writing capacity of a given state transition graph. The subDP problems are computationally hard; it is proved to be NP-complete in this paper. One of our main contributions in this paper is to show the asymptotic behavior of the writing capacity $C(G)$ for sequences of dense bidirectional graphs, that is given by C(G)=Omega(n/ln n) where n is the number of nodes. A probabilistic method called Lovasz local lemma (LLL) plays an essential role to derive the asymptotic expression. △ Less

Submitted 19 January, 2015; originally announced January 2015.

Comments: 7 pages

Band Splitting Permutations for Spatially Coupled LDPC Codes Enhancing Burst Erasure Immunity

Authors: Hiroki Mori, Tadashi Wadayama

Abstract: It is well known that spatially coupled (SC) codes with erasure-BP decoding have powerful error correcting capability over memoryless erasure channels. However, the decoding performance of SC-codes significantly degrades when they are used over burst erasure channels. In this paper, we propose band splitting permutations (BSP) suitable for $(l,r,L)$ SC-codes. The BSP splits a diagonal band in a ba… ▽ More It is well known that spatially coupled (SC) codes with erasure-BP decoding have powerful error correcting capability over memoryless erasure channels. However, the decoding performance of SC-codes significantly degrades when they are used over burst erasure channels. In this paper, we propose band splitting permutations (BSP) suitable for $(l,r,L)$ SC-codes. The BSP splits a diagonal band in a base matrix into multiple bands in order to enhance the span of the stop** sets in the base matrix. As theoretical performance guarantees, lower and upper bounds on the maximal burst correctable length of the permuted $(l,r,L)$ SC-codes are presented. Those bounds indicate that the maximal correctable burst ratio of the permuted SC-codes converges to 1/k where k=r/l. This implies the asymptotic optimality of the permuted SC-codes in terms of burst erasure correction. △ Less

Submitted 19 January, 2015; originally announced January 2015.

Comments: 5 pages, submitted to ISIT 2015

Bitwise MAP Algorithm for Group Testing based on Holographic Transformation

Authors: Tadashi Wadayama, Taisuke Izumi

Abstract: In this paper, an exact bitwise MAP (Maximum A Posteriori) estimation algorithm for group testing problems is presented. We assume a simplest non-adaptive group testing scenario including N-objects with binary status and M-disjunctive tests. If a group contains a positive object, the test result for the group is assumed to be one; otherwise, the test result becomes zero. Our inference problem is t… ▽ More In this paper, an exact bitwise MAP (Maximum A Posteriori) estimation algorithm for group testing problems is presented. We assume a simplest non-adaptive group testing scenario including N-objects with binary status and M-disjunctive tests. If a group contains a positive object, the test result for the group is assumed to be one; otherwise, the test result becomes zero. Our inference problem is to evaluate the posterior probabilities of the objects from the observation of M-test results and from our knowledge on the prior probabilities for objects. The heart of the algorithm is the dual expression of the posterior values. The derivation of the dual expression can be naturally described based on a holographic transformation to the normal factor graph (NFG) representing the inference problem. △ Less

Submitted 17 January, 2014; originally announced January 2014.

Comments: 6 pages, 5 figures

An Analysis on Minimum s-t Cut Capacity of Random Graphs with Specified Degree Distribution

Authors: Yuki Fujii, Tadashi Wadayama

Abstract: The capacity (or maximum flow) of an unicast network is known to be equal to the minimum s-t cut capacity due to the max-flow min-cut theorem. If the topology of a network (or link capacities) is dynamically changing or unknown, it is not so trivial to predict statistical properties on the maximum flow of the network. In this paper, we present a probabilistic analysis for evaluating the accumulate… ▽ More The capacity (or maximum flow) of an unicast network is known to be equal to the minimum s-t cut capacity due to the max-flow min-cut theorem. If the topology of a network (or link capacities) is dynamically changing or unknown, it is not so trivial to predict statistical properties on the maximum flow of the network. In this paper, we present a probabilistic analysis for evaluating the accumulate distribution of the minimum s-t cut capacity on random graphs. The graph ensemble treated in this paper consists of weighted graphs with arbitrary specified degree distribution. The main contribution of our work is a lower bound for the accumulate distribution of the minimum s-t cut capacity. From some computer experiments, it is observed that the lower bound derived here reflects the actual statistical behavior of the minimum s-t cut capacity of random graphs with specified degrees. △ Less

Submitted 31 January, 2013; originally announced January 2013.

Comments: 5 pages, Submitted to ISIT 2013. arXiv admin note: substantial text overlap with arXiv:1202.0876

Non-Adaptive Group Testing based on Sparse Pooling Graphs

Authors: Tadashi Wadayama

Abstract: In this paper, an information theoretic analysis on non-adaptive group testing schemes based on sparse pooling graphs is presented. The binary status of the objects to be tested are modeled by i.i.d. Bernoulli random variables with probability p. An (l, r, n)-regular pooling graph is a bipartite graph with left node degree l and right node degree r, where n is the number of left nodes. Two scenari… ▽ More In this paper, an information theoretic analysis on non-adaptive group testing schemes based on sparse pooling graphs is presented. The binary status of the objects to be tested are modeled by i.i.d. Bernoulli random variables with probability p. An (l, r, n)-regular pooling graph is a bipartite graph with left node degree l and right node degree r, where n is the number of left nodes. Two scenarios are considered: a noiseless setting and a noisy one. The main contributions of this paper are direct part theorems that give conditions for the existence of an estimator achieving arbitrary small estimation error probability. The direct part theorems are proved by averaging an upper bound on estimation error probability of the typical set estimator over an (l,r, n)-regular pooling graph ensemble. Numerical results indicate sharp threshold behaviors in the asymptotic regime. △ Less

Submitted 26 April, 2013; v1 submitted 31 January, 2013; originally announced January 2013.

Comments: 20 pages

Finite Length Analysis on Listing Failure Probability of Invertible Bloom Lookup Tables

Authors: Daichi Yugawa, Tadashi Wadayama

Abstract: The Invertible Bloom Lookup Tables (IBLT) is a data structure which supports insertion, deletion, retrieval and listing operations of the key-value pair. The IBLT can be used to realize efficient set reconciliation for database synchronization. The most notable feature of the IBLT is the complete listing operation of the key-value pairs based on the algorithm similar to the peeling algorithm for l… ▽ More The Invertible Bloom Lookup Tables (IBLT) is a data structure which supports insertion, deletion, retrieval and listing operations of the key-value pair. The IBLT can be used to realize efficient set reconciliation for database synchronization. The most notable feature of the IBLT is the complete listing operation of the key-value pairs based on the algorithm similar to the peeling algorithm for low-density generator-matrix (LDGM) codes. In this paper, we will present a stop** set (SS) analysis for the IBLT which reveals finite length behaviors of the listing failure probability. The key of the analysis is enumeration of the number of stop** matrices of given size. We derived a novel recursive formula useful for computationally efficient enumeration. An upper bound on the listing failure probability based on the union bound accurately captures the error floor behaviors. It will be shown that, in the error floor region, the dominant SS have size 2. We propose a simple modification on hash functions, which are called SS avoiding hash functions, for preventing occurrences of the SS of size 2. △ Less

Submitted 30 January, 2013; originally announced January 2013.

Comments: 5pages, Submitted to ISIT 2013

A New Direction for Counting Perfect Matchings

Authors: Taisuke Izumi, Tadashi Wadayama

Abstract: In this paper, we present a new exact algorithm for counting perfect matchings, which relies on neither inclusion-exclusion principle nor tree-decompositions. For any bipartite graph of $2n$ nodes and $Δn$ edges such that $Δ\geq 3$, our algorithm runs with $O^{\ast}(2^{(1 - 1/O(Δ\log Δ))n})$ time and exponential space. Compared to the previous algorithms, it achieves a better time bound in the sen… ▽ More In this paper, we present a new exact algorithm for counting perfect matchings, which relies on neither inclusion-exclusion principle nor tree-decompositions. For any bipartite graph of $2n$ nodes and $Δn$ edges such that $Δ\geq 3$, our algorithm runs with $O^{\ast}(2^{(1 - 1/O(Δ\log Δ))n})$ time and exponential space. Compared to the previous algorithms, it achieves a better time bound in the sense that the performance degradation to the increase of $Δ$ is quite slower. The main idea of our algorithm is a new reduction to the problem of computing the cut-weight distribution of the input graph. The primary ingredient of this reduction is MacWilliams Identity derived from elementary coding theory. The whole of our algorithm is designed by combining that reduction with a non-trivial fast algorithm computing the cut-weight distribution. To the best of our knowledge, the approach posed in this paper is new and may be of independent interest. △ Less

Submitted 11 August, 2012; originally announced August 2012.

Comments: The 53rd Annual Symposium on Foundations of Computer Science (FOCS 2012)

A Coding Theoretic Approach for Evaluating Accumulate Distribution on Minimum Cut Capacity of Weighted Random Graphs

Authors: Tadashi Wadayama, Yuki Fujii

Abstract: The multicast capacity of a directed network is closely related to the $s$-$t$ maximum flow, which is equal to the $s$-$t$ minimum cut capacity due to the max-flow min-cut theorem. If the topology of a network (or link capacities) is dynamically changing or have stochastic nature, it is not so trivial to predict statistical properties on the maximum flow. In this paper, we present a coding theoret… ▽ More The multicast capacity of a directed network is closely related to the $s$-$t$ maximum flow, which is equal to the $s$-$t$ minimum cut capacity due to the max-flow min-cut theorem. If the topology of a network (or link capacities) is dynamically changing or have stochastic nature, it is not so trivial to predict statistical properties on the maximum flow. In this paper, we present a coding theoretic approach for evaluating the accumulate distribution of the minimum cut capacity of weighted random graphs. The main feature of our approach is to utilize the correspondence between the cut space of a graph and a binary LDGM (low-density generator-matrix) code with column weight 2. The graph ensemble treated in the paper is a weighted version of Erdős-Rényi random graph ensemble. The main contribution of our work is a combinatorial lower bound for the accumulate distribution of the minimum cut capacity. From some computer experiments, it is observed that the lower bound derived here reflects the actual statistical behavior of the minimum cut capacity. △ Less

Submitted 4 February, 2012; originally announced February 2012.

Comments: 5 pages, 2 figures, submitted to IEEE ISIT 2012