Analysis and FPGA based Implementation of Permutation Binary Neural Networks
Authors:
Mikito Onuki,
Kento Saka,
Toshimichi Saito
Abstract:
This paper studies a permutation binary neural network characterized by local binary connections, global permutation connections, and the signum activation function. Depending on the permutation connections, the network can generate various periodic orbits of binary vectors. Especially, we focus on globally stable periodic orbits such that almost all initial points fall into the orbits. In order t…
▽ More
This paper studies a permutation binary neural network characterized by local binary connections, global permutation connections, and the signum activation function. Depending on the permutation connections, the network can generate various periodic orbits of binary vectors. Especially, we focus on globally stable periodic orbits such that almost all initial points fall into the orbits. In order to explore the periodic orbits, we present a simple evolutionary algorithm. Applying the algorithm to typical examples of PBNNs, existence of a variety of periodic orbits is clarified. Presenting an FPGA based hardware prototype, typical periodic orbits are confirmed experimentally. The hardware will be developed into various engineering applications such that stable control signals of switching circuits and stable approximation signals of time-series.
△ Less
Submitted 2 June, 2023;
originally announced June 2023.
Fast Singular Value Shrinkage with Chebyshev Polynomial Approximation Based on Signal Sparsity
Authors:
Masaki Onuki,
Shunsuke Ono,
Keiichiro Shirai,
Yuichi Tanaka
Abstract:
We propose an approximation method for thresholding of singular values using Chebyshev polynomial approximation (CPA). Many signal processing problems require iterative application of singular value decomposition (SVD) for minimizing the rank of a given data matrix with other cost functions and/or constraints, which is called matrix rank minimization. In matrix rank minimization, singular values o…
▽ More
We propose an approximation method for thresholding of singular values using Chebyshev polynomial approximation (CPA). Many signal processing problems require iterative application of singular value decomposition (SVD) for minimizing the rank of a given data matrix with other cost functions and/or constraints, which is called matrix rank minimization. In matrix rank minimization, singular values of a matrix are shrunk by hard-thresholding, soft-thresholding, or weighted soft-thresholding. However, the computational cost of SVD is generally too expensive to handle high dimensional signals such as images; hence, in this case, matrix rank minimization requires enormous computation time. In this paper, we leverage CPA to (approximately) manipulate singular values without computing singular values and vectors. The thresholding of singular values is expressed by a multiplication of certain matrices, which is derived from a characteristic of CPA. The multiplication is also efficiently computed using the sparsity of signals. As a result, the computational cost is significantly reduced. Experimental results suggest the effectiveness of our method through several image processing applications based on matrix rank minimization with nuclear norm relaxation in terms of computation time and approximation precision.
△ Less
Submitted 19 May, 2017;
originally announced May 2017.
