Showing 1–7 of 7 results for author: Ismailov, V

On the Kolmogorov neural networks

Authors: Aysu Ismayilova, Vugar Ismailov

Abstract: In this paper, we show that the Kolmogorov two hidden layer neural network model with a continuous, discontinuous bounded or unbounded activation function in the second hidden layer can precisely represent continuous, discontinuous bounded and all unbounded multivariate functions, respectively. In this paper, we show that the Kolmogorov two hidden layer neural network model with a continuous, discontinuous bounded or unbounded activation function in the second hidden layer can precisely represent continuous, discontinuous bounded and all unbounded multivariate functions, respectively. △ Less

Submitted 31 October, 2023; originally announced November 2023.

Comments: 14 pages, 1 figure; this article uses material from arXiv:2012.03016

MSC Class: 46A22; 46E10; 46N60; 68T05; 92B20

Measure theoretic results for approximation by neural networks with limited weights

Authors: Vugar Ismailov, Ekrem Savas

Abstract: In this paper, we study approximation properties of single hidden layer neural networks with weights varying on finitely many directions and thresholds from an open interval. We obtain a necessary and at the same time sufficient measure theoretic condition for density of such networks in the space of continuous functions. Further, we prove a density result for neural networks with a specifically c… ▽ More In this paper, we study approximation properties of single hidden layer neural networks with weights varying on finitely many directions and thresholds from an open interval. We obtain a necessary and at the same time sufficient measure theoretic condition for density of such networks in the space of continuous functions. Further, we prove a density result for neural networks with a specifically constructed activation function and a fixed number of neurons. △ Less

Submitted 4 April, 2023; originally announced April 2023.

Comments: 12 pages

MSC Class: 41A30; 41A63; 92B20

Journal ref: Numer. Funct. Anal. Optim. 38 (2017), no. 7, 819--830

Approximation error of single hidden layer neural networks with fixed weights

Authors: Vugar Ismailov

Abstract: This paper provides an explicit formula for the approximation error of single hidden layer neural networks with two fixed weights. This paper provides an explicit formula for the approximation error of single hidden layer neural networks with two fixed weights. △ Less

Submitted 4 March, 2022; v1 submitted 7 February, 2022; originally announced February 2022.

Comments: 13 pages, an example added, a typo corrected

MSC Class: 41A30; 41A63; 68T05; 92B20

Approximation capability of two hidden layer feedforward neural networks with fixed weights

Authors: Namig J. Guliyev, Vugar E. Ismailov

Abstract: We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the sing… ▽ More We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the single hidden layer feedforward neural network (SLFN) model, since SLFNs with fixed weights do not have the capability of approximating multivariate functions. △ Less

Submitted 22 January, 2021; originally announced January 2021.

Comments: 13 pages, 3 figures; this article uses the algorithm from arXiv:1708.06219; for associated SageMath worksheet, see https://sites.google.com/site/njguliyev/papers/tlfn

MSC Class: 41A30; 41A63; 65D15; 68T05; 92B20 ACM Class: C.1.3; F.1.1; I.2.6; I.5.1

Journal ref: Neurocomputing, 316 (2018), 262-269

A three layer neural network can represent any multivariate function

Authors: Vugar Ismailov

Abstract: In 1987, Hecht-Nielsen showed that any continuous multivariate function can be implemented by a certain type three-layer neural network. This result was very much discussed in neural network literature. In this paper we prove that not only continuous functions but also all discontinuous functions can be implemented by such neural networks. In 1987, Hecht-Nielsen showed that any continuous multivariate function can be implemented by a certain type three-layer neural network. This result was very much discussed in neural network literature. In this paper we prove that not only continuous functions but also all discontinuous functions can be implemented by such neural networks. △ Less

Submitted 15 January, 2022; v1 submitted 5 December, 2020; originally announced December 2020.

Comments: 9 pages; a few references added

MSC Class: 26B40; 46A22; 46E10; 46N60; 68T05; 92B20

On the approximation by single hidden layer feedforward neural networks with fixed weights

Authors: Namig J. Guliyev, Vugar E. Ismailov

Abstract: Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the nu… ▽ More Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the number of neurons in the hidden layer. The more this number, the more the probability of the considered network to give precise results. In this note, we constructively prove that SLFNs with the fixed weight $1$ and two neurons in the hidden layer can approximate any continuous function on a compact subset of the real line. The applicability of this result is demonstrated in various numerical examples. Finally, we show that SLFNs with fixed weights cannot approximate all continuous multivariate functions. △ Less

Submitted 21 August, 2017; originally announced August 2017.

Comments: 17 pages, 5 figures, submitted; for associated SageMath worksheet, see https://sites.google.com/site/njguliyev/papers/monic-sigmoidal

MSC Class: 41A30; 41A63; 65D15; 68T05; 92B20 ACM Class: C.1.3; F.1.1; I.2.6; I.5.1

Journal ref: Neural Networks, 98 (2018), 296-304

A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function

Authors: Namig J. Guliyev, Vugar E. Ismailov

Abstract: The possibility of approximating a continuous function on a compact subset of the real line by a feedforward single hidden layer neural network with a sigmoidal activation function has been studied in many papers. Such networks can approximate an arbitrary continuous function provided that an unlimited number of neurons in a hidden layer is permitted. In this paper, we consider constructive approx… ▽ More The possibility of approximating a continuous function on a compact subset of the real line by a feedforward single hidden layer neural network with a sigmoidal activation function has been studied in many papers. Such networks can approximate an arbitrary continuous function provided that an unlimited number of neurons in a hidden layer is permitted. In this paper, we consider constructive approximation on any finite interval of $\mathbb{R}$ by neural networks with only one neuron in the hidden layer. We construct algorithmically a smooth, sigmoidal, almost monotone activation function $σ$ providing approximation to an arbitrary continuous function within any degree of accuracy. This algorithm is implemented in a computer program, which computes the value of $σ$ at any reasonable point of the real axis. △ Less

Submitted 20 May, 2016; v1 submitted 31 December, 2015; originally announced January 2016.

Comments: 12 pages, 1 figure; to be published in Neural Computation; for associated SageMath worksheet, see http://sites.google.com/site/njguliyev/papers/sigmoidal

MSC Class: 41A30; 65D15; 92B20

Journal ref: Neural Computation, 28 (2016), no. 7, 1289-1304