Showing 1–20 of 20 results for author: Ismailov, V

A note on the approximation by a sum of two algebras

Authors: Aida Asgarova, Vugar Ismailov

Abstract: We consider the problem of approximation of a continuous function $f$ defined on a compact metric space $X$ by elements from a sum of two algebras. We prove a de la Vallée Poussin type theorem, which estimates the approximation error $E(f)$ from below. We also obtain a duality formula for the precise computation of $E(f)$. We consider the problem of approximation of a continuous function $f$ defined on a compact metric space $X$ by elements from a sum of two algebras. We prove a de la Vallée Poussin type theorem, which estimates the approximation error $E(f)$ from below. We also obtain a duality formula for the precise computation of $E(f)$. △ Less

Submitted 15 June, 2024; v1 submitted 16 May, 2024; originally announced May 2024.

Comments: 10 pages, typos corrected, references added; this paper includes material from arXiv:2204.07448

On the Kolmogorov neural networks

Authors: Aysu Ismayilova, Vugar Ismailov

Abstract: In this paper, we show that the Kolmogorov two hidden layer neural network model with a continuous, discontinuous bounded or unbounded activation function in the second hidden layer can precisely represent continuous, discontinuous bounded and all unbounded multivariate functions, respectively. In this paper, we show that the Kolmogorov two hidden layer neural network model with a continuous, discontinuous bounded or unbounded activation function in the second hidden layer can precisely represent continuous, discontinuous bounded and all unbounded multivariate functions, respectively. △ Less

Submitted 31 October, 2023; originally announced November 2023.

Comments: 14 pages, 1 figure; this article uses material from arXiv:2012.03016

MSC Class: 46A22; 46E10; 46N60; 68T05; 92B20

Measure theoretic results for approximation by neural networks with limited weights

Authors: Vugar Ismailov, Ekrem Savas

Abstract: In this paper, we study approximation properties of single hidden layer neural networks with weights varying on finitely many directions and thresholds from an open interval. We obtain a necessary and at the same time sufficient measure theoretic condition for density of such networks in the space of continuous functions. Further, we prove a density result for neural networks with a specifically c… ▽ More In this paper, we study approximation properties of single hidden layer neural networks with weights varying on finitely many directions and thresholds from an open interval. We obtain a necessary and at the same time sufficient measure theoretic condition for density of such networks in the space of continuous functions. Further, we prove a density result for neural networks with a specifically constructed activation function and a fixed number of neurons. △ Less

Submitted 4 April, 2023; originally announced April 2023.

Comments: 12 pages

MSC Class: 41A30; 41A63; 92B20

Journal ref: Numer. Funct. Anal. Optim. 38 (2017), no. 7, 819--830

A Chebyshev type alternation theorem for best approximation by a sum of two algebras

Authors: Aida Asgarova, Ali Huseynli, Vugar Ismailov

Abstract: Let $X$ be a compact metric space, $C(X)$ be the space of continuous real-valued functions on $X$, and $A_1$, $A_2$ be two closed subalgebras of $C(X)$ containing constant functions. We consider the problem of approximation of a function $f\in C(X)$ by elements from $A_1+A_2$. We prove a Chebyshev type alternation theorem for a function $u_0\in A_1+A_2$ to be a best approximation to $f$. Let $X$ be a compact metric space, $C(X)$ be the space of continuous real-valued functions on $X$, and $A_1$, $A_2$ be two closed subalgebras of $C(X)$ containing constant functions. We consider the problem of approximation of a function $f\in C(X)$ by elements from $A_1+A_2$. We prove a Chebyshev type alternation theorem for a function $u_0\in A_1+A_2$ to be a best approximation to $f$. △ Less

Submitted 23 November, 2023; v1 submitted 15 April, 2022; originally announced April 2022.

Comments: 9 pages, references updated, to appear in Proceedings of the Edinburgh Mathematical Society

Approximation error of single hidden layer neural networks with fixed weights

Authors: Vugar Ismailov

Abstract: This paper provides an explicit formula for the approximation error of single hidden layer neural networks with two fixed weights. This paper provides an explicit formula for the approximation error of single hidden layer neural networks with two fixed weights. △ Less

Submitted 4 March, 2022; v1 submitted 7 February, 2022; originally announced February 2022.

Comments: 13 pages, an example added, a typo corrected

MSC Class: 41A30; 41A63; 68T05; 92B20

Approximation capability of two hidden layer feedforward neural networks with fixed weights

Authors: Namig J. Guliyev, Vugar E. Ismailov

Abstract: We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the sing… ▽ More We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the single hidden layer feedforward neural network (SLFN) model, since SLFNs with fixed weights do not have the capability of approximating multivariate functions. △ Less

Submitted 22 January, 2021; originally announced January 2021.

Comments: 13 pages, 3 figures; this article uses the algorithm from arXiv:1708.06219; for associated SageMath worksheet, see https://sites.google.com/site/njguliyev/papers/tlfn

MSC Class: 41A30; 41A63; 65D15; 68T05; 92B20 ACM Class: C.1.3; F.1.1; I.2.6; I.5.1

Journal ref: Neurocomputing, 316 (2018), 262-269

A three layer neural network can represent any multivariate function

Authors: Vugar Ismailov

Abstract: In 1987, Hecht-Nielsen showed that any continuous multivariate function can be implemented by a certain type three-layer neural network. This result was very much discussed in neural network literature. In this paper we prove that not only continuous functions but also all discontinuous functions can be implemented by such neural networks. In 1987, Hecht-Nielsen showed that any continuous multivariate function can be implemented by a certain type three-layer neural network. This result was very much discussed in neural network literature. In this paper we prove that not only continuous functions but also all discontinuous functions can be implemented by such neural networks. △ Less

Submitted 15 January, 2022; v1 submitted 5 December, 2020; originally announced December 2020.

Comments: 9 pages; a few references added

MSC Class: 26B40; 46A22; 46E10; 46N60; 68T05; 92B20

The double difference property for the class of locally Hölder continuous functions

Authors: Rashid Aliev, Aysel Asgarova, Vugar Ismailov

Abstract: In this paper, we show that the pair of classes of locally Hölder continuous functions (considered on $\mathbb{R}$ and $\mathbb{R}^{2}$, respectively) has the double difference property. In this paper, we show that the pair of classes of locally Hölder continuous functions (considered on $\mathbb{R}$ and $\mathbb{R}^{2}$, respectively) has the double difference property. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Comments: 8 pages

MSC Class: 26B05; 39A70; 39B22; 47B39

Notes on ridge functions and neural networks

Authors: Vugar Ismailov

Abstract: These notes are about ridge functions. Recent years have witnessed a flurry of interest in these functions. Ridge functions appear in various fields and under various guises. They appear in fields as diverse as partial differential equations (where they are called plane waves), computerized tomography and statistics. These functions are also the underpinnings of many central models in neural netwo… ▽ More These notes are about ridge functions. Recent years have witnessed a flurry of interest in these functions. Ridge functions appear in various fields and under various guises. They appear in fields as diverse as partial differential equations (where they are called plane waves), computerized tomography and statistics. These functions are also the underpinnings of many central models in neural networks. We are interested in ridge functions from the point of view of approximation theory. The basic goal in approximation theory is to approximate complicated objects by simpler objects. Among many classes of multivariate functions, linear combinations of ridge functions are a class of simpler functions. These notes study some problems of approximation of multivariate functions by linear combinations of ridge functions. We present here various properties of these functions. The questions we ask are as follows. When can a multivariate function be expressed as a linear combination of ridge functions from a certain class? When do such linear combinations represent each multivariate function? If a precise representation is not possible, can one approximate arbitrarily well? If well approximation fails, how can one compute/estimate the error of approximation, know that a best approximation exists? How can one characterize and construct best approximations? If a smooth function is a sum of arbitrarily behaved ridge functions, can it be expressed as a sum of smooth ridge functions? We also study properties of generalized ridge functions, which are very much related to linear superpositions and Kolmogorov's famous superposition theorem. These notes end with a few applications of ridge functions to the problem of approximation by single and two hidden layer neural networks with a restricted set of weights. We hope that these notes will be useful and interesting to both researchers and students. △ Less

Submitted 28 August, 2020; v1 submitted 28 May, 2020; originally announced May 2020.

Comments: 228 pages, a section and references added

A representation problem for smooth sums of ridge functions

Authors: Rashid Aliev, Vugar Ismailov

Abstract: In this paper we prove that if a multivariate function of a certain smoothness class is represented by a sum of $k$ arbitrarily behaved ridge functions, then it can be represented by a sum of $k$ ridge functions of the same smoothness class and a polynomial of degree at most $k-1$. This solves the problem posed by A. Pinkus in his monograph "Ridge Functions" up to a multivariate polynomial. In this paper we prove that if a multivariate function of a certain smoothness class is represented by a sum of $k$ arbitrarily behaved ridge functions, then it can be represented by a sum of $k$ ridge functions of the same smoothness class and a polynomial of degree at most $k-1$. This solves the problem posed by A. Pinkus in his monograph "Ridge Functions" up to a multivariate polynomial. △ Less

Submitted 27 April, 2020; v1 submitted 3 February, 2020; originally announced February 2020.

Comments: 16 pages, two corollaries added

MSC Class: 26B40; 39B22

On the approximation by single hidden layer feedforward neural networks with fixed weights

Authors: Namig J. Guliyev, Vugar E. Ismailov

Abstract: Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the nu… ▽ More Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the number of neurons in the hidden layer. The more this number, the more the probability of the considered network to give precise results. In this note, we constructively prove that SLFNs with the fixed weight $1$ and two neurons in the hidden layer can approximate any continuous function on a compact subset of the real line. The applicability of this result is demonstrated in various numerical examples. Finally, we show that SLFNs with fixed weights cannot approximate all continuous multivariate functions. △ Less

Submitted 21 August, 2017; originally announced August 2017.

Comments: 17 pages, 5 figures, submitted; for associated SageMath worksheet, see https://sites.google.com/site/njguliyev/papers/monic-sigmoidal

MSC Class: 41A30; 41A63; 65D15; 68T05; 92B20 ACM Class: C.1.3; F.1.1; I.2.6; I.5.1

Journal ref: Neural Networks, 98 (2018), 296-304

A note on the equioscillation theorem for best ridge function approximation

Authors: Vugar Ismailov

Abstract: We consider the approximation of a continuous function, defined on a compact set of the $d$-dimensional Euclidean space, by sums of two ridge functions. We obtain a necessary and sufficient condition for such a sum to be a best approximation. The result resembles the classical Chebyshev equioscillation theorem for polynomial approximation. We consider the approximation of a continuous function, defined on a compact set of the $d$-dimensional Euclidean space, by sums of two ridge functions. We obtain a necessary and sufficient condition for such a sum to be a best approximation. The result resembles the classical Chebyshev equioscillation theorem for polynomial approximation. △ Less

Submitted 27 September, 2016; originally announced September 2016.

Comments: 8 pages

MSC Class: 41A30; 41A50; 46B50; 46E15

On the representation by bivariate ridge functions

Authors: Rashid Aliev, Vugar Ismailov

Abstract: We consider the problem of representation of a bivariate function by sums of ridge functions. We show that if a function of a certain smoothness class is represented by a sum of finitely many, arbitrarily behaved ridge functions, then it can also be represented by a sum of ridge functions of the same smoothness class. As an example, this result is applied to a homogeneous constant coefficient part… ▽ More We consider the problem of representation of a bivariate function by sums of ridge functions. We show that if a function of a certain smoothness class is represented by a sum of finitely many, arbitrarily behaved ridge functions, then it can also be represented by a sum of ridge functions of the same smoothness class. As an example, this result is applied to a homogeneous constant coefficient partial differential equation. △ Less

Submitted 25 June, 2016; originally announced June 2016.

Comments: 8 pages

MSC Class: 26B40; 35C99; 35L25

On the Diliberto-Straus algorithm for the uniform approximation by a sum of two algebras

Authors: Aida Kh. Asgarova, Vugar E. Ismailov

Abstract: In 1951, Diliberto and Straus proposed a levelling algorithm for the uniform approximation of a bivariate function, defined on a rectangle with sides parallel to the coordinate axes, by sums of univariate functions. In the current paper, we consider the problem of approximation of a continuous function defined on a compact Hausdorff space by a sum of two closed algebras containing constants. Under… ▽ More In 1951, Diliberto and Straus proposed a levelling algorithm for the uniform approximation of a bivariate function, defined on a rectangle with sides parallel to the coordinate axes, by sums of univariate functions. In the current paper, we consider the problem of approximation of a continuous function defined on a compact Hausdorff space by a sum of two closed algebras containing constants. Under reasonable assumptions, we show the convergence of the Diliberto-Straus algorithm. For the approximation by sums of univariate functions, it follows that Diliberto-Straus's original result holds for a large class of compact convex sets. △ Less

Submitted 23 March, 2016; originally announced March 2016.

Comments: 16 pages

MSC Class: 41A30; 41A65; 46B28; 65D15

A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function

Authors: Namig J. Guliyev, Vugar E. Ismailov

Abstract: The possibility of approximating a continuous function on a compact subset of the real line by a feedforward single hidden layer neural network with a sigmoidal activation function has been studied in many papers. Such networks can approximate an arbitrary continuous function provided that an unlimited number of neurons in a hidden layer is permitted. In this paper, we consider constructive approx… ▽ More The possibility of approximating a continuous function on a compact subset of the real line by a feedforward single hidden layer neural network with a sigmoidal activation function has been studied in many papers. Such networks can approximate an arbitrary continuous function provided that an unlimited number of neurons in a hidden layer is permitted. In this paper, we consider constructive approximation on any finite interval of $\mathbb{R}$ by neural networks with only one neuron in the hidden layer. We construct algorithmically a smooth, sigmoidal, almost monotone activation function $σ$ providing approximation to an arbitrary continuous function within any degree of accuracy. This algorithm is implemented in a computer program, which computes the value of $σ$ at any reasonable point of the real axis. △ Less

Submitted 20 May, 2016; v1 submitted 31 December, 2015; originally announced January 2016.

Comments: 12 pages, 1 figure; to be published in Neural Computation; for associated SageMath worksheet, see http://sites.google.com/site/njguliyev/papers/sigmoidal

MSC Class: 41A30; 65D15; 92B20

Journal ref: Neural Computation, 28 (2016), no. 7, 1289-1304

A note on the representation of continuous functions by linear superpositions

Authors: Vugar Ismailov

Abstract: We consider the problem of the representation of real continuous functions by linear superpositions $\sum_{i=1}^{k}g_{i}\circ p_{i}$ with continuous $g_{i}$ and $p_{i}$. This problem was considered by many authors. But complete, and at the same time explicit and practical solutions to the problem was given only for the case $k=2$. For $k>2$, a rather practical sufficient condition for the represen… ▽ More We consider the problem of the representation of real continuous functions by linear superpositions $\sum_{i=1}^{k}g_{i}\circ p_{i}$ with continuous $g_{i}$ and $p_{i}$. This problem was considered by many authors. But complete, and at the same time explicit and practical solutions to the problem was given only for the case $k=2$. For $k>2$, a rather practical sufficient condition for the representation can be found in Sternfeld [17] and Sproston, Strauss [16]. In this short note, we give a necessary condition of such kind for the representability of continuous functions. △ Less

Submitted 21 January, 2015; originally announced January 2015.

Journal ref: Expo. Math. 30 (2012), 96-101

On the representation by linear superpositions

Authors: Vugar Ismailov

Abstract: In a number of papers, Y. Sternfeld investigated the problems of representation of continuous and bounded functions by linear superpositions. In particular, he proved that if such representation holds for continuous functions, then it holds for bounded functions. We consider the same problem without involving any topology and establish a rather practical necessary and sufficient condition for repr… ▽ More In a number of papers, Y. Sternfeld investigated the problems of representation of continuous and bounded functions by linear superpositions. In particular, he proved that if such representation holds for continuous functions, then it holds for bounded functions. We consider the same problem without involving any topology and establish a rather practical necessary and sufficient condition for representability of an arbitrary function by linear superpositions. In particular, we show that if some representation by linear superpositions holds for continuous functions, then it holds for all functions. This will lead us to the analogue of the well-known Kolmogorov superposition theorem for multivariate functions on the $d$-dimensional unit cube. △ Less

Submitted 21 January, 2015; originally announced January 2015.

Journal ref: J. Approx. Theory 151 (2008), 113-125

On the proximinality of ridge functions

Authors: Vugar Ismailov

Abstract: Using two results of Garkavi, Medvedev and Khavinson, we give sufficient conditions for proximinality of sums of two ridge functions with bounded and continuous summands in the spaces of bounded and continuous multivariate functions respectively. In the first case, we give an example which shows that the corresponding sufficient condition cannot be made weaker for some subsets of… ▽ More Using two results of Garkavi, Medvedev and Khavinson, we give sufficient conditions for proximinality of sums of two ridge functions with bounded and continuous summands in the spaces of bounded and continuous multivariate functions respectively. In the first case, we give an example which shows that the corresponding sufficient condition cannot be made weaker for some subsets of $\mathbb{R}^{n}$. In the second case, we obtain also a necessary condition for proximinality. All the results are furnished with plenty of examples. The results, examples and following discussions naturally lead us to a conjecture on the proximinality of the considered class of ridge functions. The main purpose of the paper is to draw readers' attention to this conjecture. △ Less

Submitted 30 August, 2007; v1 submitted 29 August, 2007; originally announced August 2007.

Comments: 8 pages

MSC Class: 41A30; 41A50; 41A63

On the theorem of M.Golomb

Authors: Vugar Ismailov

Abstract: Let $X_{1},...,X_{n}$ be compact spaces and $X=X_{1}\times ... \times X_{n}.$ Consider the approximation of a function $f\in C(X)$ by sums $g_{1}(x_{1})+... g_{n}(x_{n}),$ where $g_{i}\in C(X_{i}),$ $i=1,...,n.$ In [8], M.Golomb obtained a formula for the error of this approximation in terms of measures constructed on special points of $X$, called "projection cycles". However, his proof had a ga… ▽ More Let $X_{1},...,X_{n}$ be compact spaces and $X=X_{1}\times ... \times X_{n}.$ Consider the approximation of a function $f\in C(X)$ by sums $g_{1}(x_{1})+... g_{n}(x_{n}),$ where $g_{i}\in C(X_{i}),$ $i=1,...,n.$ In [8], M.Golomb obtained a formula for the error of this approximation in terms of measures constructed on special points of $X$, called "projection cycles". However, his proof had a gap, which was pointed out by Marshall and O'Farrell [15]. But the question if the formula was correct, remained open. The purpose of the paper is to prove that Golomb's formula holds in a stronger form. △ Less

Submitted 10 July, 2008; v1 submitted 26 August, 2007; originally announced August 2007.

Comments: 8 pages; the previous main result has substantially improved

MSC Class: 41A30; 41A50; 41A63

On the approximation by weighted ridge functions

Authors: Vugar Ismailov

Abstract: We characterize the best $L_{2}$ approximation to a multivariate function by linear combinations of ridge functions multiplied by some fixed weight functions. In the special case when the weight functions are constants, we propose explicit formulas for both the best approximation and approximation error. We characterize the best $L_{2}$ approximation to a multivariate function by linear combinations of ridge functions multiplied by some fixed weight functions. In the special case when the weight functions are constants, we propose explicit formulas for both the best approximation and approximation error. △ Less

Submitted 24 August, 2007; originally announced August 2007.

Comments: 8 pages

MSC Class: 41A30; 41A50; 41A63