-
Gradient Descent in Materio
Authors:
Marcus N. Boon,
Hans-Christian Ruiz Euler,
Tao Chen,
Bram van de Ven,
Unai Alegre Ibarra,
Peter A. Bobbert,
Wilfred G. van der Wiel
Abstract:
Deep learning, a multi-layered neural network approach inspired by the brain, has revolutionized machine learning. One of its key enablers has been backpropagation, an algorithm that computes the gradient of a loss function with respect to the weights in the neural network model, in combination with its use in gradient descent. However, the implementation of deep learning in digital computers is i…
▽ More
Deep learning, a multi-layered neural network approach inspired by the brain, has revolutionized machine learning. One of its key enablers has been backpropagation, an algorithm that computes the gradient of a loss function with respect to the weights in the neural network model, in combination with its use in gradient descent. However, the implementation of deep learning in digital computers is intrinsically wasteful, with energy consumption becoming prohibitively high for many applications. This has stimulated the development of specialized hardware, ranging from neuromorphic CMOS integrated circuits and integrated photonic tensor cores to unconventional, material-based computing systems. The learning process in these material systems, taking place, e.g., by artificial evolution or surrogate neural network modelling, is still a complicated and time-consuming process. Here, we demonstrate an efficient and accurate homodyne gradient extraction method for performing gradient descent on the loss function directly in the material system. We demonstrate the method in our recently developed dopant network processing units, where we readily realize all Boolean gates. This shows that gradient descent can in principle be fully implemented in materio using simple electronics, opening up the way to autonomously learning material systems.
△ Less
Submitted 15 May, 2021;
originally announced May 2021.
-
Dopant Network Processing Units: Towards Efficient Neural-network Emulators with High-capacity Nanoelectronic Nodes
Authors:
Hans-Christian Ruiz-Euler,
Unai Alegre-Ibarra,
Bram van de Ven,
Hajo Broersma,
Peter A. Bobbert,
Wilfred G. van der Wiel
Abstract:
The rapidly growing computational demands of deep neural networks require novel hardware designs. Recently, tunable nanoelectronic devices were developed based on hop** electrons through a network of dopant atoms in silicon. These "Dopant Network Processing Units" (DNPUs) are highly energy-efficient and have potentially very high throughput. By adapting the control voltages applied to its termin…
▽ More
The rapidly growing computational demands of deep neural networks require novel hardware designs. Recently, tunable nanoelectronic devices were developed based on hop** electrons through a network of dopant atoms in silicon. These "Dopant Network Processing Units" (DNPUs) are highly energy-efficient and have potentially very high throughput. By adapting the control voltages applied to its terminals, a single DNPU can solve a variety of linearly non-separable classification problems. However, using a single device has limitations due to the implicit single-node architecture. This paper presents a promising novel approach to neural information processing by introducing DNPUs as high-capacity neurons and moving from a single to a multi-neuron framework. By implementing and testing a small multi-DNPU classifier in hardware, we show that feed-forward DNPU networks improve the performance of a single DNPU from 77% to 94% test accuracy on a binary classification task with concentric classes on a plane. Furthermore, motivated by the integration of DNPUs with memristor arrays, we study the potential of using DNPUs in combination with linear layers. We show by simulation that a single-layer MNIST classifier with only 10 DNPUs achieves over 96% test accuracy. Our results pave the road towards hardware neural-network emulators that offer atomic-scale information processing with low latency and energy consumption.
△ Less
Submitted 3 August, 2021; v1 submitted 24 July, 2020;
originally announced July 2020.
-
Single-charge occupation in ambipolar quantum dots
Authors:
A. J. Sousa de Almeida,
A. Marquez Seco,
T. van den Berg,
B. van de Ven,
F. Bruijnes,
S. V. Amitonov,
F. A. Zwanenburg
Abstract:
We demonstrate single-charge occupation of ambipolar quantum dots in silicon via charge sensing. We have fabricated ambipolar quantum dot (QD) devices in a silicon metal-oxide-semiconductor heterostructure comprising a single-electron transistor next to a single-hole transistor. Both QDs can be tuned to simultaneously sense charge transitions of the other. We further detect the few-electron and fe…
▽ More
We demonstrate single-charge occupation of ambipolar quantum dots in silicon via charge sensing. We have fabricated ambipolar quantum dot (QD) devices in a silicon metal-oxide-semiconductor heterostructure comprising a single-electron transistor next to a single-hole transistor. Both QDs can be tuned to simultaneously sense charge transitions of the other. We further detect the few-electron and few-hole regimes in the QDs of our ambipolar device by active charge sensing.
△ Less
Submitted 14 January, 2020;
originally announced January 2020.
-
Time Asymmetric Quantum Theory - I Modifying an Axiom of Quantum Physics
Authors:
A. Bohm,
Mark Loewe,
Bryan Van de Ven
Abstract:
A slight modification of one axiom of quantum theory changes a reversible theory into a time asymmetric theory. Whereas the standard Hilbert space axiom does not distinguish mathematically between the space of states (in-states of scattering theory) and the space of observables (out-``states'' of scattering theory) the new axiom associates states and observables to two different Hardy subspaces…
▽ More
A slight modification of one axiom of quantum theory changes a reversible theory into a time asymmetric theory. Whereas the standard Hilbert space axiom does not distinguish mathematically between the space of states (in-states of scattering theory) and the space of observables (out-``states'' of scattering theory) the new axiom associates states and observables to two different Hardy subspaces which are dense in the same Hilbert space and analytic in the lower and upper complex energy plane, respectively. As a consequence of this new axiom the dynamical equations (Schrödinger or Heisenberg) integrate to a semigroup evolution. Extending this new Hardy space axiom to a relativistic theory provides a relativistic theory of resonance scattering and decay with Born probablilities that fulfill Einstein causality and the exponential decay law.
△ Less
Submitted 14 February, 2003; v1 submitted 22 December, 2002;
originally announced December 2002.
