-
Mean Field Approach for Configuring Population Dynamics on a Biohybrid Neuromorphic System
Authors:
Johannes Partzsch,
Christian Mayr,
Massimiliano Giulioni,
Marko Noack,
Stefan Hänzsche,
Stefan Scholze,
Sebastian Höppner,
Paolo Del Giudice,
Rene Schüffny
Abstract:
Real-time coupling of cell cultures to neuromorphic circuits necessitates a neuromorphic network that replicates biological behaviour both on a per-neuron and on a population basis, with a network size comparable to the culture. We present a large neuromorphic system composed of 9 chips, with overall 2880 neurons and 144M conductance-based synapses. As they are realized in a robust switched-capaci…
▽ More
Real-time coupling of cell cultures to neuromorphic circuits necessitates a neuromorphic network that replicates biological behaviour both on a per-neuron and on a population basis, with a network size comparable to the culture. We present a large neuromorphic system composed of 9 chips, with overall 2880 neurons and 144M conductance-based synapses. As they are realized in a robust switched-capacitor fashion, individual neurons and synapses can be configured to replicate with high fidelity a wide range of biologically realistic behaviour. In contrast to other exploration/heuristics-based approaches, we employ a theory-guided mesoscopic approach to configure the overall network to a range of bursting behaviours, thus replicating the statistics of our targeted in-vitro network. The mesoscopic approach has implications beyond our proposed biohybrid, as it allows a targeted exploration of the behavioural space, which is a non-trivial task especially in large, recurrent networks.
△ Less
Submitted 23 April, 2019;
originally announced April 2019.
-
Pattern representation and recognition with accelerated analog neuromorphic systems
Authors:
Mihai A. Petrovici,
Sebastian Schmitt,
Johann Klähn,
David Stöckel,
Anna Schroeder,
Guillaume Bellec,
Johannes Bill,
Oliver Breitwieser,
Ilja Bytschok,
Andreas Grübl,
Maurice Güttler,
Andreas Hartel,
Stephan Hartmann,
Dan Husmann,
Kai Husmann,
Sebastian Jeltsch,
Vitali Karasenko,
Mitja Kleider,
Christoph Koke,
Alexander Kononov,
Christian Mauch,
Eric Müller,
Paul Müller,
Johannes Partzsch,
Thomas Pfeil
, et al. (11 additional authors not shown)
Abstract:
Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks with the aim of emulating them on fast, low-power neuromorphic hardware. Since…
▽ More
Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks with the aim of emulating them on fast, low-power neuromorphic hardware. Since many of these devices employ analog components, which cannot be perfectly controlled, finding ways to compensate for the resulting effects represents a key challenge. Here, we discuss three different strategies to address this problem: the addition of auxiliary network components for stabilizing activity, the utilization of inherently robust architectures and a training method for hardware-emulated networks that functions without perfect knowledge of the system's dynamics and parameters. For all three scenarios, we corroborate our theoretical considerations with experimental results on accelerated analog neuromorphic platforms.
△ Less
Submitted 3 July, 2017; v1 submitted 17 March, 2017;
originally announced March 2017.
-
Switched-Capacitor Realization of Presynaptic Short-Term-Plasticity and Stop-Learning Synapses in 28 nm CMOS
Authors:
Marko Noack,
Johannes Partzsch,
Christian Mayr,
Stefan Hänzsche,
Stefan Scholze,
Sebastian Höppner,
Georg Ellguth,
Rene Schüffny
Abstract:
Synaptic dynamics, such as long- and short-term plasticity, play an important role in the complexity and biological realism achievable when running neural networks on a neuromorphic IC. For example, they endow the IC with an ability to adapt and learn from its environment. In order to achieve the mil- lisecond to second time constants required for these synaptic dynamics, analog subthreshold circu…
▽ More
Synaptic dynamics, such as long- and short-term plasticity, play an important role in the complexity and biological realism achievable when running neural networks on a neuromorphic IC. For example, they endow the IC with an ability to adapt and learn from its environment. In order to achieve the mil- lisecond to second time constants required for these synaptic dynamics, analog subthreshold circuits are usually employed. However, due to process variation and leakage problems, it is almost impossible to port these types of circuits to modern sub-100nm technologies. In contrast, we present a neuromor- phic system in a 28 nm CMOS process that employs switched capacitor (SC) circuits to implement 128 short term plasticity presynapses as well as 8192 stop-learning synapses. The neuromorphic system consumes an area of 0.36 mm2 and runs at a power consumption of 1.9 mW. The circuit makes use of a technique for minimizing leakage effects allowing for real-time operation with time constants up to sev- eral seconds. Since we rely on SC techniques for all calculations, the system is composed of only generic mixed-signal building blocks. These generic building blocks make the system easy to port between technologies and the large digital circuit part inherent in an SC system benefits fully from technology scaling.
△ Less
Submitted 10 December, 2014;
originally announced December 2014.
-
A Biological-Realtime Neuromorphic System in 28 nm CMOS using Low-Leakage Switched Capacitor Circuits
Authors:
Christian Mayr,
Johannes Partzsch,
Marko Noack,
Stefan Hänzsche,
Stefan Scholze,
Sebastian Höppner,
Georg Ellguth,
Rene Schüffny
Abstract:
A switched-capacitor (SC) neuromorphic system for closed-loop neural coupling in 28 nm CMOS is presented, occupying 600 um by 600 um. It offers 128 input channels (i.e. presynaptic terminals), 8192 synapses and 64 output channels (i.e. neurons). Biologically realistic neuron and synapse dynam- ics are achieved via a faithful translation of the behavioural equations to SC circuits. As leakage curre…
▽ More
A switched-capacitor (SC) neuromorphic system for closed-loop neural coupling in 28 nm CMOS is presented, occupying 600 um by 600 um. It offers 128 input channels (i.e. presynaptic terminals), 8192 synapses and 64 output channels (i.e. neurons). Biologically realistic neuron and synapse dynam- ics are achieved via a faithful translation of the behavioural equations to SC circuits. As leakage currents significantly affect circuit behaviour at this technology node, dedicated compensation techniques are employed to achieve biological-realtime operation, with faithful reproduction of time constants of several 100 ms at room temperature. Power draw of the overall system is 1.9 mW.
△ Less
Submitted 10 December, 2014;
originally announced December 2014.
-
OTA based 200 GΩ resistance on 700 μm2 in 180 nm CMOS for neuromorphic applications
Authors:
Christian Mayr,
Michael Schultz,
Marko Noack,
Stephan Henker,
Johannes Partzsch,
Rene Schüffny
Abstract:
Generating an exponential decay function with a time constant on the order of hundreds of milliseconds is a mainstay for neuromorphic circuits. Usually, either subthreshold circuits or RC-decays based on transconductance amplifiers are used. In the latter case, transconductances in the 10 pS range are needed. However, state-of-the-art low-transconductance amplifiers still require too much circuit…
▽ More
Generating an exponential decay function with a time constant on the order of hundreds of milliseconds is a mainstay for neuromorphic circuits. Usually, either subthreshold circuits or RC-decays based on transconductance amplifiers are used. In the latter case, transconductances in the 10 pS range are needed. However, state-of-the-art low-transconductance amplifiers still require too much circuit area to be applicable in neuromorphic circuits where >100 of these time constant circuits may be required on a single chip. We present a silicon verified operational transconductance amplifier that achieves a gm of 5 pS in only 700 μm2, a factor of 10-100 less area than current examples. This allows a high-density integration of time constant circuits in target appliations such as synaptic learning or as driving circuit for neuromorphic memristor arrays.
△ Less
Submitted 30 August, 2014;
originally announced September 2014.
-
Gabor-like Image Filtering using a Neural Microcircuit
Authors:
C. Mayr,
A. Heittmann,
R. Schüffny
Abstract:
In this letter, we present an implementation of a neural microcircuit for image processing employing Hebbian-adaptive learning. The neuronal circuit utilizes only excitatory synapses to correlate action potentials, extracting the uncorrelated ones, which contain significant image information. This circuit is capable of approximating Gabor-like image filtering and other image processing functions
In this letter, we present an implementation of a neural microcircuit for image processing employing Hebbian-adaptive learning. The neuronal circuit utilizes only excitatory synapses to correlate action potentials, extracting the uncorrelated ones, which contain significant image information. This circuit is capable of approximating Gabor-like image filtering and other image processing functions
△ Less
Submitted 8 August, 2014;
originally announced August 2014.
-
Neighborhood Rank Order Coding for Robust Texture Analysis and Feature Extraction
Authors:
C. Mayr,
R. Schüffny
Abstract:
Research into the visual cortex and general neural information processing has led to various attempts to integrate pulse computation schemes in image analysis systems. Of interest is especially the robustness of representing an analogue signal in the phase or duration of a pulsed, quasi-digital signal, as well as the possibility of direct digital interaction, i.e. computation, among these signals.…
▽ More
Research into the visual cortex and general neural information processing has led to various attempts to integrate pulse computation schemes in image analysis systems. Of interest is especially the robustness of representing an analogue signal in the phase or duration of a pulsed, quasi-digital signal, as well as the possibility of direct digital interaction, i.e. computation, among these signals. Such a computation can also achieve information compaction for subsequent processing stages. By using a pulse order encoding scheme motivated by dendritic pulse interaction, we will show that a powerful low-level feature and texture extraction operator, called Pulsed Local Orientation Coding (PLOC), can be implemented. Feature extraction results are being presented, and a possible VLSI implementation is detailed.
△ Less
Submitted 8 August, 2014;
originally announced August 2014.
-
Applying Spiking Neural Nets to Noise Sha**
Authors:
C. Mayr,
R. Schüffny
Abstract:
-In recent years, there has been an increased focus on the mechanics of information transmission in spiking neural networks. Especially the Noise Sha** properties of these networks and their similarity to Delta-Sigma Modulators has received a lot of attention. However, very little of the research done in this area has focused on the effect the weights in these networks have on the Noise Sha**…
▽ More
-In recent years, there has been an increased focus on the mechanics of information transmission in spiking neural networks. Especially the Noise Sha** properties of these networks and their similarity to Delta-Sigma Modulators has received a lot of attention. However, very little of the research done in this area has focused on the effect the weights in these networks have on the Noise Sha** properties and on post- processing of the network output signal. This paper concerns itself with the various modes of network operation and beneficial as well as detrimental effects which the systematic generation of network weights can effect. Also, a method for post-processing of the spiking output signal is introduced, bringing the output signal more in line with conventional Delta-Sigma Modulators. Relevancy of this research to industrial application of neural nets as building blocks of oversampled A/D converters is shown. Also, further points of contention are listed, which must be thoroughly researched to add to the above mentioned applicability of spiking neural nets.
△ Less
Submitted 8 August, 2014;
originally announced August 2014.
-
65 nm CMOS Sensors Applied to Mathematically Exact Colorimetric Reconstruction
Authors:
C. Mayr,
S. Henker,
A. Krause,
J. -U. Schlüßler,
R. Schüffny
Abstract:
Extracting colorimetric image information from the spectral characteristics of image sensors is a key issue in accurate image acquisition. Technically feasible filter/sensor combinations usually do not replicate colorimetric responses with sufficient accuracy to be directly applicable to color representation. A variety of transformations have been proposed in the literature to compensate for this.…
▽ More
Extracting colorimetric image information from the spectral characteristics of image sensors is a key issue in accurate image acquisition. Technically feasible filter/sensor combinations usually do not replicate colorimetric responses with sufficient accuracy to be directly applicable to color representation. A variety of transformations have been proposed in the literature to compensate for this. However, most of those rely on heuristics and/or introduce a reconstruction dependent on the composition of the incoming illumination. In this work, we present a spectral reconstruction method that is independent of illumination and is derived in a mathematically strict way. It provides a deterministic method to arrive at a least mean squared error approximation of a target spectral characteristic from arbitrary sensor response curves. Further, we present a new CMOS sensor design in a standard digital 65nm CMOS technology. Novel circuit techniques are used to achieve performance comparable with much larger-sized specialized photo-CMOS processes. The sensor is utilized as testbed for the spectral reconstruction method.
△ Less
Submitted 8 August, 2014;
originally announced August 2014.
-
Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms
Authors:
Mihai A. Petrovici,
Bernhard Vogginger,
Paul Müller,
Oliver Breitwieser,
Mikael Lundqvist,
Lyle Muller,
Matthias Ehrlich,
Alain Destexhe,
Anders Lansner,
René Schüffny,
Johannes Schemmel,
Karlheinz Meier
Abstract:
Advancing the size and complexity of neural network models leads to an ever increasing demand for computational resources for their simulation. Neuromorphic devices offer a number of advantages over conventional computing architectures, such as high emulation speed or low power consumption, but this usually comes at the price of reduced configurability and precision. In this article, we investigat…
▽ More
Advancing the size and complexity of neural network models leads to an ever increasing demand for computational resources for their simulation. Neuromorphic devices offer a number of advantages over conventional computing architectures, such as high emulation speed or low power consumption, but this usually comes at the price of reduced configurability and precision. In this article, we investigate the consequences of several such factors that are common to neuromorphic devices, more specifically limited hardware resources, limited parameter configurability and parameter variations. Our final aim is to provide an array of methods for co** with such inevitable distortion mechanisms. As a platform for testing our proposed strategies, we use an executable system specification (ESS) of the BrainScaleS neuromorphic system, which has been designed as a universal emulation back-end for neuroscientific modeling. We address the most essential limitations of this device in detail and study their effects on three prototypical benchmark network models within a well-defined, systematic workflow. For each network model, we start by defining quantifiable functionality measures by which we then assess the effects of typical hardware-specific distortion mechanisms, both in idealized software simulations and on the ESS. For those effects that cause unacceptable deviations from the original network dynamics, we suggest generic compensation mechanisms and demonstrate their effectiveness. Both the suggested workflow and the investigated compensation mechanisms are largely back-end independent and do not require additional hardware configurability beyond the one required to emulate the benchmark networks in the first place. We hereby provide a generic methodological environment for configurable neuromorphic devices that are targeted at emulating large-scale, functional neural networks.
△ Less
Submitted 10 February, 2015; v1 submitted 29 April, 2014;
originally announced April 2014.
-
A Comprehensive Workflow for General-Purpose Neural Modeling with Highly Configurable Neuromorphic Hardware Systems
Authors:
Daniel Brüderle,
Mihai A. Petrovici,
Bernhard Vogginger,
Matthias Ehrlich,
Thomas Pfeil,
Sebastian Millner,
Andreas Grübl,
Karsten Wendt,
Eric Müller,
Marc-Olivier Schwartz,
Dan Husmann de Oliveira,
Sebastian Jeltsch,
Johannes Fieres,
Moritz Schilling,
Paul Müller,
Oliver Breitwieser,
Venelin Petkov,
Lyle Muller,
Andrew P. Davison,
Pradeep Krishnamurthy,
Jens Kremkow,
Mikael Lundqvist,
Eilif Muller,
Johannes Partzsch,
Stefan Scholze
, et al. (9 additional authors not shown)
Abstract:
In this paper we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More spe…
▽ More
In this paper we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More specifically, we aim at the establishment of this neuromorphic system as a flexible and neuroscientifically valuable modeling tool that can be used by non-hardware-experts. We consider various functional aspects to be crucial for this purpose, and we introduce a consistent workflow with detailed descriptions of all involved modules that implement the suggested steps: The integration of the hardware interface into the simulator-independent model description language PyNN; a fully automated translation between the PyNN domain and appropriate hardware configurations; an executable specification of the future neuromorphic system that can be seamlessly integrated into this biology-to-hardware map** process as a test bench for all software layers and possible hardware design modifications; an evaluation scheme that deploys models from a dedicated benchmark library, compares the results generated by virtual or prototype hardware devices with reference software simulations and analyzes the differences. The integration of these components into one hardware-software workflow provides an ecosystem for ongoing preparative studies that support the hardware design process and represents the basis for the maturity of the model-to-hardware map** software. The functionality and flexibility of the latter is proven with a variety of experimental results.
△ Less
Submitted 21 July, 2011; v1 submitted 12 November, 2010;
originally announced November 2010.
