-
EON-1: A Brain-Inspired Processor for Near-Sensor Extreme Edge Online Feature Extraction
Authors:
Alexandra Dobrita,
Amirreza Yousefzadeh,
Simon Thorpe,
Kanishkan Vadivel,
Paul Detterer,
Guangzhi Tang,
Gert-Jan van Schaik,
Mario Konijnenburg,
Anteneh Gebregiorgis,
Said Hamdioui,
Manolis Sifalakis
Abstract:
For Edge AI applications, deploying online learning and adaptation on resource-constrained embedded devices can deal with fast sensor-generated streams of data in changing environments. However, since maintaining low-latency and power-efficient inference is paramount at the Edge, online learning and adaptation on the device should impose minimal additional overhead for inference. With this goal in…
▽ More
For Edge AI applications, deploying online learning and adaptation on resource-constrained embedded devices can deal with fast sensor-generated streams of data in changing environments. However, since maintaining low-latency and power-efficient inference is paramount at the Edge, online learning and adaptation on the device should impose minimal additional overhead for inference. With this goal in mind, we explore energy-efficient learning and adaptation on-device for streaming-data Edge AI applications using Spiking Neural Networks (SNNs), which follow the principles of brain-inspired computing, such as high-parallelism, neuron co-located memory and compute, and event-driven processing. We propose EON-1, a brain-inspired processor for near-sensor extreme edge online feature extraction, that integrates a fast online learning and adaptation algorithm. We report results of only 1% energy overhead for learning, by far the lowest overhead when compared to other SoTA solutions, while attaining comparable inference accuracy. Furthermore, we demonstrate that EON-1 is up for the challenge of low-latency processing of HD and UHD streaming video in real-time, with learning enabled.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Integrative Imaging Informatics for Cancer Research: Workflow Automation for Neuro-oncology (I3CR-WANO)
Authors:
Satrajit Chakrabarty,
Syed Amaan Abidi,
Mina Mousa,
Mahati Mokkarala,
Isabelle Hren,
Divya Yadav,
Matthew Kelsey,
Pamela LaMontagne,
John Wood,
Michael Adams,
Yuzhuo Su,
Sherry Thorpe,
Caroline Chung,
Aristeidis Sotiras,
Daniel S. Marcus
Abstract:
Efforts to utilize growing volumes of clinical imaging data to generate tumor evaluations continue to require significant manual data wrangling owing to the data heterogeneity. Here, we propose an artificial intelligence-based solution for the aggregation and processing of multisequence neuro-oncology MRI data to extract quantitative tumor measurements. Our end-to-end framework i) classifies MRI s…
▽ More
Efforts to utilize growing volumes of clinical imaging data to generate tumor evaluations continue to require significant manual data wrangling owing to the data heterogeneity. Here, we propose an artificial intelligence-based solution for the aggregation and processing of multisequence neuro-oncology MRI data to extract quantitative tumor measurements. Our end-to-end framework i) classifies MRI sequences using an ensemble classifier, ii) preprocesses the data in a reproducible manner, iii) delineates tumor tissue subtypes using convolutional neural networks, and iv) extracts diverse radiomic features. Moreover, it is robust to missing sequences and adopts an expert-in-the-loop approach, where the segmentation results may be manually refined by radiologists. Following the implementation of the framework in Docker containers, it was applied to two retrospective glioma datasets collected from the Washington University School of Medicine (WUSM; n = 384) and the M.D. Anderson Cancer Center (MDA; n = 30) comprising preoperative MRI scans from patients with pathologically confirmed gliomas. The scan-type classifier yielded an accuracy of over 99%, correctly identifying sequences from 380/384 and 30/30 sessions from the WUSM and MDA datasets, respectively. Segmentation performance was quantified using the Dice Similarity Coefficient between the predicted and expert-refined tumor masks. Mean Dice scores were 0.882 ($\pm$0.244) and 0.977 ($\pm$0.04) for whole tumor segmentation for WUSM and MDA, respectively. This streamlined framework automatically curated, processed, and segmented raw MRI data of patients with varying grades of gliomas, enabling the curation of large-scale neuro-oncology datasets and demonstrating a high potential for integration as an assistive tool in clinical practice.
△ Less
Submitted 6 October, 2022;
originally announced October 2022.
-
2022 Roadmap on Neuromorphic Computing and Engineering
Authors:
Dennis V. Christensen,
Regina Dittmann,
Bernabé Linares-Barranco,
Abu Sebastian,
Manuel Le Gallo,
Andrea Redaelli,
Stefan Slesazeck,
Thomas Mikolajick,
Sabina Spiga,
Stephan Menzel,
Ilia Valov,
Gianluca Milano,
Carlo Ricciardi,
Shi-Jun Liang,
Feng Miao,
Mario Lanza,
Tyler J. Quill,
Scott T. Keene,
Alberto Salleo,
Julie Grollier,
Danijela Marković,
Alice Mizrahi,
Peng Yao,
J. Joshua Yang,
Giacomo Indiveri
, et al. (34 additional authors not shown)
Abstract:
Modern computation based on the von Neumann architecture is today a mature cutting-edge science. In the Von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exas…
▽ More
Modern computation based on the von Neumann architecture is today a mature cutting-edge science. In the Von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 1018 calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices.
The aim of this Roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The Roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges. We hope that this Roadmap will be a useful resource to readers outside this field, for those who are just entering the field, and for those who are well established in the neuromorphic community.
https://doi.org/10.1088/2634-4386/ac4a83
△ Less
Submitted 13 January, 2022; v1 submitted 12 May, 2021;
originally announced May 2021.
-
Micromirror total internal reflection microscopy for high-performance single particle tracking at interfaces
Authors:
Xuanhui Meng,
Adar Sonn-Segev,
Anne Schumacher,
Daniel Cole,
Gavin Young,
Stephen Thorpe,
Robert W. Style,
Eric R. Dufresne,
Philipp Kukura
Abstract:
Single particle tracking has found broad applications in the life and physical sciences, enabling the observation and characterisation of nano- and microscopic motion. Fluorescence-based approaches are ideally suited for high-background environments, such as tracking lipids or proteins in or on cells, due to superior background rejection. Scattering-based detection is preferable when localisation…
▽ More
Single particle tracking has found broad applications in the life and physical sciences, enabling the observation and characterisation of nano- and microscopic motion. Fluorescence-based approaches are ideally suited for high-background environments, such as tracking lipids or proteins in or on cells, due to superior background rejection. Scattering-based detection is preferable when localisation precision and imaging speed are paramount due to the in principle infinite photon budget. Here, we show that micromirror-based total internal reflection dark field microscopy enables background suppression previously only reported for interferometric scattering microscopy, resulting in nm localisation precision at 6 $μ$s exposure time for 20 nm gold nanoparticles with a 25 x 25 $μ$m$^{2}$ field of view. We demonstrate the capabilities of our implementation by characterizing sub-nm deterministic flows of 20 nm gold nanoparticles at liquid-liquid interfaces. Our results approach the optimal combination of background suppression, localisation precision and temporal resolution achievable with pure scattering-based imaging and tracking of nanoparticles at regular interfaces.
△ Less
Submitted 17 March, 2021;
originally announced March 2021.
-
Attachment and antibiotic response of early-stage biofilms studied using resonant hyperspectral imaging
Authors:
Yue Wang,
Christopher P. Reardon,
Nicholas Read,
Stephen Thorpe,
Adrian Evans,
Neil Todd,
Marjan Van Der Woude,
Thomas F. Krauss
Abstract:
Many bacterial species readily develop biofilms that act as a protective matrix against external challenge, e.g. from antimicrobial treatment. Therefore, biofilms are often responsible for persistent and recurring infections. Established methods for studying biofilms are either destructive or they focus on the biofilm surface. A non-destructive method that is sensitive to the underside of the biof…
▽ More
Many bacterial species readily develop biofilms that act as a protective matrix against external challenge, e.g. from antimicrobial treatment. Therefore, biofilms are often responsible for persistent and recurring infections. Established methods for studying biofilms are either destructive or they focus on the biofilm surface. A non-destructive method that is sensitive to the underside of the biofilm is highly desirable, as it allows studying the penetration of antibiotics through the film. Here, we demonstrate that the high surface sensitivity of resonant hyperspectral imaging provides this capability. The method allows us to monitor the early stages of Escherichia coli biofilm formation, cell attachment and microcolony formation, in-situ and in real time. We study the response of the biofilm to a number of different antibiotics and verify our observations using confocal microscopy. Based on this ability to closely monitor the surface-bound cells, resonant hyperspectral imaging gives new insights into the antimicrobial resistance of biofilms.
△ Less
Submitted 7 September, 2020;
originally announced September 2020.
-
Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
Authors:
Milad Mozafari,
Mohammad Ganjtabesh,
Abbas Nowzari-Dalini,
Simon J. Thorpe,
Timothée Masquelier
Abstract:
The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological counterparts, and they are typically trained with backpropagation, which is extremely data-hungry. To address these limitations, we used a deep convolutional spiking neural network (D…
▽ More
The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological counterparts, and they are typically trained with backpropagation, which is extremely data-hungry. To address these limitations, we used a deep convolutional spiking neural network (DCSNN) and a latency-coding scheme. We trained it using a combination of spike-timing-dependent plasticity (STDP) for the lower layers and reward-modulated STDP (R-STDP) for the higher ones. In short, with R-STDP a correct (resp. incorrect) decision leads to STDP (resp. anti-STDP). This approach led to an accuracy of $97.2\%$ on MNIST, without requiring an external classifier. In addition, we demonstrated that R-STDP extracts features that are diagnostic for the task at hand, and discards the other ones, whereas STDP extracts any feature that repeats. Finally, our approach is biologically plausible, hardware friendly, and energy-efficient.
△ Less
Submitted 27 May, 2019; v1 submitted 31 March, 2018;
originally announced April 2018.
-
STDP-based spiking deep convolutional neural networks for object recognition
Authors:
Saeed Reza Kheradpisheh,
Mohammad Ganjtabesh,
Simon J Thorpe,
Timothée Masquelier
Abstract:
Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-p…
▽ More
Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-propagation - that having many layers increases the recognition robustness, an approach known as deep learning. We thus designed a deep SNN, comprising several convolutional (trainable with STDP) and pooling layers. We used a temporal coding scheme where the most strongly activated neurons fire first, and less activated neurons fire later or not at all. The network was exposed to natural images. Thanks to STDP, neurons progressively learned features corresponding to prototypical patterns that were both salient and frequent. Only a few tens of examples per category were required and no label was needed. After learning, the complexity of the extracted features increased along the hierarchy, from edge detectors in the first layer to object prototypes in the last layer. Coding was very sparse, with only a few thousands spikes per image, and in some cases the object category could be reasonably well inferred from the activity of a single higher-order neuron. More generally, the activity of a few hundreds of such neurons contained robust category information, as demonstrated using a classifier on Caltech 101, ETH-80, and MNIST databases. We also demonstrate the superiority of STDP over other unsupervised techniques such as random crops (HMAX) or auto-encoders. Taken together, our results suggest that the combination of STDP with latency coding may be a key to understanding the way that the primate visual system learns, its remarkable processing speed and its low energy consumption.
△ Less
Submitted 25 December, 2017; v1 submitted 4 November, 2016;
originally announced November 2016.
-
Proceedings of the Ninth Workshop on Web 2.0 Security and Privacy (W2SP) 2015
Authors:
Abigail Goldsteen,
Tyrone Grandison,
Mike Just,
Larry Koved,
Rohan Malcolm,
Sean Thorpe
Abstract:
This is the Proceedings of the Ninth Workshop on Web 2.0 Security and Privacy (W2SP) 2015, held in San Jose, CA, USA, on May 21, 2015. The workshop was held as part of the IEEE Computer Society Security and Privacy Workshops, in conjunction with the IEEE Symposium on Security and Privacy.
This is the Proceedings of the Ninth Workshop on Web 2.0 Security and Privacy (W2SP) 2015, held in San Jose, CA, USA, on May 21, 2015. The workshop was held as part of the IEEE Computer Society Security and Privacy Workshops, in conjunction with the IEEE Symposium on Security and Privacy.
△ Less
Submitted 22 June, 2015;
originally announced June 2015.
-
The Maximal Graph Dirichlet Problem in Semi-Euclidean Spaces
Authors:
Benjamin Stuart Thorpe
Abstract:
The maximal graph Dirichlet problem asks whether there exists a spacelike graph, in a semi-Euclidean space, with a given boundary and with mean curvature everywhere zero. We prove the existence of solutions to this problem under certain assumptions on the given boundary. Most importantly, the results proved here will hold for graphs of codimension greater than 1.
The maximal graph Dirichlet problem asks whether there exists a spacelike graph, in a semi-Euclidean space, with a given boundary and with mean curvature everywhere zero. We prove the existence of solutions to this problem under certain assumptions on the given boundary. Most importantly, the results proved here will hold for graphs of codimension greater than 1.
△ Less
Submitted 18 December, 2011;
originally announced December 2011.
-
Coding static natural images using spiking event times: do neurons cooperate?
Authors:
Laurent Perrinet,
Manuel Samuelides,
Simon Thorpe
Abstract:
To understand possible strategies of temporal spike coding in the central nervous system, we study functional neuromimetic models of visual processing for static images. We will first present the retinal model which was introduced by Van Rullen and Thorpe and which represents the multiscale contrast values of the image using an orthonormal wavelet transform. These analog values activate a set of…
▽ More
To understand possible strategies of temporal spike coding in the central nervous system, we study functional neuromimetic models of visual processing for static images. We will first present the retinal model which was introduced by Van Rullen and Thorpe and which represents the multiscale contrast values of the image using an orthonormal wavelet transform. These analog values activate a set of spiking neurons which each fire once to produce an asynchronous wave of spikes. According to this model, the image may be progressively reconstructed from this spike wave thanks to regularities in the statistics of the coefficients determined with natural images. Here, we study mathematically how the quality of information transmission carried by this temporal representation varies over time. In particular, we study how these regularities can be used to optimize information transmission by using a form of temporal cooperation of neurons to code analog values. The original model used wavelet transforms that are close to orthogonal. However, the selectivity of realistic neurons overlap, and we propose an extension of the previous model by adding a spatial cooperation between filters. This model extends the previous scheme for arbitrary--and possibly nonorthogonal--representations of features in the images. In particular, we compared the performance of increasingly over-complete representations in the retina. Results show that this algorithm provides an efficient spike coding strategy for low-level visual processing which may adapt to the complexity of the visual input.
△ Less
Submitted 1 November, 2006;
originally announced November 2006.
