Showing 1–8 of 8 results for author: Trefzer, M

Reservoir Computing Benchmarks: a review, a taxonomy, some best practices

Authors: Chester Wringe, Martin Trefzer, Susan Stepney

Abstract: Reservoir Computing is an Unconventional Computation model to perform computation on various different substrates, such as RNNs or physical materials. The method takes a "black-box" approach, training only the outputs of the system it is built on. As such, evaluating the computational capacity of these systems can be challenging. We review and critique the evaluation methods used in the field of R… ▽ More Reservoir Computing is an Unconventional Computation model to perform computation on various different substrates, such as RNNs or physical materials. The method takes a "black-box" approach, training only the outputs of the system it is built on. As such, evaluating the computational capacity of these systems can be challenging. We review and critique the evaluation methods used in the field of Reservoir Computing. We introduce a categorisation of benchmark tasks. We review multiple examples of benchmarks from the literature as applied to reservoir computing, and note their strengths and shortcomings. We suggest ways in which benchmarks and their uses may be improved to the benefit of the reservoir computing community △ Less

Submitted 10 May, 2024; originally announced May 2024.

Comments: 36pp, 15figs, review article

Artificial Neural Microcircuits as Building Blocks: Concept and Challenges

Authors: Andrew Walter, Shimeng Wu, Andy M. Tyrrell, Liam McDaid, Malachy McElholm, Nidhin Thandassery Sumithran, Jim Harkin, Martin A. Trefzer

Abstract: Artificial Neural Networks (ANNs) are one of the most widely employed forms of bio-inspired computation. However the current trend is for ANNs to be structurally homogeneous. Furthermore, this structural homogeneity requires the application of complex training and learning tools that produce application specific ANNs, susceptible to pitfalls such as overfitting. In this paper, an new approach is e… ▽ More Artificial Neural Networks (ANNs) are one of the most widely employed forms of bio-inspired computation. However the current trend is for ANNs to be structurally homogeneous. Furthermore, this structural homogeneity requires the application of complex training and learning tools that produce application specific ANNs, susceptible to pitfalls such as overfitting. In this paper, an new approach is explored, inspired by the role played in biology by Neural Microcircuits, the so called ``fundamental processing elements'' of organic nervous systems. How large neural networks, particularly Spiking Neural Networks (SNNs) can be assembled using Artificial Neural Microcircuits (ANMs), intended as off-the-shelf components, is articulated; the results of initial work to produce a catalogue of such Microcircuits though the use of Novelty Search is shown; followed by efforts to expand upon this initial work, including a discussion of challenges uncovered during these efforts and explorations of methods by which they might be overcome. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Comments: 12 pages, 31 figures, 3 tables, submitted to A-Life Journal for review

A perspective on physical reservoir computing with nanomagnetic devices

Authors: Dan A Allwood, Matthew O A Ellis, David Griffin, Thomas J Hayward, Luca Manneschi, Mohammad F KH Musameh, Simon O'Keefe, Susan Stepney, Charles Swindells, Martin A Trefzer, Eleni Vasilaki, Guru Venkat, Ian Vidamour, Chester Wringe

Abstract: Neural networks have revolutionized the area of artificial intelligence and introduced transformative applications to almost every scientific field and industry. However, this success comes at a great price; the energy requirements for training advanced models are unsustainable. One promising way to address this pressing issue is by develo** low-energy neuromorphic hardware that directly support… ▽ More Neural networks have revolutionized the area of artificial intelligence and introduced transformative applications to almost every scientific field and industry. However, this success comes at a great price; the energy requirements for training advanced models are unsustainable. One promising way to address this pressing issue is by develo** low-energy neuromorphic hardware that directly supports the algorithm's requirements. The intrinsic non-volatility, non-linearity, and memory of spintronic devices make them appealing candidates for neuromorphic devices. Here we focus on the reservoir computing paradigm, a recurrent network with a simple training algorithm suitable for computation with spintronic devices since they can provide the properties of non-linearity and memory. We review technologies and methods for develo** neuromorphic spintronic devices and conclude with critical open issues to address before such devices become widely used. △ Less

Submitted 9 December, 2022; originally announced December 2022.

Multi-objective Evolutionary Approach for Efficient Kernel Size and Shape for CNN

Authors: Ziwei Wang, Martin A. Trefzer, Simon J. Bale, Andy M. Tyrrell

Abstract: While state-of-the-art development in CNN topology, such as VGGNet and ResNet, have become increasingly accurate, these networks are computationally expensive involving billions of arithmetic operations and parameters. To improve the classification accuracy, state-of-the-art CNNs usually involve large and complex convolutional layers. However, for certain applications, e.g. Internet of Things (IoT… ▽ More While state-of-the-art development in CNN topology, such as VGGNet and ResNet, have become increasingly accurate, these networks are computationally expensive involving billions of arithmetic operations and parameters. To improve the classification accuracy, state-of-the-art CNNs usually involve large and complex convolutional layers. However, for certain applications, e.g. Internet of Things (IoT), where such CNNs are to be implemented on resource-constrained platforms, the CNN architectures have to be small and efficient. To deal with this problem, reducing the resource consumption in convolutional layers has become one of the most significant solutions. In this work, a multi-objective optimisation approach is proposed to trade-off between the amount of computation and network accuracy by using Multi-Objective Evolutionary Algorithms (MOEAs). The number of convolution kernels and the size of these kernels are proportional to computational resource consumption of CNNs. Therefore, this paper considers optimising the computational resource consumption by reducing the size and number of kernels in convolutional layers. Additionally, the use of unconventional kernel shapes has been investigated and results show these clearly outperform the commonly used square convolution kernels. The main contributions of this paper are therefore a methodology to significantly reduce computational cost of CNNs, based on unconventional kernel shapes, and provide different trade-offs for specific use cases. The experimental results further demonstrate that the proposed method achieves large improvements in resource consumption with no significant reduction in network performance. Compared with the benchmark CNN, the best trade-off architecture shows a reduction in multiplications of up to 6X and with slight increase in classification accuracy on CIFAR-10 dataset. △ Less

Submitted 28 June, 2021; originally announced June 2021.

Comments: 13 pages paper, plus 17 papers supplementary materials

Multi-objective Digital Design Optimisation via Improved Drive Granularity Standard Cells

Authors: Linan Cao, Simon J. Bale, Martin A. Trefzer

Abstract: To tackle the complexity of state-of-the-art electronic systems, silicon foundries continuously shrink the technology nodes and electronic design automation (EDA) vendors offer hierarchical design flows to decompose systems into smaller blocks. However, such a staged design methodology consists of various levels of abstraction, where margins will be accumulated and result in degradation of the ove… ▽ More To tackle the complexity of state-of-the-art electronic systems, silicon foundries continuously shrink the technology nodes and electronic design automation (EDA) vendors offer hierarchical design flows to decompose systems into smaller blocks. However, such a staged design methodology consists of various levels of abstraction, where margins will be accumulated and result in degradation of the overall design quality. This limits the full use of capabilities of both the process technology and EDA tools. In this work, a study of drive granularity of standard cells is performed and an interpolation method is proposed for drive option expansion within original cell libraries. These aim to investigate how industrial synthesis tools deal with the drive strength selection using different granularity sets. In addition, a fully-automated, multi-objective (MO) EDA digital flow is introduced for power, performance, area (PPA) optimisation based on drive strength refinement. This population-based search method better handles the increased difficulty of cell selection when using larger logic libraries, producing better optimised solutions than standard tool flow in this case. The achieved experimental results demonstrate how the improved drive granularity cells overall enhance the quality of designs and how a significant improvement in trading off PPA is achieved by the MOEDA flow. △ Less

Submitted 6 September, 2021; v1 submitted 21 May, 2021; originally announced May 2021.

Comments: This work has been accepted for publication in IEEE TCAS-1

Multi-objective Optimisation of Digital Circuits based on Cell Map** in an Industrial EDA Flow

Authors: Linan Cao, Simon J. Bale, Martin A. Trefzer

Abstract: Modern electronic design automation (EDA) tools can handle the complexity of state-of-the-art electronic systems by decomposing them into smaller blocks or cells, introducing different levels of abstraction and staged design flows. However, throughout each independent-optimised design step, overhead and inefficiency can accumulate in the resulting overall design. Performing design-specific optimis… ▽ More Modern electronic design automation (EDA) tools can handle the complexity of state-of-the-art electronic systems by decomposing them into smaller blocks or cells, introducing different levels of abstraction and staged design flows. However, throughout each independent-optimised design step, overhead and inefficiency can accumulate in the resulting overall design. Performing design-specific optimisation from a more global viewpoint requires more time due to the larger search space, but has the potential to provide solutions with improved performance. In this work, a fully-automated, multi-objective (MO) EDA flow is introduced to address this issue. It specifically tunes drive strength map**, preceding physical implementation, through multi-objective population-based search algorithms. Designs are evaluated with respect to their power, performance and area (PPA). The proposed approach is aimed at digital circuit optimisation at the block-level, where it is capable of expanding the design space and offers a set of trade-off solutions for different case-specific utilisation. We have applied the proposed MOEDA framework to ISCAS-85 and EPFL benchmark circuits using a commercial 65nm standard cell library. The experimental results demonstrate how the MOEDA flow enhances the solutions initially generated by the standard digital flow, and how simultaneously a significant improvement in PPA metrics is achieved. △ Less

Submitted 19 May, 2022; v1 submitted 21 May, 2021; originally announced May 2021.

Comments: This work has been submitted to the IEEE for possible publication

Reservoir Computing with Magnetic Thin Films

Authors: Matthew Dale, David Griffin, Richard F. L. Evans, Sarah Jenkins, Simon O'Keefe, Angelika Sebald, Susan Stepney, Fernando Torre, Martin Trefzer

Abstract: Advances in artificial intelligence are driven by technologies inspired by the brain, but these technologies are orders of magnitude less powerful and energy efficient than biological systems. Inspired by the nonlinear dynamics of neural networks, new unconventional computing hardware has emerged with the potential to exploit natural phenomena and gain efficiency, in a similar manner to biological… ▽ More Advances in artificial intelligence are driven by technologies inspired by the brain, but these technologies are orders of magnitude less powerful and energy efficient than biological systems. Inspired by the nonlinear dynamics of neural networks, new unconventional computing hardware has emerged with the potential to exploit natural phenomena and gain efficiency, in a similar manner to biological systems. Physical reservoir computing demonstrates this with a variety of unconventional systems, from optical-based to memristive systems. Reservoir computers provide a nonlinear projection of the task input into a high-dimensional feature space by exploiting the system's internal dynamics. A trained readout layer then combines features to perform tasks, such as pattern recognition and time-series analysis. Despite progress, achieving state-of-the-art performance without external signal processing to the reservoir remains challenging. Here we perform an initial exploration of three magnetic materials in thin-film geometries via microscale simulation. Our results reveal that basic spin properties of magnetic films generate the required nonlinear dynamics and memory to solve machine learning tasks (although there would be practical challenges in exploiting these particular materials in physical implementations). The method of exploration can be applied to other materials, so this work opens up the possibility of testing different materials, from relatively simple (alloys) to significantly complex (antiferromagnetic reservoirs). △ Less

Submitted 30 October, 2023; v1 submitted 29 January, 2021; originally announced January 2021.

Comments: 30 pages, 10 figures, updated and clarified

Journal ref: International Journal of Unconventional Computing, 19(1):63-92, 2024

A Substrate-Independent Framework to Characterise Reservoir Computers

Authors: Matthew Dale, Julian F. Miller, Susan Stepney, Martin A. Trefzer

Abstract: The Reservoir Computing (RC) framework states that any non-linear, input-driven dynamical system (the reservoir) exhibiting properties such as a fading memory and input separability can be trained to perform computational tasks. This broad inclusion of systems has led to many new physical substrates for RC. Properties essential for reservoirs to compute are tuned through reconfiguration of the sub… ▽ More The Reservoir Computing (RC) framework states that any non-linear, input-driven dynamical system (the reservoir) exhibiting properties such as a fading memory and input separability can be trained to perform computational tasks. This broad inclusion of systems has led to many new physical substrates for RC. Properties essential for reservoirs to compute are tuned through reconfiguration of the substrate, such as change in virtual topology or physical morphology. As a result, each substrate possesses a unique `quality' -- obtained through reconfiguration -- to realise different reservoirs for different tasks. Here we describe an experimental framework to characterise the quality of potentially any substrate for RC. Our framework reveals that a definition of quality is not only useful to compare substrates, but can help map the non-trivial relationship between properties and task performance. In the wider context, the framework offers a greater understanding as to what makes a dynamical system compute, hel** improve the design of future substrates for RC. △ Less

Submitted 20 May, 2019; v1 submitted 16 October, 2018; originally announced October 2018.

Comments: addition of a further substrate: a simulated delay-based reservoir

Journal ref: Proc.Roy.Soc.A 475(2226), 2019