-
Learning over time using a neuromorphic adaptive control algorithm for robotic arms
Authors:
Lazar Supic,
Terrence C. Stewart
Abstract:
In this paper, we explore the ability of a robot arm to learn the underlying operation space defined by the positions (x, y, z) that the arm's end-effector can reach, including disturbances, by deploying and thoroughly evaluating a Spiking Neural Network SNN-based adaptive control algorithm. While traditional control algorithms for robotics have limitations in both adapting to new and dynamic envi…
▽ More
In this paper, we explore the ability of a robot arm to learn the underlying operation space defined by the positions (x, y, z) that the arm's end-effector can reach, including disturbances, by deploying and thoroughly evaluating a Spiking Neural Network SNN-based adaptive control algorithm. While traditional control algorithms for robotics have limitations in both adapting to new and dynamic environments, we show that the robot arm can learn the operational space and complete tasks faster over time. We also demonstrate that the adaptive robot control algorithm based on SNNs enables a fast response while maintaining energy efficiency. We obtained these results by performing an extensive search of the adaptive algorithm parameter space, and evaluating algorithm performance for different SNN network sizes, learning rates, dynamic robot arm trajectories, and response times. We show that the robot arm learns to complete tasks 15% faster in specific experiment scenarios such as scenarios with six or nine random target points.
△ Less
Submitted 3 October, 2022;
originally announced October 2022.
-
Bioinspired Smooth Neuromorphic Control for Robotic Arms
Authors:
Ioannis Polykretis,
Lazar Supic,
Andreea Danielescu
Abstract:
Beyond providing accurate movements, achieving smooth motion trajectories is a long-standing goal of robotics control theory for arms aiming to replicate natural human movements. Drawing inspiration from biological agents, whose reaching control networks effortlessly give rise to smooth and precise movements, can simplify these control objectives for robot arms. Neuromorphic processors, which mimi…
▽ More
Beyond providing accurate movements, achieving smooth motion trajectories is a long-standing goal of robotics control theory for arms aiming to replicate natural human movements. Drawing inspiration from biological agents, whose reaching control networks effortlessly give rise to smooth and precise movements, can simplify these control objectives for robot arms. Neuromorphic processors, which mimic the brain's computational principles, are an ideal platform to approximate the accuracy and smoothness of biological controllers while maximizing their energy efficiency and robustness. However, the incompatibility of conventional control methods with neuromorphic hardware limits the computational efficiency and explainability of their existing adaptations. In contrast, the neuronal subnetworks underlying smooth and accurate reaching movements are effective, minimal, and inherently compatible with neuromorphic hardware. In this work, we emulate these networks with a biologically realistic spiking neural network for motor control on neuromorphic hardware. The proposed controller incorporates experimentally-identified short-term synaptic plasticity and specialized neurons that regulate sensory feedback gain to provide smooth and accurate joint control across a wide motion range. Concurrently, it preserves the minimal complexity of its biological counterpart and is directly deployable on Intel's neuromorphic processor. Using the joint controller as a building block and inspired by joint coordination in human arms, we scaled up this approach to control real-world robot arms. The trajectories and smooth, bell-shaped velocity profiles of the resulting motions resembled those of humans, verifying the biological relevance of the controller. Notably, the method achieved state-of-the-art control performance while decreasing the motion jerk by 19% to improve motion smoothness.
△ Less
Submitted 8 March, 2023; v1 submitted 6 September, 2022;
originally announced September 2022.
-
Visual Odometry with Neuromorphic Resonator Networks
Authors:
Alpha Renner,
Lazar Supic,
Andreea Danielescu,
Giacomo Indiveri,
E. Paxon Frady,
Friedrich T. Sommer,
Yulia Sandamirskaya
Abstract:
Visual Odometry (VO) is a method to estimate self-motion of a mobile robot using visual sensors. Unlike odometry based on integrating differential measurements that can accumulate errors, such as inertial sensors or wheel encoders, visual odometry is not compromised by drift. However, image-based VO is computationally demanding, limiting its application in use cases with low-latency, -memory, and…
▽ More
Visual Odometry (VO) is a method to estimate self-motion of a mobile robot using visual sensors. Unlike odometry based on integrating differential measurements that can accumulate errors, such as inertial sensors or wheel encoders, visual odometry is not compromised by drift. However, image-based VO is computationally demanding, limiting its application in use cases with low-latency, -memory, and -energy requirements. Neuromorphic hardware offers low-power solutions to many vision and AI problems, but designing such solutions is complicated and often has to be assembled from scratch. Here we propose to use Vector Symbolic Architecture (VSA) as an abstraction layer to design algorithms compatible with neuromorphic hardware. Building from a VSA model for scene analysis, described in our companion paper, we present a modular neuromorphic algorithm that achieves state-of-the-art performance on two-dimensional VO tasks. Specifically, the proposed algorithm stores and updates a working memory of the presented visual environment. Based on this working memory, a resonator network estimates the changing location and orientation of the camera. We experimentally validate the neuromorphic VSA-based approach to VO with two benchmarks: one based on an event camera dataset and the other in a dynamic scene with a robotic task.
△ Less
Submitted 26 June, 2024; v1 submitted 5 September, 2022;
originally announced September 2022.
-
Neuromorphic Visual Scene Understanding with Resonator Networks
Authors:
Alpha Renner,
Lazar Supic,
Andreea Danielescu,
Giacomo Indiveri,
Bruno A. Olshausen,
Yulia Sandamirskaya,
Friedrich T. Sommer,
E. Paxon Frady
Abstract:
Analyzing a visual scene by inferring the configuration of a generative model is widely considered the most flexible and generalizable approach to scene understanding. Yet, one major problem is the computational challenge of the inference procedure, involving a combinatorial search across object identities and poses. Here we propose a neuromorphic solution exploiting three key concepts: (1) a comp…
▽ More
Analyzing a visual scene by inferring the configuration of a generative model is widely considered the most flexible and generalizable approach to scene understanding. Yet, one major problem is the computational challenge of the inference procedure, involving a combinatorial search across object identities and poses. Here we propose a neuromorphic solution exploiting three key concepts: (1) a computational framework based on Vector Symbolic Architectures (VSA) with complex-valued vectors; (2) the design of Hierarchical Resonator Networks (HRN) to factorize the non-commutative transforms translation and rotation in visual scenes; (3) the design of a multi-compartment spiking phasor neuron model for implementing complex-valued resonator networks on neuromorphic hardware. The VSA framework uses vector binding operations to form a generative image model in which binding acts as the equivariant operation for geometric transformations. A scene can, therefore, be described as a sum of vector products, which can then be efficiently factorized by a resonator network to infer objects and their poses. The HRN features a partitioned architecture in which vector binding is equivariant for horizontal and vertical translation within one partition and for rotation and scaling within the other partition. The spiking neuron model allows map** the resonator network onto efficient and low-power neuromorphic hardware. Our approach is demonstrated on synthetic scenes composed of simple 2D shapes undergoing rigid geometric transformations and color changes. A companion paper demonstrates the same approach in real-world application scenarios for machine vision and robotics.
△ Less
Submitted 26 June, 2024; v1 submitted 26 August, 2022;
originally announced August 2022.
-
Tuning Algorithms and Generators for Efficient Edge Inference
Authors:
Rawan Naous,
Lazar Supic,
Yoonhwan Kang,
Ranko Sredojevic,
Anish Singhani,
Vladimir Stojanovic
Abstract:
A surge in artificial intelligence and autonomous technologies have increased the demand toward enhanced edge-processing capabilities. Computational complexity and size of state-of-the-art Deep Neural Networks (DNNs) are rising exponentially with diverse network models and larger datasets. This growth limits the performance scaling and energy-efficiency of both distributed and embedded inference p…
▽ More
A surge in artificial intelligence and autonomous technologies have increased the demand toward enhanced edge-processing capabilities. Computational complexity and size of state-of-the-art Deep Neural Networks (DNNs) are rising exponentially with diverse network models and larger datasets. This growth limits the performance scaling and energy-efficiency of both distributed and embedded inference platforms. Embedded designs at the edge are constrained by energy and speed limitations of available processor substrates and processor to memory communication required to fetch the model coefficients. While many hardware accelerator and network deployment frameworks have been in development, a framework is needed to allow the variety of existing architectures, and those in development, to be expressed in critical parts of the flow that perform various optimization steps. Moreover, premature architecture-blind network selection and optimization diminish the effectiveness of schedule optimizations and hardware-specific map**s. In this paper, we address these issues by creating a cross-layer software-hardware design framework that encompasses network training and model compression that is aware of and tuned to the underlying hardware architecture. This approach leverages the available degrees of DNN structure and sparsity to create a converged network that can be partitioned and efficiently scheduled on the target hardware platform, minimizing data movement, and improving the overall throughput and energy. To further streamline the design, we leverage the high-level, flexible SoC generator platform based on RISC-V ROCC framework. This integration allows seamless extensions of the RISC-V instruction set and Chisel-based rapid generator design. Utilizing this approach, we implemented a silicon prototype in a 16 nm TSMC process node achieving record processing efficiency of up to 18 TOPS/W.
△ Less
Submitted 10 May, 2020; v1 submitted 30 July, 2019;
originally announced August 2019.
-
MPDCompress - Matrix Permutation Decomposition Algorithm for Deep Neural Network Compression
Authors:
Lazar Supic,
Rawan Naous,
Ranko Sredojevic,
Aleksandra Faust,
Vladimir Stojanovic
Abstract:
Deep neural networks (DNNs) have become the state-of-the-art technique for machine learning tasks in various applications. However, due to their size and the computational complexity, large DNNs are not readily deployable on edge devices in real-time. To manage complexity and accelerate computation, network compression techniques based on pruning and quantization have been proposed and shown to be…
▽ More
Deep neural networks (DNNs) have become the state-of-the-art technique for machine learning tasks in various applications. However, due to their size and the computational complexity, large DNNs are not readily deployable on edge devices in real-time. To manage complexity and accelerate computation, network compression techniques based on pruning and quantization have been proposed and shown to be effective in reducing network size. However, such network compression can result in irregular matrix structures that are mismatched with modern hardware-accelerated platforms, such as graphics processing units (GPUs) designed to perform the DNN matrix multiplications in a structured (block-based) way. We propose MPDCompress, a DNN compression algorithm based on matrix permutation decomposition via random mask generation. In-training application of the masks molds the synaptic weight connection matrix to a sub-graph separation format. Aided by the random permutations, a hardware-desirable block matrix is generated, allowing for a more efficient implementation and compression of the network. To show versatility, we empirically verify MPDCompress on several network models, compression rates, and image datasets. On the LeNet 300-100 model (MNIST dataset), Deep MNIST, and CIFAR10, we achieve 10 X network compression with less than 1% accuracy loss compared to non-compressed accuracy performance. On AlexNet for the full ImageNet ILSVRC-2012 dataset, we achieve 8 X network compression with less than 1% accuracy loss, with top-5 and top-1 accuracies of 79.6% and 56.4%, respectively. Finally, we observe that the algorithm can offer inference speedups across various hardware platforms, with 4 X faster operation achieved on several mobile GPUs.
△ Less
Submitted 30 May, 2018;
originally announced May 2018.
-
Structured Deep Neural Network Pruning via Matrix Pivoting
Authors:
Ranko Sredojevic,
Shaoyi Cheng,
Lazar Supic,
Rawan Naous,
Vladimir Stojanovic
Abstract:
Deep Neural Networks (DNNs) are the key to the state-of-the-art machine vision, sensor fusion and audio/video signal processing. Unfortunately, their computation complexity and tight resource constraints on the Edge make them hard to leverage on mobile, embedded and IoT devices. Due to great diversity of Edge devices, DNN designers have to take into account the hardware platform and application re…
▽ More
Deep Neural Networks (DNNs) are the key to the state-of-the-art machine vision, sensor fusion and audio/video signal processing. Unfortunately, their computation complexity and tight resource constraints on the Edge make them hard to leverage on mobile, embedded and IoT devices. Due to great diversity of Edge devices, DNN designers have to take into account the hardware platform and application requirements during network training. In this work we introduce pruning via matrix pivoting as a way to improve network pruning by compromising between the design flexibility of architecture-oblivious and performance efficiency of architecture-aware pruning, the two dominant techniques for obtaining resource-efficient DNNs. We also describe local and global network optimization techniques for efficient implementation of the resulting pruned networks. In combination, the proposed pruning and implementation result in close to linear speed up with the reduction of network coefficients during pruning.
△ Less
Submitted 30 November, 2017;
originally announced December 2017.
