-
Temporally Layered Architecture for Efficient Continuous Control
Authors:
Devdhar Patel,
Terrence Sejnowski,
Hava Siegelmann
Abstract:
We present a temporally layered architecture (TLA) for temporally adaptive control with minimal energy expenditure. The TLA layers a fast and a slow policy together to achieve temporal abstraction that allows each layer to focus on a different time scale. Our design draws on the energy-saving mechanism of the human brain, which executes actions at different timescales depending on the environment'…
▽ More
We present a temporally layered architecture (TLA) for temporally adaptive control with minimal energy expenditure. The TLA layers a fast and a slow policy together to achieve temporal abstraction that allows each layer to focus on a different time scale. Our design draws on the energy-saving mechanism of the human brain, which executes actions at different timescales depending on the environment's demands. We demonstrate that beyond energy saving, TLA provides many additional advantages, including persistent exploration, fewer required decisions, reduced jerk, and increased action repetition. We evaluate our method on a suite of continuous control tasks and demonstrate the significant advantages of TLA over existing methods when measured over multiple important metrics. We also introduce a multi-objective score to qualitatively assess continuous control policies and demonstrate a significantly better score for TLA. Our training algorithm uses minimal communication between the slow and fast layers to train both policies simultaneously, making it viable for future applications in distributed control.
△ Less
Submitted 8 August, 2023; v1 submitted 29 May, 2023;
originally announced May 2023.
-
Temporally Layered Architecture for Adaptive, Distributed and Continuous Control
Authors:
Devdhar Patel,
Joshua Russell,
Francesca Walsh,
Tauhidur Rahman,
Terrence Sejnowski,
Hava Siegelmann
Abstract:
We present temporally layered architecture (TLA), a biologically inspired system for temporally adaptive distributed control. TLA layers a fast and a slow controller together to achieve temporal abstraction that allows each layer to focus on a different time-scale. Our design is biologically inspired and draws on the architecture of the human brain which executes actions at different timescales de…
▽ More
We present temporally layered architecture (TLA), a biologically inspired system for temporally adaptive distributed control. TLA layers a fast and a slow controller together to achieve temporal abstraction that allows each layer to focus on a different time-scale. Our design is biologically inspired and draws on the architecture of the human brain which executes actions at different timescales depending on the environment's demands. Such distributed control design is widespread across biological systems because it increases survivability and accuracy in certain and uncertain environments. We demonstrate that TLA can provide many advantages over existing approaches, including persistent exploration, adaptive control, explainable temporal behavior, compute efficiency and distributed control. We present two different algorithms for training TLA: (a) Closed-loop control, where the fast controller is trained over a pre-trained slow controller, allowing better exploration for the fast controller and closed-loop control where the fast controller decides whether to "act-or-not" at each timestep; and (b) Partially open loop control, where the slow controller is trained over a pre-trained fast controller, allowing for open loop-control where the slow controller picks a temporally extended action or defers the next n-actions to the fast controller. We evaluated our method on a suite of continuous control tasks and demonstrate the advantages of TLA over several strong baselines.
△ Less
Submitted 5 February, 2023; v1 submitted 25 December, 2022;
originally announced January 2023.
-
Internal feedback in the cortical perception-action loop enables fast and accurate behavior
Authors:
**g Shuang Li,
Anish A. Sarma,
Terrence J. Sejnowski,
John C. Doyle
Abstract:
Animals move smoothly and reliably in unpredictable environments. Models of sensorimotor control have assumed that sensory information from the environment leads to actions, which then act back on the environment, creating a single, unidirectional perception-action loop. This loop contains internal delays in sensory and motor pathways, which can lead to unstable control. We show here that these de…
▽ More
Animals move smoothly and reliably in unpredictable environments. Models of sensorimotor control have assumed that sensory information from the environment leads to actions, which then act back on the environment, creating a single, unidirectional perception-action loop. This loop contains internal delays in sensory and motor pathways, which can lead to unstable control. We show here that these delays can be compensated by internal feedback signals that flow backwards, from motor towards sensory areas. Internal feedback is ubiquitous in neural sensorimotor systems and recent advances in control theory show how internal feedback compensates internal delays. This is accomplished by filtering out self-generated and other predictable changes in early sensory areas so that unpredicted, actionable information can be rapidly transmitted toward action by the fastest components. For example, fast, giant neurons are necessarily less accurate than smaller neurons, but they are crucial for fast and accurate behavior. We use a mathematically tractable control model to show that internal feedback has an indispensable role in achieving state estimation, localization of function -- how different parts of cortex control different parts of the body -- and attention, all of which are crucial for effective sensorimotor control. This control model can explain anatomical, physiological and behavioral observations, including motor signals in visual cortex, heterogeneous kinetics of sensory receptors and the presence of giant Betz cells in motor cortex, Meynert cells in visual cortex and giant von Economo cells in the prefrontal cortex of humans as well as internal feedback patterns and unexplained heterogeneity in other neural systems.
△ Less
Submitted 10 January, 2023; v1 submitted 10 November, 2022;
originally announced November 2022.
-
Internal Feedback in Biological Control: Architectures and Examples
Authors:
Anish A. Sarma,
**g Shuang Li,
Josefin Stenberg,
Gwyneth Card,
Elizabeth S. Heckscher,
Narayanan Kasthuri,
Terrence Sejnowski,
John C. Doyle
Abstract:
Feedback is ubiquitous in both biological and engineered control systems. In biology, in addition to typical feedback between plant and controller, we observe feedback pathways within control systems, which we call internal feedback pathways (IFPs), that are often very complex. IFPs are most familiar in neural systems, our primary motivation, but they appear everywhere from bacterial signal transd…
▽ More
Feedback is ubiquitous in both biological and engineered control systems. In biology, in addition to typical feedback between plant and controller, we observe feedback pathways within control systems, which we call internal feedback pathways (IFPs), that are often very complex. IFPs are most familiar in neural systems, our primary motivation, but they appear everywhere from bacterial signal transduction to the human immune system. In this paper, we describe these very different motivating examples and introduce the concepts necessary to explain their complex IFPs, particularly the severe speed-accuracy tradeoffs that constrain the hardware in biology. We also sketch some minimal theory for extremely simplified toy models that nevertheless highlight the importance of diversity-enabled sweet spots (DESS) in mitigating the impact of hardware tradeoffs. For more realistic models, standard modern and robust control theory can give some insights into previously cryptic IFPs, and the new System Level Synthesis theory expands this substantially. These additional theories explaining IFPs will be explored in more detail in several companion papers.
△ Less
Submitted 22 October, 2021; v1 submitted 11 October, 2021;
originally announced October 2021.
-
Diversity-enabled sweet spots in layered architectures and speed-accuracy trade-offs in sensorimotor control
Authors:
Yorie Nakahira,
Quanying Liu,
Terrence J. Sejnowski,
John C. Doyle
Abstract:
Nervous systems sense, communicate, compute and actuate movement using distributed components with severe trade-offs in speed, accuracy, sparsity, noise and saturation. Nevertheless, brains achieve remarkably fast, accurate, and robust control performance due to a highly effective layered control architecture. Here we introduce a driving task to study how a mountain biker mitigates the immediate d…
▽ More
Nervous systems sense, communicate, compute and actuate movement using distributed components with severe trade-offs in speed, accuracy, sparsity, noise and saturation. Nevertheless, brains achieve remarkably fast, accurate, and robust control performance due to a highly effective layered control architecture. Here we introduce a driving task to study how a mountain biker mitigates the immediate disturbance of trail bumps and responds to changes in trail direction. We manipulated the time delays and accuracy of the control input from the wheel as a surrogate for manipulating the characteristics of neurons in the control loop. The observed speed-accuracy trade-offs (SATs) motivated a theoretical framework consisting of layers of control loops with components having diverse speeds and accuracies within each physical level, such as nerve bundles containing axons with a wide range of sizes. Our model explains why the errors from two control loops -- one fast but inaccurate reflexive layer that corrects for bumps, and a planning layer that is slow but accurate -- are additive, and show how the errors in each control loop can be decomposed into the errors caused by the limited speeds and accuracies of the components. These results demonstrate that an appropriate diversity in the properties of neurons across layers helps to create "diversity-enabled sweet spots" (DESSs) so that both fast and accurate control is achieved using slow or inaccurate components.
△ Less
Submitted 2 May, 2021; v1 submitted 18 September, 2019;
originally announced September 2019.
-
Fitts' Law for speed-accuracy trade-off describes a diversity-enabled sweet spot in sensorimotor control
Authors:
Yorie Nakahira,
Quanying Liu,
Terrence J. Sejnowski,
John C. Doyle
Abstract:
Human sensorimotor control exhibits remarkable speed and accuracy, and the tradeoff between them is encapsulated in Fitts' Law for reaching and pointing. While Fitts related this to Shannon's channel capacity theorem, despite widespread study of Fitts' Law, a theory that connects implementation of sensorimotor control at the system and hardware level has not emerged. Here we describe a theory that…
▽ More
Human sensorimotor control exhibits remarkable speed and accuracy, and the tradeoff between them is encapsulated in Fitts' Law for reaching and pointing. While Fitts related this to Shannon's channel capacity theorem, despite widespread study of Fitts' Law, a theory that connects implementation of sensorimotor control at the system and hardware level has not emerged. Here we describe a theory that connects hardware (neurons and muscles with inherent severe speed-accuracy tradeoffs) with system level control to explain Fitts' Law for reaching and related laws. The results supporting the theory show that diversity between hardware components is exploited to achieve both fast and accurate control performance despite slow or inaccurate hardware. Such "diversity-enabled sweet spots" (DESSs) are ubiquitous in biology and technology, and explain why large heterogeneities exist in biological and technical components and how both engineers and natural selection routinely evolve fast and accurate systems using imperfect hardware.
△ Less
Submitted 18 September, 2019; v1 submitted 3 June, 2019;
originally announced June 2019.
-
Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computing
Authors:
Siddharth Siddharth,
Tzyy-** Jung,
Terrence J. Sejnowski
Abstract:
In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as object detection. Yet, these advances in deep-learning have not adequately transl…
▽ More
In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as object detection. Yet, these advances in deep-learning have not adequately translated into bio-sensing research. This work applies novel deep-learning-based methods to various bio-sensing and video data of four publicly available multi-modal emotion datasets. For each dataset, we first individually evaluate the emotion-classification performance obtained by each modality. We then evaluate the performance obtained by fusing the features from these modalities. We show that our algorithms outperform the results reported by other studies for emotion/valence/arousal/liking classification on DEAP and MAHNOB-HCI datasets and set up benchmarks for the newer AMIGOS and DREAMER datasets. We also evaluate the performance of our algorithms by combining the datasets and by using transfer learning to show that the proposed method overcomes the inconsistencies between the datasets. Hence, we do a thorough analysis of multi-modal affective data from more than 120 subjects and 2,800 trials. Finally, utilizing a convolution-deconvolution network, we propose a new technique towards identifying salient brain regions corresponding to various affective states.
△ Less
Submitted 16 May, 2019;
originally announced May 2019.
