Showing 1–50 of 58 results for author: Kilpatrick, P

Robustly encoding certainty in a metastable neural circuit model

Authors: Heather L Cihak, Zachary P Kilpatrick

Abstract: Localized persistent neural activity has been shown to serve delayed estimation of continuous variables. Common experiments require that subjects store and report the feature value (e.g., orientation) of a particular cue (e.g., oriented bar on a screen) after a delay. Visualizing recorded activity of neurons according to their feature tuning reveals activity bumps whose centers wander stochastical… ▽ More Localized persistent neural activity has been shown to serve delayed estimation of continuous variables. Common experiments require that subjects store and report the feature value (e.g., orientation) of a particular cue (e.g., oriented bar on a screen) after a delay. Visualizing recorded activity of neurons according to their feature tuning reveals activity bumps whose centers wander stochastically, degrading the estimate over time. Bump position therefore represents the remembered estimate. Recent work suggests that bump amplitude may represent estimate certainty reflecting a probabilistic population code for a Bayesian posterior. Idealized models of this type are fragile due to the fine tuning common to constructed continuum attractors in dynamical systems. Here we propose an alternative metastable model for robustly supporting multiple bump amplitudes by extending neural circuit models to include quantized nonlinearities. Asymptotic projections of circuit activity produce low-dimensional evolution equations for the amplitude and position of bump solutions in response to external stimuli and noise perturbations. Analysis of reduced equations accurately characterizes phase variance and the dynamics of amplitude transitions between stable discrete values. More salient cues generate bumps of higher amplitude which wander less, consistent with the experimental finding that greater certainty correlates with more accurate memories. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: 14 pages, 10 figures

Fast decisions reflect biases, slow decisions do not

Authors: Samantha Linn, Sean D. Lawley, Bhargav R. Karamched, Zachary P. Kilpatrick, Krešimir Josić

Abstract: Decisions are often made by heterogeneous groups of individuals, each with distinct initial biases and access to information of different quality. We show that in large groups of independent agents who accumulate evidence the first to decide are those with the strongest initial biases. Their decisions align with their initial bias, regardless of the underlying truth. In contrast, agents who decide… ▽ More Decisions are often made by heterogeneous groups of individuals, each with distinct initial biases and access to information of different quality. We show that in large groups of independent agents who accumulate evidence the first to decide are those with the strongest initial biases. Their decisions align with their initial bias, regardless of the underlying truth. In contrast, agents who decide last make decisions as if they were initially unbiased, and hence make better choices. We obtain asymptotic expressions in the large population limit that quantify how agents' initial inclinations shape early decisions. Our analysis shows how bias, information quality, and decision order interact in non-trivial ways to determine the reliability of decisions in a group. △ Less

Submitted 2 January, 2024; v1 submitted 30 December, 2023; originally announced January 2024.

Comments: 12 pages, 4 figures

Representing stimulus motion with waves in adaptive neural fields

Authors: Sage Shaw, Zachary P Kilpatrick

Abstract: Traveling waves of neural activity emerge in cortical networks both spontaneously and in response to stimuli. The spatiotemporal structure of waves can indicate the information they encode and the physiological processes that sustain them. Here, we investigate the stimulus-response relationships of traveling waves emerging in adaptive neural fields as a model of visual motion processing. Neural fi… ▽ More Traveling waves of neural activity emerge in cortical networks both spontaneously and in response to stimuli. The spatiotemporal structure of waves can indicate the information they encode and the physiological processes that sustain them. Here, we investigate the stimulus-response relationships of traveling waves emerging in adaptive neural fields as a model of visual motion processing. Neural field equations model the activity of cortical tissue as a continuum excitable medium, and adaptive processes provide negative feedback, generating localized activity patterns. Synaptic connectivity in our model is described by an integral kernel that weakens dynamically due to activity-dependent synaptic depression, leading to marginally stable traveling fronts (with attenuated backs) or pulses of a fixed speed. Our analysis quantifies how weak stimuli shift the relative position of these waves over time, characterized by a wave response function we obtain perturbatively. Persistent and continuously visible stimuli model moving visual objects. Intermittent flashes that hop across visual space can produce the experience of smooth apparent visual motion. Entrainment of waves to both kinds of moving stimuli are well characterized by our theory and numerical simulations, providing a mechanistic description of the perception of visual motion. △ Less

Submitted 10 December, 2023; originally announced December 2023.

Comments: 31 pages, 6 figures

DNNShifter: An Efficient DNN Pruning System for Edge Computing

Authors: Bailey J. Eccles, Philip Rodgers, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Deep neural networks (DNNs) underpin many machine learning applications. Production quality DNN models achieve high inference accuracy by training millions of DNN parameters which has a significant resource footprint. This presents a challenge for resources operating at the extreme edge of the network, such as mobile and embedded devices that have limited computational and memory resources. To add… ▽ More Deep neural networks (DNNs) underpin many machine learning applications. Production quality DNN models achieve high inference accuracy by training millions of DNN parameters which has a significant resource footprint. This presents a challenge for resources operating at the extreme edge of the network, such as mobile and embedded devices that have limited computational and memory resources. To address this, models are pruned to create lightweight, more suitable variants for these devices. Existing pruning methods are unable to provide similar quality models compared to their unpruned counterparts without significant time costs and overheads or are limited to offline use cases. Our work rapidly derives suitable model variants while maintaining the accuracy of the original model. The model variants can be swapped quickly when system and network conditions change to match workload demand. This paper presents DNNShifter, an end-to-end DNN training, spatial pruning, and model switching system that addresses the challenges mentioned above. At the heart of DNNShifter is a novel methodology that prunes sparse models using structured pruning. The pruned model variants generated by DNNShifter are smaller in size and thus faster than dense and sparse model predecessors, making them suitable for inference at the edge while retaining near similar accuracy as of the original dense model. DNNShifter generates a portfolio of model variants that can be swiftly interchanged depending on operational conditions. DNNShifter produces pruned model variants up to 93x faster than conventional training methods. Compared to sparse models, the pruned model variants are up to 5.14x smaller and have a 1.67x inference latency speedup, with no compromise to sparse model accuracy. In addition, DNNShifter has up to 11.9x lower overhead for switching models and up to 3.8x lower memory utilisation than existing approaches. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 14 pages, 7 figures, 5 tables

MSC Class: 68T07 ACM Class: I.2.1

MS-BioGraphs: Sequence Similarity Graph Datasets

Authors: Mohsen Koohi Esfahani, Paolo Boldi, Hans Vandierendonck, Peter Kilpatrick, Sebastiano Vigna

Abstract: Progress in High-Performance Computing in general, and High-Performance Graph Processing in particular, is highly dependent on the availability of publicly-accessible, relevant, and realistic data sets. To ensure continuation of this progress, we (i) investigate and optimize the process of generating large sequence similarity graphs as an HPC challenge and (ii) demonstrate this process in creati… ▽ More Progress in High-Performance Computing in general, and High-Performance Graph Processing in particular, is highly dependent on the availability of publicly-accessible, relevant, and realistic data sets. To ensure continuation of this progress, we (i) investigate and optimize the process of generating large sequence similarity graphs as an HPC challenge and (ii) demonstrate this process in creating MS-BioGraphs, a new family of publicly available real-world edge-weighted graph datasets with up to $2.5$ trillion edges, that is, $6.6$ times greater than the largest graph published recently. The largest graph is created by matching (i.e., all-to-all similarity aligning) $1.7$ billion protein sequences. The MS-BioGraphs family includes also seven subgraphs with different sizes and direction types. We describe two main challenges we faced in generating large graph datasets and our solutions, that are, (i) optimizing data structures and algorithms for this multi-step process and (ii) WebGraph parallel compression technique. We present a comparative study of structural characteristics of MS-BioGraphs. The datasets are available online on https://blogs.qub.ac.uk/DIPSA/MS-BioGraphs . △ Less

Submitted 31 August, 2023; originally announced August 2023.

Multiscale motion and deformation of bumps in stochastic neural fields with dynamic connectivity

Authors: Heather L Cihak, Zachary P Kilpatrick

Abstract: The distinct timescales of synaptic plasticity and neural activity dynamics play an important role in the brain's learning and memory systems. Activity-dependent plasticity reshapes neural circuit architecture, determining spontaneous and stimulus-encoding spatiotemporal patterns of neural activity. Neural activity bumps maintain short term memories of continuous parameter values, emerging in spat… ▽ More The distinct timescales of synaptic plasticity and neural activity dynamics play an important role in the brain's learning and memory systems. Activity-dependent plasticity reshapes neural circuit architecture, determining spontaneous and stimulus-encoding spatiotemporal patterns of neural activity. Neural activity bumps maintain short term memories of continuous parameter values, emerging in spatially-organized models with short term excitation and long-range inhibition. Previously, we demonstrated nonlinear Langevin equations derived using an interface method accurately describe the dynamics of bumps in continuum neural fields with separate excitatory/inhibitory populations. Here we extend this analysis to incorporate effects of slow short term plasticity that modifies connectivity described by an integral kernel. Linear stability analysis adapted to these piecewise smooth models with Heaviside firing rates further indicate how plasticity shapes bumps' local dynamics. Facilitation (depression), which strengthens (weakens) synaptic connectivity originating from active neurons, tends to increase (decrease) stability of bumps when acting on excitatory synapses. The relationship is inverted when plasticity acts on inhibitory synapses. Multiscale approximations of the stochastic dynamics of bumps perturbed by weak noise reveal the plasticity variables evolve to slowly diffusing and blurred versions of that arising in the stationary solution. Nonlinear Langevin equations associated with bump positions or interfaces coupled to slowly evolving projections of plasticity variables accurately describe the wandering of bumps underpinned by these smoothed synaptic efficacy profiles. △ Less

Submitted 28 June, 2023; originally announced June 2023.

Comments: 19 pages, 11 figures

EcoFed: Efficient Communication for DNN Partitioning-based Federated Learning

Authors: Di Wu, Rehmat Ullah, Philip Rodgers, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Efficiently running federated learning (FL) on resource-constrained devices is challenging since they are required to train computationally intensive deep neural networks (DNN) independently. DNN partitioning-based FL (DPFL) has been proposed as one mechanism to accelerate training where the layers of a DNN (or computation) are offloaded from the device to the server. However, this creates signifi… ▽ More Efficiently running federated learning (FL) on resource-constrained devices is challenging since they are required to train computationally intensive deep neural networks (DNN) independently. DNN partitioning-based FL (DPFL) has been proposed as one mechanism to accelerate training where the layers of a DNN (or computation) are offloaded from the device to the server. However, this creates significant communication overheads since the intermediate activation and gradient need to be transferred between the device and the server during training. While current research reduces the communication introduced by DNN partitioning using local loss-based methods, we demonstrate that these methods are ineffective in improving the overall efficiency (communication overhead and training speed) of a DPFL system. This is because they suffer from accuracy degradation and ignore the communication costs incurred when transferring the activation from the device to the server. This article proposes EcoFed - a communication efficient framework for DPFL systems. EcoFed eliminates the transmission of the gradient by develo** pre-trained initialization of the DNN model on the device for the first time. This reduces the accuracy degradation seen in local loss-based methods. In addition, EcoFed proposes a novel replay buffer mechanism and implements a quantization-based compression technique to reduce the transmission of the activation. It is experimentally demonstrated that EcoFed can reduce the communication cost by up to 133x and accelerate training by up to 21x when compared to classic FL. Compared to vanilla DPFL, EcoFed achieves a 16x communication reduction and 2.86x training time speed-up. EcoFed is available from https://github.com/blessonvar/EcoFed. △ Less

Submitted 3 January, 2024; v1 submitted 11 April, 2023; originally announced April 2023.

Correlated Information Reduces Accuracy of Pioneering Decision-Makers

Authors: Megan Stickler, William Ott, Zachary P. Kilpatrick, Krešimir Josić, Bhargav R. Karamched

Abstract: Normative models are often used to describe how humans and animals make decisions. These models treat deliberation as the accumulation of uncertain evidence that terminates with a commitment to a choice. When extended to social groups, such models often assume that individuals make independent observations. However, individuals typically gather evidence from common sources, and their observations… ▽ More Normative models are often used to describe how humans and animals make decisions. These models treat deliberation as the accumulation of uncertain evidence that terminates with a commitment to a choice. When extended to social groups, such models often assume that individuals make independent observations. However, individuals typically gather evidence from common sources, and their observations are rarely independent. Here we ask: For a group of ideal observers who do not exchange information, what is the impact of correlated evidence on decision accuracy? We show that even when agents are identical, correlated evidence causes decision accuracy to depend on temporal decision order. Surprisingly, the first decider is less accurate than a lone observer. Early deciders are less accurate than late deciders. These phenomena occur despite the fact that the rational observers use the same decision criterion, so they are equally confident in their decisions. We analyze discrete and continuum evidence-gathering models to explain why the first decider is less accurate than a lone observer when evidence is correlated. Pooling the decisions of early deciders using a majority rule does not rescue accuracy in the sense that such pooling results in only modest accuracy gain. Although we analyze an idealized model, we believe that our analysis offers insights that do not depend on exactly how groups integrate evidence and form decisions. △ Less

Submitted 29 March, 2023; originally announced April 2023.

Comments: 19 pages, 5 figures

MSC Class: 92F05

PiPar: Pipeline Parallelism for Collaborative Machine Learning

Authors: Zihan Zhang, Philip Rodgers, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Collaborative machine learning (CML) techniques, such as federated learning, have been proposed to train deep learning models across multiple mobile devices and a server. CML techniques are privacy-preserving as a local model that is trained on each device instead of the raw data from the device is shared with the server. However, CML training is inefficient due to low resource utilization. We ide… ▽ More Collaborative machine learning (CML) techniques, such as federated learning, have been proposed to train deep learning models across multiple mobile devices and a server. CML techniques are privacy-preserving as a local model that is trained on each device instead of the raw data from the device is shared with the server. However, CML training is inefficient due to low resource utilization. We identify idling resources on the server and devices due to sequential computation and communication as the principal cause of low resource utilization. A novel framework PiPar that leverages pipeline parallelism for CML techniques is developed to substantially improve resource utilization. A new training pipeline is designed to parallelize the computations on different hardware resources and communication on different bandwidth resources, thereby accelerating the training process in CML. A low overhead automated parameter selection method is proposed to optimize the pipeline, maximizing the utilization of available resources. The experimental results confirm the validity of the underlying approach of PiPar and highlight that when compared to federated learning: (i) the idle time of the server can be reduced by up to 64.1x, and (ii) the overall training time can be accelerated by up to 34.6x under varying network conditions for a collection of six small and large popular deep neural networks and four datasets without sacrificing accuracy. It is also experimentally demonstrated that PiPar achieves performance benefits when incorporating differential privacy methods and operating in environments with heterogeneous devices and changing bandwidths. △ Less

Submitted 25 June, 2024; v1 submitted 1 December, 2022; originally announced February 2023.

CONTINUER: Maintaining Distributed DNN Services During Edge Failures

Authors: Ayesha Abdul Majeed, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Partitioning and deploying Deep Neural Networks (DNNs) across edge nodes may be used to meet performance objectives of applications. However, the failure of a single node may result in cascading failures that will adversely impact the delivery of the service and will result in failure to meet specific objectives. The impact of these failures needs to be minimised at runtime. Three techniques are e… ▽ More Partitioning and deploying Deep Neural Networks (DNNs) across edge nodes may be used to meet performance objectives of applications. However, the failure of a single node may result in cascading failures that will adversely impact the delivery of the service and will result in failure to meet specific objectives. The impact of these failures needs to be minimised at runtime. Three techniques are explored in this paper, namely repartitioning, early-exit and skip-connection. When an edge node fails, the repartitioning technique will repartition and redeploy the DNN thus avoiding the failed nodes. The early-exit technique makes provision for a request to exit (early) before the failed node. The skip connection technique dynamically routes the request by skip** the failed nodes. This paper will leverage trade-offs in accuracy, end-to-end latency and downtime for selecting the best technique given user-defined objectives (accuracy, latency and downtime thresholds) when an edge node fails. To this end, CONTINUER is developed. Two key activities of the framework are estimating the accuracy and latency when using the techniques for distributed DNNs and selecting the best technique. It is demonstrated on a lab-based experimental testbed that CONTINUER estimates accuracy and latency when using the techniques with no more than an average error of 0.28% and 13.06%, respectively and selects the suitable technique with a low overhead of no more than 16.82 milliseconds and an accuracy of up to 99.86%. △ Less

Submitted 25 April, 2022; originally announced June 2022.

Comments: 10 pages

Adaptive Bayesian Inference of Markov Transition Rates

Authors: Nicholas W. Barendregt, Emily G. Webb, Zachary P. Kilpatrick

Abstract: Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past observations, adaptive approaches adjust sampling constraints online as model parameter estimates are refined, continually maximizing expected information gained… ▽ More Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past observations, adaptive approaches adjust sampling constraints online as model parameter estimates are refined, continually maximizing expected information gained or variance reduced. We apply adaptive Bayesian inference to estimate transition rates of Markov chains, a common class of models for stochastic processes in nature. Unlike most previous studies, our sequential Bayesian optimal design is updated with each observation, and can be simply extended beyond two-state models to birth-death processes and multistate models. By iteratively finding the best time to obtain each sample, our adaptive algorithm maximally reduces variance, resulting in lower overall error in ground truth parameter estimates across a wide range of Markov chain parameterizations and conformations. △ Less

Submitted 1 February, 2023; v1 submitted 20 May, 2022; originally announced May 2022.

Comments: 21 pages, 6 figures

Distinct excitatory and inhibitory bump wandering in a stochastic neural field

Authors: Heather L Cihak, Tahra L Eissa, Zachary P Kilpatrick

Abstract: Localized persistent cortical neural activity is a validated neural substrate of parametric working memory. Such activity `bumps' represent the continuous location of a cue over several seconds. Pyramidal (excitatory) and interneuronal (inhibitory) subpopulations exhibit tuned bumps of activity, linking neural dynamics to behavioral inaccuracies observed in memory recall. However, many bump attrac… ▽ More Localized persistent cortical neural activity is a validated neural substrate of parametric working memory. Such activity `bumps' represent the continuous location of a cue over several seconds. Pyramidal (excitatory) and interneuronal (inhibitory) subpopulations exhibit tuned bumps of activity, linking neural dynamics to behavioral inaccuracies observed in memory recall. However, many bump attractor models collapse these subpopulations into a single joint excitatory/inhibitory (lateral inhibitory) population, and do not consider the role of interpopulation neural architecture and noise correlations. Both factors have a high potential to im**e upon the stochastic dynamics of these bumps, ultimately sha** behavioral response variance. In our study, we consider a neural field model with separate excitatory/inhibitory (E/I) populations and leverage asymptotic analysis to derive a nonlinear Langevin system describing E/I bump interactions. While the E bump attracts the I bump, the I bump stabilizes but can also repel the E bump, which can result in prolonged relaxation dynamics when both bumps are perturbed. Furthermore, the structure of noise correlations within and between subpopulations strongly shapes the variance in bump position. Surprisingly, higher interpopulation correlations reduce variance. △ Less

Submitted 4 March, 2022; originally announced March 2022.

Comments: 28 pages; 10 figures

Stochastic dynamics of social patch foraging decisions

Authors: Subekshya Bidari, Ahmed El Hady, Jacob Davidson, Zachary P Kilpatrick

Abstract: Animals typically forage in groups. Social foraging can help animals avoid predation and decrease their uncertainty about the richness of food resources. Despite this, theoretical mechanistic models of patch foraging have overwhelmingly focused on the behavior of single foragers. In this study, we develop a mechanistic model describing the behavior of individuals foraging together and departing fo… ▽ More Animals typically forage in groups. Social foraging can help animals avoid predation and decrease their uncertainty about the richness of food resources. Despite this, theoretical mechanistic models of patch foraging have overwhelmingly focused on the behavior of single foragers. In this study, we develop a mechanistic model describing the behavior of individuals foraging together and departing food patches following an evidence accumulation process. Each individual's belief about patch quality is represented by a stochastically accumulating variable coupled to others' belief, representing the transfer of information. We consider a cohesive group, and model information sharing as either intermittent pulsatile coupling (communicate decision to leave) or continuous diffusive coupling (communicate online belief). Foraging efficiency under pulsatile coupling has a stronger dependence on the coupling strength parameter compared to diffusive. Despite employing minimal information transfer, pulsatile coupling can still provide similar or higher foraging efficiency compared to diffusive coupling. Conversely, diffusive coupling is more robust to parameter detuning and performs better when individuals have heterogeneous departure criteria and social information weighting. Efficiency is measured by a reward rate function that balances the amount of energy accumulated against the time spent in a patch, computed by solving an ordered first passage time problem for the patch departures of each individual. Using synthetic data we show that we can distinguish between the two modes of communication and identify the model parameters. Our model establishes a social patch foraging framework to parse and identify deliberative decision strategies, to distinguish different forms of social communication, and to allow model fitting to real world animal behavior data. △ Less

Submitted 11 February, 2022; originally announced February 2022.

Comments: 24 pages, 7 figures

FedFly: Towards Migration in Edge-based Distributed Federated Learning

Authors: Rehmat Ullah, Di Wu, Paul Harvey, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Federated learning (FL) is a privacy-preserving distributed machine learning technique that trains models while kee** all the original data generated on devices locally. Since devices may be resource constrained, offloading can be used to improve FL performance by transferring computational workload from devices to edge servers. However, due to mobility, devices participating in FL may leave the… ▽ More Federated learning (FL) is a privacy-preserving distributed machine learning technique that trains models while kee** all the original data generated on devices locally. Since devices may be resource constrained, offloading can be used to improve FL performance by transferring computational workload from devices to edge servers. However, due to mobility, devices participating in FL may leave the network during training and need to connect to a different edge server. This is challenging because the offloaded computations from edge server need to be migrated. In line with this assertion, we present FedFly, which is, to the best of our knowledge, the first work to migrate a deep neural network (DNN) when devices move between edge servers during FL training. Our empirical results on the CIFAR10 dataset, with both balanced and imbalanced data distribution, support our claims that FedFly can reduce training time by up to 33% when a device moves after 50% of the training is completed, and by up to 45% when 90% of the training is completed when compared to state-of-the-art offloading approach in FL. FedFly has negligible overhead of up to two seconds and does not compromise accuracy. Finally, we highlight a number of open research issues for further investigation. FedFly can be downloaded from https://github.com/qub-blesson/FedFly. △ Less

Submitted 14 July, 2022; v1 submitted 2 November, 2021; originally announced November 2021.

Comments: 7 pages, 6 figures

FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning

Authors: Di Wu, Rehmat Ullah, Paul Harvey, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Applying Federated Learning (FL) on Internet-of-Things devices is necessitated by the large volumes of data they produce and growing concerns of data privacy. However, there are three challenges that need to be addressed to make FL efficient: (i) execution on devices with limited computational capabilities, (ii) accounting for stragglers due to computational heterogeneity of devices, and (iii) ada… ▽ More Applying Federated Learning (FL) on Internet-of-Things devices is necessitated by the large volumes of data they produce and growing concerns of data privacy. However, there are three challenges that need to be addressed to make FL efficient: (i) execution on devices with limited computational capabilities, (ii) accounting for stragglers due to computational heterogeneity of devices, and (iii) adaptation to the changing network bandwidths. This paper presents FedAdapt, an adaptive offloading FL framework to mitigate the aforementioned challenges. FedAdapt accelerates local training in computationally constrained devices by leveraging layer offloading of deep neural networks (DNNs) to servers. Further, FedAdapt adopts reinforcement learning based optimization and clustering to adaptively identify which layers of the DNN should be offloaded for each individual device on to a server to tackle the challenges of computational heterogeneity and changing network bandwidth. Experimental studies are carried out on a lab-based testbed and it is demonstrated that by offloading a DNN from the device to the server FedAdapt reduces the training time of a typical IoT device by over half compared to classic FL. The training time of extreme stragglers and the overall training time can be reduced by up to 57%. Furthermore, with changing network bandwidth, FedAdapt is demonstrated to reduce the training time by up to 40% when compared to classic FL, without sacrificing accuracy. △ Less

Submitted 18 May, 2022; v1 submitted 9 July, 2021; originally announced July 2021.

Comments: 13 pages

NEUKONFIG: Reducing Edge Service Downtime When Repartitioning DNNs

Authors: Ayesha Abdul Majeed, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Deep Neural Networks (DNNs) may be partitioned across the edge and the cloud to improve the performance efficiency of inference. DNN partitions are determined based on operational conditions such as network speed. When operational conditions change DNNs will need to be repartitioned to maintain the overall performance. However, repartitioning using existing approaches, such as Pause and Resume, wi… ▽ More Deep Neural Networks (DNNs) may be partitioned across the edge and the cloud to improve the performance efficiency of inference. DNN partitions are determined based on operational conditions such as network speed. When operational conditions change DNNs will need to be repartitioned to maintain the overall performance. However, repartitioning using existing approaches, such as Pause and Resume, will incur a service downtime on the edge. This paper presents the NEUKONFIG framework that identifies the service downtime incurred when repartitioning DNNs and proposes approaches for reducing edge service downtime. The proposed approaches are based on 'Dynamic Switching' in which, when the network speed changes and given an existing edge-cloud pipeline, a new edge-cloud pipeline is initialised with new DNN partitions. Incoming inference requests are switched to the new pipeline for processing data. Two dynamic switching scenarios are considered: when a second edge-cloud pipeline is always running and when a second pipeline is only initialised when the network speed changes. Experimental studies are carried out on a lab-based testbed to demonstrate that Dynamic Switching reduces the downtime by at least an order of magnitude when compared to a baseline using Pause and Resume that has a downtime of 6 seconds. A trade-off in the edge service downtime and memory required is noted. The Dynamic Switching approach that requires the same amount of memory as the baseline reduces the edge service downtime to 0.6 seconds and to less than 1 millisecond in the best case when twice the amount of memory as the baseline is available. △ Less

Submitted 29 June, 2021; originally announced June 2021.

Comments: 10 pages

Hive geometry shapes the recruitment rate of honeybee colonies

Authors: Subekshya Bidari, Zachary P Kilpatrick

Abstract: Honey bees make decisions regarding foraging and nest-site selection in groups ranging from hundreds to thousands of individuals. To effectively make these decisions bees need to communicate within a spatially distributed group. However, the spatiotemporal dynamics of honey bee communication have been mostly overlooked in models of collective decisions, focusing primarily on mean field models of o… ▽ More Honey bees make decisions regarding foraging and nest-site selection in groups ranging from hundreds to thousands of individuals. To effectively make these decisions bees need to communicate within a spatially distributed group. However, the spatiotemporal dynamics of honey bee communication have been mostly overlooked in models of collective decisions, focusing primarily on mean field models of opinion dynamics. We analyze how the spatial properties of the nest or hive, and the movement of individuals with different belief states (uncommitted or committed) therein affect the rate of information transmission using spatially-extended models of collective decision-making within a hive. Honeybees waggle-dance to recruit conspecifics with an intensity that is a threshold nonlinear function of the waggler concentration. Our models range from treating the hive as a chain of discrete patches to a continuous line (long narrow hive). The combination of population-thresholded recruitment and compartmentalized populations generates tradeoffs between rapid information propagation with strong population dispersal and recruitment failures resulting from excessive population diffusion and also creates an effective colony-level signal-detection mechanism whereby recruitment to low quality objectives is blocked. △ Less

Submitted 30 November, 2020; originally announced December 2020.

Comments: 32 pages; 10 figures

A Case For Adaptive Deep Neural Networks in Edge Computing

Authors: Francis McNamee, Schahram Dustadar, Peter Kilpatrick, Weisong Shi, Ivor Spence, Blesson Varghese

Abstract: Edge computing offers an additional layer of compute infrastructure closer to the data source before raw data from privacy-sensitive and performance-critical applications is transferred to a cloud data center. Deep Neural Networks (DNNs) are one class of applications that are reported to benefit from collaboratively computing between the edge and the cloud. A DNN is partitioned such that specific… ▽ More Edge computing offers an additional layer of compute infrastructure closer to the data source before raw data from privacy-sensitive and performance-critical applications is transferred to a cloud data center. Deep Neural Networks (DNNs) are one class of applications that are reported to benefit from collaboratively computing between the edge and the cloud. A DNN is partitioned such that specific layers of the DNN are deployed onto the edge and the cloud to meet performance and privacy objectives. However, there is limited understanding of: (a) whether and how evolving operational conditions (increased CPU and memory utilization at the edge or reduced data transfer rates between the edge and the cloud) affect the performance of already deployed DNNs, and (b) whether a new partition configuration is required to maximize performance. A DNN that adapts to changing operational conditions is referred to as an 'adaptive DNN'. This paper investigates whether there is a case for adaptive DNNs in edge computing by considering three questions: (i) Are DNNs sensitive to operational conditions? (ii) How sensitive are DNNs to operational conditions? (iii) Do individual or a combination of operational conditions equally affect DNNs? (iv) Is DNN partitioning sensitive to hardware architectures on the cloud/edge? The exploration is carried out in the context of 8 pre-trained DNN models and the results presented are from analyzing nearly 8 million data points. The results highlight that network conditions affects DNN performance more than CPU or memory related operational conditions. Repartitioning is noted to provide a performance gain in a number of cases, but a specific trend was not noted in relation to its correlation to the underlying hardware architecture. Nonetheless, the need for adaptive DNNs is confirmed. △ Less

Submitted 16 December, 2020; v1 submitted 4 August, 2020; originally announced August 2020.

Normative theory of patch foraging decisions

Authors: Zachary P Kilpatrick, Jacob D Davidson, Ahmed El Hady

Abstract: Foraging is a fundamental behavior as animals' search for food is crucial for their survival. Patch leaving is a canonical foraging behavior, but classic theoretical conceptions of patch leaving decisions lack some key naturalistic details. Optimal foraging theory provides general rules for when an animal should leave a patch, but does not provide mechanistic insights about how those rules change… ▽ More Foraging is a fundamental behavior as animals' search for food is crucial for their survival. Patch leaving is a canonical foraging behavior, but classic theoretical conceptions of patch leaving decisions lack some key naturalistic details. Optimal foraging theory provides general rules for when an animal should leave a patch, but does not provide mechanistic insights about how those rules change with the structure of the environment. Such a mechanistic framework would aid in designing quantitative experiments to unravel behavioral and neural underpinnings of foraging. To address these shortcomings, we develop a normative theory of patch foraging decisions. Using a Bayesian approach, we treat patch leaving behavior as a statistical inference problem. We derive the animals' optimal decision strategies in both non-depleting and depleting environments. A majority of these cases can be analyzed explicitly using methods from stochastic processes. Our behavioral predictions are expressed in terms of the optimal patch residence time and the decision rule by which an animal departs a patch. We also extend our theory to a hierarchical model in which the forager learns the environmental food resource distribution. The quantitative framework we develop will therefore help experimenters move from analyzing trial based behavior to continuous behavior without the loss of quantitative rigor. Our theoretical framework both extends optimal foraging theory and motivates a variety of behavioral and neuroscientific experiments investigating patch foraging behavior. △ Less

Submitted 22 April, 2020; originally announced April 2020.

Comments: 28 pages, 10 figures, 1 table

Cross Architectural Power Modelling

Authors: Kai Chen, Peter Kilpatrick, Dimitrios S. Nikolopoulos, Blesson Varghese

Abstract: Existing power modelling research focuses on the model rather than the process for develo** models. An automated power modelling process that can be deployed on different processors for develo** power models with high accuracy is developed. For this, (i) an automated hardware performance counter selection method that selects counters best correlated to power on both ARM and Intel processors, (… ▽ More Existing power modelling research focuses on the model rather than the process for develo** models. An automated power modelling process that can be deployed on different processors for develo** power models with high accuracy is developed. For this, (i) an automated hardware performance counter selection method that selects counters best correlated to power on both ARM and Intel processors, (ii) a noise filter based on clustering that can reduce the mean error in power models, and (iii) a two stage power model that surmounts challenges in using existing power models across multiple architectures are proposed and developed. The key results are: (i) the automated hardware performance counter selection method achieves comparable selection to the manual method reported in the literature, (ii) the noise filter reduces the mean error in power models by up to 55%, and (iii) the two stage power model can predict dynamic power with less than 8% error on both ARM and Intel processors, which is an improvement over classic models. △ Less

Submitted 17 March, 2020; originally announced March 2020.

Comments: 10 pages; IEEE/ACM CCGrid 2020. arXiv admin note: text overlap with arXiv:1710.10325

Modelling Fog Offloading Performance

Authors: Ayesha Abdul Majeed, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Fog computing has emerged as a computing paradigm aimed at addressing the issues of latency, bandwidth and privacy when mobile devices are communicating with remote cloud services. The concept is to offload compute services closer to the data. However many challenges exist in the realisation of this approach. During offloading, (part of) the application underpinned by the services may be unavailab… ▽ More Fog computing has emerged as a computing paradigm aimed at addressing the issues of latency, bandwidth and privacy when mobile devices are communicating with remote cloud services. The concept is to offload compute services closer to the data. However many challenges exist in the realisation of this approach. During offloading, (part of) the application underpinned by the services may be unavailable, which the user will experience as down time. This paper describes work aimed at building models to allow prediction of such down time based on metrics (operational data) of the underlying and surrounding infrastructure. Such prediction would be invaluable in the context of automated Fog offloading and adaptive decision making in Fog orchestration. Models that cater for four container-based stateless and stateful offload techniques, namely Save and Load, Export and Import, Push and Pull and Live Migration, are built using four (linear and non-linear) regression techniques. Experimental results comprising over 42 million data points from multiple lab-based Fog infrastructure are presented. The results highlight that reasonably accurate predictions (measured by the coefficient of determination for regression models, mean absolute percentage error, and mean absolute error) may be obtained when considering 25 metrics relevant to the infrastructure. △ Less

Submitted 12 February, 2020; originally announced February 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:1909.04945

Performance Estimation of Container-Based Cloud-to-Fog Offloading

Authors: Ayesha Abdul Majeed, Peter Kilpatrick, Ivor Spence, Blesson Varghese

Abstract: Fog computing offloads latency critical application services running on the Cloud in close proximity to end-user devices onto resources located at the edge of the network. The research in this paper is motivated towards characterising and estimating the time taken to offload a service using containers, which is investigated in the context of the `Save and Load' container migration technique. To th… ▽ More Fog computing offloads latency critical application services running on the Cloud in close proximity to end-user devices onto resources located at the edge of the network. The research in this paper is motivated towards characterising and estimating the time taken to offload a service using containers, which is investigated in the context of the `Save and Load' container migration technique. To this end, the research addresses questions such as whether fog offloading can be accurately modelled and which system and network related parameters influence offloading. These are addressed by exploring a catalogue of 21 different metrics both at the system and process levels that is used as input to four estimation techniques using collective model and individual models to predict the time taken for offloading. The study is pursued by collecting over 1.1 million data points and the preliminary results indicate that offloading can be modelled accurately. △ Less

Submitted 11 September, 2019; originally announced September 2019.

Social inhibition maintains adaptivity and consensus of foraging honeybee swarms in dynamic environments

Authors: Subekshya Bidari, Orit Peleg, Zachary P Kilpatrick

Abstract: To effectively forage in natural environments, organisms must adapt to changes in the quality and yield of food sources across multiple timescales. Individuals foraging in groups act based on both their private observations and the opinions of their neighbors. How do these information sources interact in changing environments? We address this problem in the context of honeybee swarms, showing inhi… ▽ More To effectively forage in natural environments, organisms must adapt to changes in the quality and yield of food sources across multiple timescales. Individuals foraging in groups act based on both their private observations and the opinions of their neighbors. How do these information sources interact in changing environments? We address this problem in the context of honeybee swarms, showing inhibitory social interactions help maintain adaptivity and consensus needed for effective foraging. Individual and social interactions of a mathematical swarm model shape the nutrition yield of a group foraging from feeders with temporally switching food quality. Social interactions improve foraging from a single feeder if temporal switching is fast or feeder quality is low. When the swarm chooses from multiple feeders, the most effective form of social interaction is direct switching, whereby bees flip the opinion of nestmates foraging at lower yielding feeders. Model linearization shows that effective social interactions increase the fraction of the swarm at the correct feeder (consensus) and the rate at which bees reach that feeder (adaptivity). Our mathematical framework allows us to compare a suite of social inhibition mechanisms, suggesting experimental protocols for revealing effective swarm foraging strategies in dynamic environments. △ Less

Submitted 5 July, 2019; originally announced July 2019.

Comments: 27 pages, 13 figures

Analyzing dynamic decision-making models using Chapman-Kolmogorov equations

Authors: Nicholas W. Barendregt, Krešimir Josić, Zachary P. Kilpatrick

Abstract: Decision-making in dynamic environments typically requires adaptive evidence accumulation that weights new evidence more heavily than old observations. Recent experimental studies of dynamic decision tasks require subjects to make decisions for which the correct choice switches stochastically throughout a single trial. In such cases, an ideal observer's belief is described by an evolution equation… ▽ More Decision-making in dynamic environments typically requires adaptive evidence accumulation that weights new evidence more heavily than old observations. Recent experimental studies of dynamic decision tasks require subjects to make decisions for which the correct choice switches stochastically throughout a single trial. In such cases, an ideal observer's belief is described by an evolution equation that is doubly stochastic, reflecting stochasticity in the both observations and environmental changes. In these contexts, we show that the probability density of the belief can be represented using differential Chapman-Kolmogorov equations, allowing efficient computation of ensemble statistics. This allows us to reliably compare normative models to near-normative approximations using, as model performance metrics, decision response accuracy and Kullback-Leibler divergence of the belief distributions. Such belief distributions could be obtained empirically from subjects by asking them to report their decision confidence. We also study how response accuracy is affected by additional internal noise, showing optimality requires longer integration timescales as more noise is added. Lastly, we demonstrate that our method can be applied to tasks in which evidence arrives in a discrete, pulsatile fashion, rather than continuously. △ Less

Submitted 25 March, 2019; originally announced March 2019.

Comments: 24 pages, 9 figures

Performance of normative and approximate evidence accumulation on the dynamic clicks task

Authors: Adrian E. Radillo, Alan Veliz-Cuba, Krešimir Josić, Zachary P. Kilpatrick

Abstract: The aim of a number of psychophysics tasks is to uncover how mammals make decisions in a world that is in flux. Here we examine the characteristics of ideal and near-ideal observers in a task of this type. We ask when and how performance depends on task parameters and design, and, in turn, what observer performance tells us about their decision-making process. In the dynamic clicks task subjects h… ▽ More The aim of a number of psychophysics tasks is to uncover how mammals make decisions in a world that is in flux. Here we examine the characteristics of ideal and near-ideal observers in a task of this type. We ask when and how performance depends on task parameters and design, and, in turn, what observer performance tells us about their decision-making process. In the dynamic clicks task subjects hear two streams (left and right) of Poisson clicks with different rates. Subjects are rewarded when they correctly identify the side with the higher rate, as this side switches unpredictably. We show that a reduced set of task parameters defines regions in parameter space in which optimal, but not near-optimal observers, maintain constant response accuracy. We also show that for a range of task parameters an approximate normative model must be finely tuned to reach near-optimal performance, illustrating a potential way to distinguish between normative models and their approximations. In addition, we show that using the negative log-likelihood and the 0/1-loss functions to fit these types of models is not equivalent: the 0/1-loss leads to a bias in parameter recovery that increases with sensory noise. These findings suggest ways to tease apart models that are hard to distinguish when tuned exactly, and point to general pitfalls in experimental design, model fitting, and interpretation of the resulting data. △ Less

Submitted 8 October, 2019; v1 submitted 4 February, 2019; originally announced February 2019.

Comments: 34 pages, 6 figures, includes reviews and responses

Journal ref: Neurons, Behavior, Data analysis, and Theory (2019) 4

Optimal models of decision-making in dynamic environments

Authors: Zachary P. Kilpatrick, William R. Holmes, Tahra L. Eissa, Krešimir Josić

Abstract: Nature is in constant flux, so animals must account for changes in their environment when making decisions. How animals learn the timescale of such changes and adapt their decision strategies accordingly is not well understood. Recent psychophysical experiments have shown humans and other animals can achieve near-optimal performance at two alternative forced choice (2AFC) tasks in dynamically chan… ▽ More Nature is in constant flux, so animals must account for changes in their environment when making decisions. How animals learn the timescale of such changes and adapt their decision strategies accordingly is not well understood. Recent psychophysical experiments have shown humans and other animals can achieve near-optimal performance at two alternative forced choice (2AFC) tasks in dynamically changing environments. Characterization of performance requires the derivation and analysis of computational models of optimal decision-making policies on such tasks. We review recent theoretical work in this area, and discuss how models compare with subjects' behavior in tasks where the correct choice or evidence quality changes in dynamic, but predictable, ways. △ Less

Submitted 21 December, 2018; v1 submitted 4 December, 2018; originally announced December 2018.

RADS: Real-time Anomaly Detection System for Cloud Data Centres

Authors: Sakil Barbhuiya, Zafeirios Papazachos, Peter Kilpatrick, Dimitrios S. Nikolopoulos

Abstract: Cybersecurity attacks in Cloud data centres are increasing alongside the growth of the Cloud services market. Existing research proposes a number of anomaly detection systems for detecting such attacks. However, these systems encounter a number of challenges, specifically due to the unknown behaviour of the attacks and the occurrence of genuine Cloud workload spikes, which must be distinguished fr… ▽ More Cybersecurity attacks in Cloud data centres are increasing alongside the growth of the Cloud services market. Existing research proposes a number of anomaly detection systems for detecting such attacks. However, these systems encounter a number of challenges, specifically due to the unknown behaviour of the attacks and the occurrence of genuine Cloud workload spikes, which must be distinguished from attacks. In this paper, we discuss these challenges and investigate the issues with the existing Cloud anomaly detection approaches. Then, we propose a Real-time Anomaly Detection System (RADS) for Cloud data centres, which uses a one class classification algorithm and a window-based time series analysis to address the challenges. Specifically, RADS can detect VM-level anomalies occurring due to DDoS and cryptomining attacks. We evaluate the performance of RADS by running lab-based experiments and by using real-world Cloud workload traces. Evaluation results demonstrate that RADS can achieve 90-95% accuracy with a low false positive rate of 0-3%. The results further reveal that RADS experiences fewer false positives when using its window-based time series analysis in comparison to using state-of-the-art average or entropy based analysis. △ Less

Submitted 11 November, 2018; originally announced November 2018.

Comments: 14 pages

Optimizing a jump-diffusion model of a starving forager

Authors: Nikhil Krishnan, Zachary P. Kilpatrick

Abstract: We analyze the movement of a starving forager on a one-dimensional periodic lattice, where each location contains one unit of food. As the forager lands on sites with food, it consumes the food, leaving the sites empty. If the forager lands consecutively on $s$ empty sites, then it will starve. The forager has two modes of movement: it can either diffuse, by moving with equal probability to adjace… ▽ More We analyze the movement of a starving forager on a one-dimensional periodic lattice, where each location contains one unit of food. As the forager lands on sites with food, it consumes the food, leaving the sites empty. If the forager lands consecutively on $s$ empty sites, then it will starve. The forager has two modes of movement: it can either diffuse, by moving with equal probability to adjacent sites on the lattice, or it can jump to a uniformly randomly chosen site on the lattice. We show that the lifetime $T$ of the forager in either paradigm can be approximated by the sum of the cover time $τ_{\rm cover}$ and the starvation time $s$, when $s$ far exceeds the number $n$ of lattice sites. Our main findings focus on the hybrid model, where the forager has a probability of either jum** or diffusing. The lifetime of the forager varies non-monotonically according to $p_j$, the probability of jum**. By examining a small system, analyzing a heuristic model, and using direct numerical simulation, we explore the tradeoff between jumps and diffusion, and show that the strategy that maximizes the forager lifetime is a mixture of both modes of movement. △ Less

Submitted 17 July, 2018; originally announced July 2018.

Comments: 8 pages, 5 figures

Journal ref: Phys. Rev. E 98, 052406 (2018)

Optimizing sequential decisions in the drift-diffusion model

Authors: Khanh P Nguyen, Kresimir Josic, Zachary P Kilpatrick

Abstract: To make decisions organisms often accumulate information across multiple timescales. However, most experimental and modeling studies of decision-making focus on sequences of independent trials. On the other hand, natural environments are characterized by long temporal correlations, and evidence used to make a present choice is often relevant to future decisions. To understand decision-making under… ▽ More To make decisions organisms often accumulate information across multiple timescales. However, most experimental and modeling studies of decision-making focus on sequences of independent trials. On the other hand, natural environments are characterized by long temporal correlations, and evidence used to make a present choice is often relevant to future decisions. To understand decision-making under these conditions we analyze how a model ideal observer accumulates evidence to freely make choices across a sequence of correlated trials. We use principles of probabilistic inference to show that an ideal observer incorporates information obtained on one trial as an initial bias on the next. This bias decreases the time, but not the accuracy of the next decision. Furthermore, in finite sequences of trials the rate of reward is maximized when the observer deliberates longer for early decisions, but responds more quickly towards the end of the sequence. Our model also explains experimentally observed patterns in decision times and choices, thus providing a mathematically principled foundation for evidence-accumulation models of sequential decisions. △ Less

Submitted 11 June, 2018; originally announced June 2018.

Comments: 20 pages, 6 figures

Threshold of front propagation in neural fields: An interface dynamics approach

Authors: Gregory Faye, Zachary P Kilpatrick

Abstract: Neural field equations model population dynamics of large-scale networks of neurons. Wave propagation in neural fields is often studied by constructing traveling wave solutions in the wave coordinate frame. Nonequilibrium dynamics are more challenging to study, due to the nonlinearity and nonlocality of neural fields, whose interactions are described by the kernel of an integral term. Here, we lev… ▽ More Neural field equations model population dynamics of large-scale networks of neurons. Wave propagation in neural fields is often studied by constructing traveling wave solutions in the wave coordinate frame. Nonequilibrium dynamics are more challenging to study, due to the nonlinearity and nonlocality of neural fields, whose interactions are described by the kernel of an integral term. Here, we leverage interface methods to describe the threshold of wave initiation away from equilibrium. In particular, we focus on traveling front initiation in an excitatory neural field. In a neural field with a Heaviside firing rate, neural activity can be described by the dynamics of the interfaces, where the neural activity is at the firing threshold. This allows us to derive conditions for the portion of the neural field that must be activated for traveling fronts to be initiated in a purely excitatory neural field. Explicit equations are possible for a single active (superthreshold) region, and special cases of multiple disconnected active regions. The dynamic spreading speed of the excited region can also be approximated asymptotically. We also discuss extensions to the problem of finding the critical spatiotemporal input needed to initiate waves. △ Less

Submitted 17 January, 2018; originally announced January 2018.

Comments: 27 pages, 9 figures

Synaptic efficacy shapes resource limitations in working memory

Authors: Nikhil Krishnan, Daniel B Poll, Zachary P Kilpatrick

Abstract: Working memory (WM) is limited in its temporal length and capacity. Classic conceptions of WM capacity assume the system possesses a finite number of slots, but recent evidence suggests WM may be a continuous resource. Resource models typically assume there is no hard upper bound on the number of items that can be stored, but WM fidelity decreases with the number of items. We analyze a neural fiel… ▽ More Working memory (WM) is limited in its temporal length and capacity. Classic conceptions of WM capacity assume the system possesses a finite number of slots, but recent evidence suggests WM may be a continuous resource. Resource models typically assume there is no hard upper bound on the number of items that can be stored, but WM fidelity decreases with the number of items. We analyze a neural field model of multi-item WM that associates each item with the location of a bump in a finite spatial domain, considering items that span a one-dimensional continuous feature space. Our analysis relates the neural architecture of the network to accumulated errors and capacity limitations arising during the delay period of a multi-item WM task. Networks with stronger synapses support wider bumps that interact more, whereas networks with weaker synapses support narrower bumps that are more susceptible to noise perturbations. There is an optimal synaptic strength that both limits bump interaction events and the effects of noise perturbations. This optimum shifts to weaker synapses as the number of items stored in the network is increased. Our model not only provides a neural circuit explanation for WM capacity, but also speaks to how capacity relates to the arrangement of stored items in a feature space. △ Less

Submitted 10 February, 2018; v1 submitted 31 October, 2017; originally announced October 2017.

Comments: 26 pages, 12 figures

Power Modelling for Heterogeneous Cloud-Edge Data Centers

Authors: Kai Chen, Blesson Varghese, Peter Kilpatrick, Dimitrios S. Nikolopoulos

Abstract: Existing power modelling research focuses not on the method used for develo** models but rather on the model itself. This paper aims to develop a method for deploying power models on emerging processors that will be used, for example, in cloud-edge data centers. Our research first develops a hardware counter selection method that appropriately selects counters most correlated to power on ARM and… ▽ More Existing power modelling research focuses not on the method used for develo** models but rather on the model itself. This paper aims to develop a method for deploying power models on emerging processors that will be used, for example, in cloud-edge data centers. Our research first develops a hardware counter selection method that appropriately selects counters most correlated to power on ARM and Intel processors. Then, we propose a two stage power model that works across multiple architectures. The key results are: (i) the automated hardware performance counter selection method achieves comparable selection to the manual selection methods reported in literature, and (ii) the two stage power model can predict dynamic power more accurately on both ARM and Intel processors when compared to classic power models. △ Less

Submitted 27 October, 2017; originally announced October 2017.

Comments: 10 pages,10 figures,conference

Neural field model of memory-guided search

Authors: Zachary P Kilpatrick, Daniel B Poll

Abstract: Many organisms can remember locations they have previously visited during a search. Visual search experiments have shown exploration is guided away from these locations, reducing the overlap of the search path before finding a hidden target. We develop and analyze a two-layer neural field model that encodes positional information during a search task. A position-encoding layer sustains a bump attr… ▽ More Many organisms can remember locations they have previously visited during a search. Visual search experiments have shown exploration is guided away from these locations, reducing the overlap of the search path before finding a hidden target. We develop and analyze a two-layer neural field model that encodes positional information during a search task. A position-encoding layer sustains a bump attractor corresponding to the searching agent's current location, and search is modeled by velocity input that propagates the bump. A memory layer sustains persistent activity bounded by a wave front, whose edges expand in response to excitatory input from the position layer. Search can then be biased in response to remembered locations, influencing velocity inputs to the position layer. Asymptotic techniques are used to reduce the dynamics of our model to a low-dimensional system of equations that track the bump position and front boundary. Performance is compared for different target-finding tasks. △ Less

Submitted 6 September, 2017; originally announced September 2017.

Comments: 17 pages, 10 figures

Journal ref: Phys. Rev. E 96, 062411 (2017)

Synaptic mechanisms of interference in working memory

Authors: Zachary P Kilpatrick

Abstract: Information from preceding trials of cognitive tasks can bias performance in the current trial, a phenomenon referred to as interference. Subjects performing visual working memory tasks exhibit interference in their trial-to-trial response correlations: the recalled target location in the current trial is biased in the direction of the target presented on the previous trial. We present modeling wo… ▽ More Information from preceding trials of cognitive tasks can bias performance in the current trial, a phenomenon referred to as interference. Subjects performing visual working memory tasks exhibit interference in their trial-to-trial response correlations: the recalled target location in the current trial is biased in the direction of the target presented on the previous trial. We present modeling work that (a) develops a probabilistic inference model of this history-dependent bias, and (b) links our probabilistic model to computations of a recurrent network wherein short-term facilitation accounts for the dynamics of the observed bias. Network connectivity is reshaped dynamically during each trial, providing a mechanism for generating predictions from prior trial observations. Applying timescale separation methods, we can obtain a low-dimensional description of the trial-to-trial bias based on the history of target locations. The model has response statistics whose mean is centered at the true target location across many trials, typical of such visual working memory tasks. Furthermore, we demonstrate task protocols for which the plastic model performs better than a model with static connectivity: repetitively presented targets are better retained in working memory than targets drawn from uncorrelated sequences. △ Less

Submitted 24 July, 2017; v1 submitted 16 June, 2017; originally announced June 2017.

Comments: 28 pages, 7 figures

Velocity integration in a multilayer neural field model of spatial working memory

Authors: Daniel B. Poll, Zachary P. Kilpatrick

Abstract: We analyze a multilayer neural field model of spatial working memory, focusing on the impact of interlaminar connectivity, spatial heterogeneity, and velocity inputs. Models of spatial working memory typically employ networks that generate persistent activity via a combination of local excitation and lateral inhibition. Our model is comprised of a multilayer set of equations that describes connect… ▽ More We analyze a multilayer neural field model of spatial working memory, focusing on the impact of interlaminar connectivity, spatial heterogeneity, and velocity inputs. Models of spatial working memory typically employ networks that generate persistent activity via a combination of local excitation and lateral inhibition. Our model is comprised of a multilayer set of equations that describes connectivity between neurons in the same and different layers using an integral term. The kernel of this integral term then captures the impact of different interlaminar connection strengths, spatial heterogeneity, and velocity input. We begin our analysis by focusing on how interlaminar connectivity shapes the form and stability of (persistent) bump attractor solutions to the model. Subsequently, we derive a low-dimensional approximation that describes how spatial heterogeneity, velocity input, and noise combine to determine the position of bump solutions. The main impact of spatial heterogeneity is to break the translation symmetry of the network, so bumps prefer to reside at one of a finite number of local attractors in the domain. With the reduced model in hand, we can then approximate the dynamics of the bump position using a continuous time Markov chain model that describes bump motion between local attractors. While heterogeneity reduces the effective diffusion of the bumps, it also disrupts the processing of velocity inputs by slowing the velocity-induced propagation of bumps. However, we demonstrate that noise can play a constructive role by promoting bump motion transitions, restoring a mean bump velocity that is close to the input velocity. △ Less

Submitted 16 January, 2017; v1 submitted 7 November, 2016; originally announced November 2016.

Comments: 37 pages, 9 figures

State access patterns in embarrassingly parallel computations

Authors: Marco Danelutto, Massimo Torquati, Peter Kilpatrick

Abstract: We introduce a set of state access patterns suitable for managing state in embarrassingly parallel computations on streams. The state access patterns are useful to model typical stream parallel applications. We present a classification of the patterns according to the extent and way in which the state is modified. We define precisely the state access patterns and discuss possible implementation sc… ▽ More We introduce a set of state access patterns suitable for managing state in embarrassingly parallel computations on streams. The state access patterns are useful to model typical stream parallel applications. We present a classification of the patterns according to the extent and way in which the state is modified. We define precisely the state access patterns and discuss possible implementation schemas, performances and possibilities to manage adaptivity (parallelism degree) in the patterns. We present experimental results relative to an implementations on top of the structured parallel programming framework FastFlow that demonstrate the feasibility and efficiency of the proposed access patterns. △ Less

Submitted 16 September, 2016; originally announced September 2016.

Comments: 8 pages, accepted and presented at HLPGPU 2016 (Prague, Czech Republic, Tuesday, Jan 19th 2016. Co-Located with HiPEAC 2016)

A parallel pattern for iterative stencil + reduce

Authors: M. Aldinucci, M. Danelutto, M. Drocco, P. Kilpatrick, C. Misale, G. Peretti Pezzi, M. Torquati

Abstract: We advocate the Loop-of-stencil-reduce pattern as a means of simplifying the implementation of data-parallel programs on heterogeneous multi-core platforms. Loop-of-stencil-reduce is general enough to subsume map, reduce, map-reduce, stencil, stencil-reduce, and, crucially, their usage in a loop in both data-parallel and streaming applications, or a combination of both. The pattern makes it possib… ▽ More We advocate the Loop-of-stencil-reduce pattern as a means of simplifying the implementation of data-parallel programs on heterogeneous multi-core platforms. Loop-of-stencil-reduce is general enough to subsume map, reduce, map-reduce, stencil, stencil-reduce, and, crucially, their usage in a loop in both data-parallel and streaming applications, or a combination of both. The pattern makes it possible to deploy a single stencil computation kernel on different GPUs. We discuss the implementation of Loop-of-stencil-reduce in FastFlow, a framework for the implementation of applications based on the parallel patterns. Experiments are presented to illustrate the use of Loop-of-stencil-reduce in develo** data-parallel kernels running on heterogeneous systems. △ Less

Submitted 15 September, 2016; originally announced September 2016.

Challenges and Opportunities in Edge Computing

Authors: Blesson Varghese, Nan Wang, Sakil Barbhuiya, Peter Kilpatrick, Dimitrios S. Nikolopoulos

Abstract: Many cloud-based applications employ a data centre as a central server to process data that is generated by edge devices, such as smartphones, tablets and wearables. This model places ever increasing demands on communication and computational infrastructure with inevitable adverse effect on Quality-of-Service and Experience. The concept of Edge Computing is predicated on moving some of this comput… ▽ More Many cloud-based applications employ a data centre as a central server to process data that is generated by edge devices, such as smartphones, tablets and wearables. This model places ever increasing demands on communication and computational infrastructure with inevitable adverse effect on Quality-of-Service and Experience. The concept of Edge Computing is predicated on moving some of this computational load towards the edge of the network to harness computational capabilities that are currently untapped in edge nodes, such as base stations, routers and switches. This position paper considers the challenges and opportunities that arise out of this new direction in the computing landscape. △ Less

Submitted 7 September, 2016; originally announced September 2016.

Comments: 6 pages, accepted to IEEE SmartCloud 2016

Evidence accumulation and change rate inference in dynamic environments

Authors: Adrian E Radillo, Alan Veliz-Cuba, Kresimir Josic, Zachary P Kilpatrick

Abstract: In a constantly changing world, animals must account for environmental volatility when making decisions. To appropriately discount older, irrelevant information, they need to learn the rate at which the environment changes. We develop an ideal observer model capable of inferring the present state of the environment along with its rate of change. Key to this computation is an update of the posterio… ▽ More In a constantly changing world, animals must account for environmental volatility when making decisions. To appropriately discount older, irrelevant information, they need to learn the rate at which the environment changes. We develop an ideal observer model capable of inferring the present state of the environment along with its rate of change. Key to this computation is an update of the posterior probability of all possible changepoint counts. This computation can be challenging, as the number of possibilities grows rapidly with time. However, we show how the computations can be simplified in the continuum limit by a moment closure approximation. The resulting low-dimensional system can be used to infer the environmental state and change rate with accuracy comparable to the ideal observer. The approximate computations can be performed by a neural network model via a rate-correlation based plasticity rule. We thus show how optimal observers accumulate evidence in changing environments, and map this computation to reduced models which perform inference using plausible neural mechanisms. △ Less

Submitted 11 January, 2017; v1 submitted 28 July, 2016; originally announced July 2016.

Comments: 43 pages, 8 figures, in press

Journal ref: Neural Computation (2017)

Persistent search in confined domains: a velocity-jump process model

Authors: Daniel B. Poll, Zachary P. Kilpatrick

Abstract: We analyze velocity-jump process models of persistent search for a single target on a bounded domain. The searcher proceeds along ballistic trajectories and is absorbed upon collision with the target boundary. When reaching the domain boundary, the searcher chooses a random direction for its new trajectory. For circular domains and targets, we can approximate the mean first passage time (MFPT) usi… ▽ More We analyze velocity-jump process models of persistent search for a single target on a bounded domain. The searcher proceeds along ballistic trajectories and is absorbed upon collision with the target boundary. When reaching the domain boundary, the searcher chooses a random direction for its new trajectory. For circular domains and targets, we can approximate the mean first passage time (MFPT) using a Markov chain approximation of the search process. Our analysis and numerical simulations reveal that the time to find the target decreases for targets closer to the domain boundary. When there is a small probability of direction-switching within the domain, we find the time to find the target decreases slightly with the turning probability. We also extend our exit time analysis to the case of partitioned domains, where there is a single target within one of multiple disjoint subdomains. Given an average time of transition between domains $\langle T \rangle$, we find that the optimal rate of transition that minimizes the time to find the target obeys $β_{\rm min} \propto 1/ \sqrt{\langle T \rangle}$. △ Less

Submitted 16 February, 2016; originally announced February 2016.

Comments: 20 pages, 7 figures

Sensory feedback in a bump attractor model of path integration

Authors: Daniel B Poll, Khanh Nguyen, Zachary P Kilpatrick

Abstract: The mammalian spatial navigation system makes use of several different sensory information channels. This information is then converted into a neural code that represents the animal's current position in space by engaging place cell, grid cell, and head direction cell networks. In particular, sensory landmark (allothetic) cues can be utilized in concert with an animal's knowledge of its own veloci… ▽ More The mammalian spatial navigation system makes use of several different sensory information channels. This information is then converted into a neural code that represents the animal's current position in space by engaging place cell, grid cell, and head direction cell networks. In particular, sensory landmark (allothetic) cues can be utilized in concert with an animal's knowledge of its own velocity (idiothetic) cues to generate a more accurate representation of position than (idiothetic) path integration provides on its own (Battaglia et al, 2004). We develop a computational model that merges path integration with information from external sensory cues that provide a reliable representation of spatial position along an annular track. Starting with a continuous bump attractor model, we allow for the possibility of synaptic spatial heterogeneity that would break the translation symmetry of space. We use asymptotic analysis to reduce the bump attractor model to a single scalar equation whose potential represents the impact of heterogeneity. Such heterogeneity causes errors to build up when the network performs path integration, but these errors can be corrected by an external control signal representing the effects of sensory cues. We demonstrate that there is an optimal strength and decay rate of the control signal when cues are placed both periodically and randomly. A similar analysis is performed when errors in path integration arise from dynamic noise fluctuations. Again, there is an optimal strength and decay of discrete control that minimizes the path integration error. △ Less

Submitted 15 July, 2015; originally announced July 2015.

Comments: 24 pages, 10 figures

Ghosts of bump attractors in stochastic neural fields: Bottlenecks and extinction

Authors: Zachary P. Kilpatrick

Abstract: We study the effects of additive noise on stationary bump solutions to spatially extended neural fields near a saddle-node bifurcation. The integral terms of these evolution equations have a weight kernel describing synaptic interactions between neurons at different locations of the network. Excited regions of the neural field correspond to parts of the domain whose fraction of active neurons exce… ▽ More We study the effects of additive noise on stationary bump solutions to spatially extended neural fields near a saddle-node bifurcation. The integral terms of these evolution equations have a weight kernel describing synaptic interactions between neurons at different locations of the network. Excited regions of the neural field correspond to parts of the domain whose fraction of active neurons exceeds a sharp threshold of a firing rate nonlinearity. For sufficiently low firing threshold, a stable bump coexists with an unstable bump and a homogeneous quiescent state. As the threshold is increased, the stable and unstable branch of bump solutions annihilate in a saddle node bifurcation. Near this criticality, we derive a quadratic amplitude equation that describes the slow evolution of the even mode (bump contractions) as it depends on the distance from the bifurcation. Beyond the bifurcation, bumps eventually become extinct, and the time it takes for this to occur increases for systems nearer the bifurcation. When noise is incorporated, a stochastic amplitude equation for the even mode can be derived, which can be analyzed to reveal bump extinction time both below and above the saddle-node. △ Less

Submitted 24 August, 2015; v1 submitted 22 May, 2015; originally announced May 2015.

Comments: 21 pages, 5 figures

Stochastic models of evidence accumulation in changing environments

Authors: Alan Veliz-Cuba, Zachary P. Kilpatrick, Kresimir Josic

Abstract: Organisms and ecological groups accumulate evidence to make decisions. Classic experiments and theoretical studies have explored this process when the correct choice is fixed during each trial. However, we live in a constantly changing world. What effect does such impermanence have on classical results about decision making? To address this question we use sequential analysis to derive a tractable… ▽ More Organisms and ecological groups accumulate evidence to make decisions. Classic experiments and theoretical studies have explored this process when the correct choice is fixed during each trial. However, we live in a constantly changing world. What effect does such impermanence have on classical results about decision making? To address this question we use sequential analysis to derive a tractable model of evidence accumulation when the correct option changes in time. Our analysis shows that ideal observers discount prior evidence at a rate determined by the volatility of the environment, and the dynamics of evidence accumulation is governed by the information gained over an average environmental epoch. A plausible neural implementation of an optimal observer in a changing environment shows that, in contrast to previous models, neural populations representing alternate choices are coupled through excitation. Our work builds a bridge between statistical decision making in volatile environments and stochastic nonlinear dynamics. △ Less

Submitted 30 September, 2015; v1 submitted 15 May, 2015; originally announced May 2015.

Comments: 26 pages, 7 figures

Entrainment in up and down states of neural populations: non-smooth and stochastic models

Authors: Zachary McCleney, Zachary P. Kilpatrick

Abstract: We study the impact of noise on a neural population rate model of up and down states. Up and down states are typically observed in neuronal networks as a slow oscillation, where the population switches between high and low firing rates (Sanchez-Vivez and McCormick, 2000). A neural population model with spike rate adaptation is used to model such slow oscillations, and the timescale of adaptation d… ▽ More We study the impact of noise on a neural population rate model of up and down states. Up and down states are typically observed in neuronal networks as a slow oscillation, where the population switches between high and low firing rates (Sanchez-Vivez and McCormick, 2000). A neural population model with spike rate adaptation is used to model such slow oscillations, and the timescale of adaptation determines the oscillation period. Furthermore, the period depends non-monotonically on the background tonic input driving the population, having long periods for very weak and very strong stimuli. Using both linearization and fast-slow timescale separation methods, we can compute the phase sensitivity function of the slow oscillation. We find that the phase response is most strongly impacted by perturbations to the adaptation variable. Phase sensitivity functions can then be utilized to quantify the impact of noise on oscillating populations. Noise alters the period of oscillations by speeding up the rate of transition between the up and down states. When common noise is presented to two distinct populations, their transitions will eventually become entrained to one another through stochastic synchrony. △ Less

Submitted 23 April, 2015; originally announced April 2015.

Comments: 23 pages, 7 figures

Stochastic synchronization of neural activity waves

Authors: Zachary P. Kilpatrick

Abstract: We demonstrate that waves in distinct layers of a neuronal network can become phase-locked by common spatiotemporal noise. This phenomenon is studied for stationary bumps, traveling waves, and breathers. A weak noise expansion is used to derive an effective equation for the position of the wave in each layer, yielding a stochastic differential equation with multiplicative noise. Stability of the s… ▽ More We demonstrate that waves in distinct layers of a neuronal network can become phase-locked by common spatiotemporal noise. This phenomenon is studied for stationary bumps, traveling waves, and breathers. A weak noise expansion is used to derive an effective equation for the position of the wave in each layer, yielding a stochastic differential equation with multiplicative noise. Stability of the synchronous state is characterized by a Lyapunov exponent, which we can compute analytically from the reduced system. Our results extend previous work on limit-cycle oscillators, showing common noise can synchronize waves in a broad class of models. △ Less

Submitted 28 March, 2015; v1 submitted 12 December, 2014; originally announced December 2014.

Comments: 6 pages, 4 figures

Journal ref: Phys. Rev. E 91, 040701 (2015)

Stochastic motion of bumps in planar neural fields

Authors: Daniel Poll, Zachary P. Kilpatrick

Abstract: We analyze the effects of spatiotemporal noise on stationary pulse solutions (bumps) in neural field equations on planar domains. Neural fields are integrodifferential equations whose integral kernel describes the strength and polarity of synaptic interactions between neurons at different spatial locations of the network. Fluctuations in neural activity are incorporated by modeling the system as a… ▽ More We analyze the effects of spatiotemporal noise on stationary pulse solutions (bumps) in neural field equations on planar domains. Neural fields are integrodifferential equations whose integral kernel describes the strength and polarity of synaptic interactions between neurons at different spatial locations of the network. Fluctuations in neural activity are incorporated by modeling the system as a Langevin equation evolving on a planar domain. Noise causes bumps to wander about the domain in a purely diffusive way. Utilizing a small noise expansion along with a solvability condition, we can derive an effective stochastic equation describing the bump dynamics as two-dimensional Brownian motion. The diffusion coefficient can then be computed explicitly. We also show that weak external inputs can pin the bump so it no longer wanders diffusively. Inputs reshape the effective potential that guides the dynamics of the bump position, so it tends to lie near attractors which can be single points or contours in the plane. Perturbative analysis shows the bump position evolves as a multivariate Ornstein-Uhlenbeck process whose relaxation constants are determined by the shape of the input. Our analytical approximations all compare well to statistics of bump motion in numerical simulations. △ Less

Submitted 19 April, 2015; v1 submitted 10 December, 2014; originally announced December 2014.

Comments: 25 pages, 7 figures

Networks that learn the precise timing of event sequences

Authors: Alan Veliz-Cuba, Harel Shouval, Kresimir Josic, Zachary P. Kilpatrick

Abstract: Neuronal circuits can learn and replay firing patterns evoked by sequences of sensory stimuli. After training, a brief cue can trigger a spatiotemporal pattern of neural activity similar to that evoked by a learned stimulus sequence. Network models show that such sequence learning can occur through the sha** of feedforward excitatory connectivity via long term plasticity. Previous models describ… ▽ More Neuronal circuits can learn and replay firing patterns evoked by sequences of sensory stimuli. After training, a brief cue can trigger a spatiotemporal pattern of neural activity similar to that evoked by a learned stimulus sequence. Network models show that such sequence learning can occur through the sha** of feedforward excitatory connectivity via long term plasticity. Previous models describe how event order can be learned, but they typically do not explain how precise timing can be recalled. We propose a mechanism for learning both the order and precise timing of event sequences. In our recurrent network model, long term plasticity leads to the learning of the sequence, while short term facilitation enables temporally precise replay of events. Learned synaptic weights between populations determine the time necessary for one population to activate another. Long term plasticity adjusts these weights so that the trained event times are matched during playback. While we chose short term facilitation as a time-tracking process, we also demonstrate that other mechanisms, such as spike rate adaptation, can fulfill this role. We also analyze the impact of trial-to-trial variability, showing how observational errors as well as neuronal noise result in variability in learned event times. The dynamics of the playback process determine how stochasticity is inherited in learned sequence timings. Future experiments that characterize such variability can therefore shed light on the neural mechanisms of sequence learning. △ Less

Submitted 2 July, 2015; v1 submitted 4 December, 2014; originally announced December 2014.

Comments: 27 pages, 8 figures

Nonlinear Langevin equations for wandering patterns in stochastic neural fields

Authors: Paul C. Bressloff, Zachary P. Kilpatrick

Abstract: We analyze the effects of additive, spatially extended noise on spatiotemporal patterns in continuum neural fields. Our main focus is how fluctuations impact patterns when they are weakly coupled to an external stimulus or another equivalent pattern. Showing the generality of our approach, we study both propagating fronts and stationary bumps. Using a separation of time scales, we represent the ef… ▽ More We analyze the effects of additive, spatially extended noise on spatiotemporal patterns in continuum neural fields. Our main focus is how fluctuations impact patterns when they are weakly coupled to an external stimulus or another equivalent pattern. Showing the generality of our approach, we study both propagating fronts and stationary bumps. Using a separation of time scales, we represent the effects of noise in terms of a phase-shift of a pattern from its uniformly translating position at long time scales, and fluctuations in the pattern profile around its instantaneous position at short time scales. In the case of a stimulus-locked front, we show that the phase-shift satisfies a nonlinear Langevin equation (SDE) whose deterministic part has a unique stable fixed point. Using a linear-noise approximation, we thus establish that wandering of the front about the stimulus-locked state is given by an Ornstein-Uhlenbeck (OU) process. Analogous results hold for the relative phase-shift between a pair of mutually coupled fronts, provided that the coupling is excitatory. On the other hand, if the mutual coupling is given by a Mexican hat function (difference of exponentials), then the linear-noise approximation breaks down due to the co-existence of stable and unstable phase-locked states in the deterministic limit. Similarly, the stochastic motion of mutually coupled bumps can be described by a system of nonlinearly coupled SDEs, which can be linearized to yield a multivariate OU process. As in the case of fronts, large deviations can cause bumps to temporarily decouple, leading to a phase-slip in the bump positions. △ Less

Submitted 6 January, 2015; v1 submitted 6 October, 2014; originally announced October 2014.

Comments: 28 pages, 8 figures

Delay stabilizes stochastic motion of bumps in layered neural fields

Authors: Zachary P. Kilpatrick

Abstract: We study the effects of propagation delays on the stochastic dynamics of bumps in neural fields with multiple layers. In the absence of noise, each layer supports a stationary bump. Using linear stability analysis, we show that delayed coupling between layers causes translating perturbations of the bumps to decay in the noise-free system. Adding noise to the system causes bumps to wander as a rand… ▽ More We study the effects of propagation delays on the stochastic dynamics of bumps in neural fields with multiple layers. In the absence of noise, each layer supports a stationary bump. Using linear stability analysis, we show that delayed coupling between layers causes translating perturbations of the bumps to decay in the noise-free system. Adding noise to the system causes bumps to wander as a random walk. However, coupling between layers can reduce the variability of this stochastic motion by canceling noise that perturbs bumps in opposite directions. Delays in interlaminar coupling can further reduce variability, since they couple bump positions to states from the past. We demonstrate these relationships by deriving an asymptotic approximation for the effective motion of bumps. This yields a stochastic delay-differential equation where each delayed term arises from an interlaminar coupling. The impact of delays is well approximated by using a small delay expansion, which allows us to compute the effective diffusion in bumps' positions, accurately matching results from numerical simulations. △ Less

Submitted 23 December, 2014; v1 submitted 25 September, 2014; originally announced September 2014.

Comments: 21 pages, 7 figures; To appear in Physica D

Molecular mechanisms that regulate the coupled period of the mammalian circadian clock

Authors: Jae Kyoung Kim, Zachary P. Kilpatrick, Matthew R. Bennett, Krešimir Josić

Abstract: In mammals, most cells in the brain and peripheral tissues generate circadian (~24hr) rhythms autonomously. These self-sustained rhythms are coordinated and entrained by a master circadian clock in the suprachiasmatic nucleus (SCN). Within the SCN, the individual rhythms of each neuron are synchronized through intercellular signaling. One important feature of SCN is that the synchronized period is… ▽ More In mammals, most cells in the brain and peripheral tissues generate circadian (~24hr) rhythms autonomously. These self-sustained rhythms are coordinated and entrained by a master circadian clock in the suprachiasmatic nucleus (SCN). Within the SCN, the individual rhythms of each neuron are synchronized through intercellular signaling. One important feature of SCN is that the synchronized period is close to the cell population mean of intrinsic periods. In this way, the synchronized period of the SCN stays close to the periods of cells in peripheral tissues. This is important for SCN to entrain cells throughout the body. However, the mechanism that drives the period of the coupled SCN cells to the population mean is not known. We use mathematical modeling and analysis to show that the mechanism of transcription repression plays a pivotal role in regulating the coupled period. Specifically, we use phase response curve analysis to show that the coupled period within the SCN stays near the population mean if transcriptional repression occurs via protein sequestration. In contrast, the coupled period is far from the mean if repression occurs through highly nonlinear Hill-type regulation (e.g. oligomer- or phosphorylation-based repression). Furthermore, we find that the timescale of intercellular coupling needs to be fast compared to that of intracellular feedback to maintain the mean period. These findings reveal the important relationship between the intracellular transcriptional feedback loop and intercellular coupling. This relationship explains why transcriptional repression appears to occur via protein sequestration in multicellular organisms, mammals and Drosophila, in contrast with the phosphorylation-based repression in unicellular organisms. That is, transition to protein sequestration is essential for synchronizing multiple cells with a period close to the population mean (~24hr). △ Less

Submitted 6 March, 2014; originally announced March 2014.

Comments: 21 pages, 16 figures

MSC Class: 92C42

Journal ref: Biophyjical Journal 106 (2014)