-
Variational Recurrent Models for Solving Partially Observable Control Tasks
Authors:
Dongqi Han,
Kenji Doya,
Jun Tani
Abstract:
In partially observable (PO) environments, deep reinforcement learning (RL) agents often suffer from unsatisfactory performance, since two problems need to be tackled together: how to extract information from the raw observations to solve the task, and how to improve the policy. In this study, we propose an RL algorithm for solving PO tasks. Our method comprises two parts: a variational recurrent…
▽ More
In partially observable (PO) environments, deep reinforcement learning (RL) agents often suffer from unsatisfactory performance, since two problems need to be tackled together: how to extract information from the raw observations to solve the task, and how to improve the policy. In this study, we propose an RL algorithm for solving PO tasks. Our method comprises two parts: a variational recurrent model (VRM) for modeling the environment, and an RL controller that has access to both the environment and the VRM. The proposed algorithm was tested in two types of PO robotic control tasks, those in which either coordinates or velocities were not observable and those that require long-term memorization. Our experiments show that the proposed algorithm achieved better data efficiency and/or learned more optimal policy than other alternative approaches in tasks in which unobserved states cannot be inferred from raw observations in a simple manner.
△ Less
Submitted 24 December, 2019; v1 submitted 23 December, 2019;
originally announced December 2019.
-
Optimization and Validation of Diffusion MRI-based Fiber Tracking with Neural Tracer Data as a Reference
Authors:
Carlos Enrique Gutierrez,
Henrik Skibbe,
Ken Nakae,
Hiromichi Tsukada,
Jean Lienard,
Akiya Watakabe,
Junichi Hata,
Marco Reisert,
Alexander Woodward,
Hideyuki Okano,
Tetsuo Yamamori,
Yoko Yamaguchi,
Shin Ishii,
Kenji Doya
Abstract:
Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can potentially help to reveal the brain's global network architecture and abnormalities involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false…
▽ More
Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can potentially help to reveal the brain's global network architecture and abnormalities involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false positives, and inaccurate and incomplete reconstruction of long-range connections. Furthermore, parameters of tracking algorithms are typically tuned in a heuristic way, which leaves room for manipulation of an intended result. Here we propose a data-driven framework to optimize and validate parameters of dMRI-based fiber-tracking algorithms using neural tracer data as a reference. Japan's Brain/MINDS Project provides invaluable datasets containing both dMRI and neural tracer data from the same primates. We considered four criteria for goodness of fiber tracking: distance-weighted coverage, true/false positive ratio, projection coincidence, and commissural passage, applied using a multi-objective optimization algorithm. We implemented a variant of non-dominated sorting genetic algorithm II (NSGA-II) to optimize five major parameters of a global fiber-tracking algorithm over multiple brain samples in parallel. Using optimized parameters compared to the default parameters, dMRI-based fiber tracking performance was significantly improved, while minimizing false positives and impossible cross-hemisphere connections. Parameters optimized for 10 tracer injection sites showed good generalization capability for other brain samples. These results demonstrate the importance of data-driven adjustment of fiber-tracking algorithms and support the validity of dMRI-based tractography, if appropriate adjustments are employed.
△ Less
Submitted 24 April, 2020; v1 submitted 27 November, 2019;
originally announced November 2019.
-
MarmoNet: a pipeline for automated projection map** of the common marmoset brain from whole-brain serial two-photon tomography
Authors:
Henrik Skibbe,
Akiya Watakabe,
Ken Nakae,
Carlos Enrique Gutierrez,
Hiromichi Tsukada,
Junichi Hata,
Takashi Kawase,
Rui Gong,
Alexander Woodward,
Kenji Doya,
Hideyuki Okano,
Tetsuo Yamamori,
Shin Ishii
Abstract:
Understanding the connectivity in the brain is an important prerequisite for understanding how the brain processes information. In the Brain/MINDS project, a connectivity study on marmoset brains uses two-photon microscopy fluorescence images of axonal projections to collect the neuron connectivity from defined brain regions at the mesoscopic scale. The processing of the images requires the detect…
▽ More
Understanding the connectivity in the brain is an important prerequisite for understanding how the brain processes information. In the Brain/MINDS project, a connectivity study on marmoset brains uses two-photon microscopy fluorescence images of axonal projections to collect the neuron connectivity from defined brain regions at the mesoscopic scale. The processing of the images requires the detection and segmentation of the axonal tracer signal. The objective is to detect as much tracer signal as possible while not misclassifying other background structures as the signal. This can be challenging because of imaging noise, a cluttered image background, distortions or varying image contrast cause problems.
We are develo** MarmoNet, a pipeline that processes and analyzes tracer image data of the common marmoset brain. The pipeline incorporates state-of-the-art machine learning techniques based on artificial convolutional neural networks (CNN) and image registration techniques to extract and map all relevant information in a robust manner. The pipeline processes new images in a fully automated way.
This report introduces the current state of the tracer signal analysis part of the pipeline.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
