Showing 1–13 of 13 results for author: Giese, M

Parallel Backpropagation for Shared-Feature Visualization

Authors: Alexander Lappe, Anna Bognár, Ghazaleh Ghamkhari Nejad, Albert Mukovskiy, Lucas Martini, Martin A. Giese, Rufin Vogels

Abstract: High-level visual brain regions contain subareas in which neurons appear to respond more strongly to examples of a particular semantic category, like faces or bodies, rather than objects. However, recent work has shown that while this finding holds on average, some out-of-category stimuli also activate neurons in these regions. This may be due to visual features common among the preferred class al… ▽ More High-level visual brain regions contain subareas in which neurons appear to respond more strongly to examples of a particular semantic category, like faces or bodies, rather than objects. However, recent work has shown that while this finding holds on average, some out-of-category stimuli also activate neurons in these regions. This may be due to visual features common among the preferred class also being present in other images. Here, we propose a deep-learning-based approach for visualizing these features. For each neuron, we identify relevant visual features driving its selectivity by modelling responses to images based on latent activations of a deep neural network. Given an out-of-category image which strongly activates the neuron, our method first identifies a reference image from the preferred category yielding a similar feature activation pattern. We then backpropagate latent activations of both images to the pixel level, while enhancing the identified shared dimensions and attenuating non-shared features. The procedure highlights image regions containing shared features driving responses of the model neuron. We apply the algorithm to novel recordings from body-selective regions in macaque IT cortex in order to understand why some images of objects excite these neurons. Visualizations reveal object parts which resemble parts of a macaque body, shedding light on neural preference of these objects. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements

Authors: Jhon Charaja, Isabell Wochner, Pierre Schumacher, Winfried Ilg, Martin Giese, Christophe Maufroy, Andreas Bulling, Syn Schmitt, Daniel F. B. Haeufle

Abstract: The mimicking of human-like arm movement characteristics involves the consideration of three factors during control policy synthesis: (a) chosen task requirements, (b) inclusion of noise during movement execution and (c) chosen optimality principles. Previous studies showed that when considering these factors (a-c) individually, it is possible to synthesize arm movements that either kinematically… ▽ More The mimicking of human-like arm movement characteristics involves the consideration of three factors during control policy synthesis: (a) chosen task requirements, (b) inclusion of noise during movement execution and (c) chosen optimality principles. Previous studies showed that when considering these factors (a-c) individually, it is possible to synthesize arm movements that either kinematically match the experimental data or reproduce the stereotypical triphasic muscle activation pattern. However, to date no quantitative comparison has been made on how realistic the arm movement generated by each factor is; as well as whether a partial or total combination of all factors results in arm movements with human-like kinematic characteristics and a triphasic muscle pattern. To investigate this, we used reinforcement learning to learn a control policy for a musculoskeletal arm model, aiming to discern which combination of factors (a-c) results in realistic arm movements according to four frequently reported stereotypical characteristics. Our findings indicate that incorporating velocity and acceleration requirements into the reaching task, employing reward terms that encourage minimization of mechanical work, hand jerk, and control effort, along with the inclusion of noise during movement, leads to the emergence of realistic human arm movements in reinforcement learning. We expect that the gained insights will help in the future to better predict desired arm movements and corrective forces in wearable assistive devices. △ Less

Submitted 21 February, 2024; originally announced February 2024.

Map** Physical Conditions in Neighboring Hot Cores: NOEMA Studies of W3(H$_2$O) and W3(OH)

Authors: Morgan M. Giese, Will E. Thompson, Dariusz C. Lis, Susanna L. Widicus Weaver

Abstract: The complex chemistry that occurs in star-forming regions can provide insight into the formation of prebiotic molecules at various evolutionary stages of star formation. To study this process, we present millimeter-wave interferometric observations of the neighboring hot cores W3(H$_2$O) and W3(OH) carried out using the NOEMA interferometer. We have analyzed distributions of six molecules that acc… ▽ More The complex chemistry that occurs in star-forming regions can provide insight into the formation of prebiotic molecules at various evolutionary stages of star formation. To study this process, we present millimeter-wave interferometric observations of the neighboring hot cores W3(H$_2$O) and W3(OH) carried out using the NOEMA interferometer. We have analyzed distributions of six molecules that account for most observed lines across both cores and have constructed physical parameter maps for rotational temperature, column density, and velocity field with corresponding uncertainties. We discuss the derived spatial distributions of these parameters in the context of the physical structure of the source. We propose the use of HCOOCH$_3$ as a new temperature tracer in W3(H$_2$O) and W3(OH) in addition to the more commonly used CH$_3$CN. By analyzing the physically-derived parameters for each molecule across both W3(H$_2$O) and W3(OH), the work presented herein further demonstrates the impact of physical environment on hot cores at different evolutionary stages. △ Less

Submitted 16 November, 2023; v1 submitted 18 October, 2023; originally announced October 2023.

Comments: Accepted to The Astrophysical Journal

The RS Oph outburst of 2021 monitored in X-rays with NICER

Authors: Marina Orio, Keith Gendreau, Morgan Giese, Gerardo Juna M. Luna, Jozef Magdolen, Tod E. Strohmayer, Andy E. Zhang, Diego Altamirano, Andrej Dobrotka, Teruaki Enoto, Elizabeth C. Ferrara, Richard Ignace, Sebastian heinz, Craig Markwardt, Joy S. Nichols, Micahel L. Parker, Dheerajay R. Pasham, Songpeng Pei, Pragati Pradhan, Ron Remillard, James F. Steiner, Francesco Tombesi

Abstract: The 2021 outburst of the symbiotic recurrent nova RS Oph was monitored with the Neutron Star Interior Composition Explorer Mission (NICER) in the 0.2-12 keV range from day one after the optical maximum, until day 88, producing an unprecedented, detailed view of the outburst development. The X-ray flux preceding the supersoft X-ray phase peaked almost 5 days after optical maximum and originated onl… ▽ More The 2021 outburst of the symbiotic recurrent nova RS Oph was monitored with the Neutron Star Interior Composition Explorer Mission (NICER) in the 0.2-12 keV range from day one after the optical maximum, until day 88, producing an unprecedented, detailed view of the outburst development. The X-ray flux preceding the supersoft X-ray phase peaked almost 5 days after optical maximum and originated only in shocked ejecta for 21 to 25 days. The emission was thermal; in the first 5 days only a non-collisional-ionization equilibrium model fits the spectrum, and a transition to equilibrium occurred between days 6 and 12. The ratio of peak X-rays flux measured in the NICER range to that measured with Fermi in the 60 MeV-500 GeV range was about 0.1, and the ratio to the peak flux measured with H.E.S.S. in the 250 GeV-2.5 TeV range was about 100. The central supersoft X-ray source (SSS), namely the shell hydrogen burning white dwarf (WD), became visible in the fourth week, initially with short flares. A huge increase in flux occurred on day 41, but the SSS flux remained variable. A quasi-periodic oscillation every ~35 s was always observed during the SSS phase, with variations in amplitude and a period drift that appeared to decrease in the end. The SSS has characteristics of a WD of mass >1 M(solar). Thermonuclear burning switched off shortly after day 75, earlier than in 2006 outburst. We discuss implications for the nova physics. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Comments: Accepted for publication in the Astrophysical Journal

Comparing Complex Chemistry in Neighboring Hot Cores: NOEMA Studies of W3(H$_{2}$O) and W3(OH)

Authors: Will E. Thompson, Morgan M. Giese, Dariusz C. Lis, Susanna L. Widicus Weaver

Abstract: Presented here are NOEMA interferometric observations of the neighboring hot cores W3(H$_{2}$O) and W3(OH). The presence of two star-forming cores at different evolutionary stages within the same parent cloud presents a unique opportunity to study how the physics of the source and its evolutionary stage impact the chemistry. Through spectral analysis and imaging, we identify over twenty molecules… ▽ More Presented here are NOEMA interferometric observations of the neighboring hot cores W3(H$_{2}$O) and W3(OH). The presence of two star-forming cores at different evolutionary stages within the same parent cloud presents a unique opportunity to study how the physics of the source and its evolutionary stage impact the chemistry. Through spectral analysis and imaging, we identify over twenty molecules in these cores. Most notably, we have detected HDO and CH$_{3}$CH$_{2}$CN in W3(OH), which were previously not detected in this core. We have imaged the molecular emission, revealing new structural features within these sources. W3(OH) shows absorption in a "dusty cocoon" surrounded by molecular emission. These observations also reveal extended emission that is potentially indicative of a low-velocity shock. From the information obtained herein, we have constructed column density and temperature maps for methanol and compared this information to the molecular images. By comparing the spatial distribution of molecules which may be destroyed at later stages of star formation, this work demonstrates the impact of physical environment on chemistry in star-forming regions at different evolutionary stages. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Comments: Published in The Astrophysical Journal

Journal ref: ApJ 952 50 (2023)

Multi-Domain Norm-referenced Encoding Enables Data Efficient Transfer Learning of Facial Expression Recognition

Authors: Michael Stettler, Alexander Lappe, Nick Taubert, Martin Giese

Abstract: People can innately recognize human facial expressions in unnatural forms, such as when depicted on the unusual faces drawn in cartoons or when applied to an animal's features. However, current machine learning algorithms struggle with out-of-domain transfer in facial expression recognition (FER). We propose a biologically-inspired mechanism for such transfer learning, which is based on norm-refer… ▽ More People can innately recognize human facial expressions in unnatural forms, such as when depicted on the unusual faces drawn in cartoons or when applied to an animal's features. However, current machine learning algorithms struggle with out-of-domain transfer in facial expression recognition (FER). We propose a biologically-inspired mechanism for such transfer learning, which is based on norm-referenced encoding, where patterns are encoded in terms of difference vectors relative to a domain-specific reference vector. By incorporating domain-specific reference frames, we demonstrate high data efficiency in transfer learning across multiple domains. Our proposed architecture provides an explanation for how the human brain might innately recognize facial expressions on varying head shapes (humans, monkeys, and cartoon avatars) without extensive training. Norm-referenced encoding also allows the intensity of the expression to be read out directly from neural unit activity, similar to face-selective neurons in the brain. Our model achieves a classification accuracy of 92.15\% on the FERG dataset with extreme data efficiency. We train our proposed mechanism with only 12 images, including a single image of each class (facial expression) and one image per domain (avatar). In comparison, the authors of the FERG dataset achieved a classification accuracy of 89.02\% with their FaceExpr model, which was trained on 43,000 images. △ Less

Submitted 5 April, 2023; originally announced April 2023.

GeoFault: A well-founded fault ontology for interoperability in geological modeling

Authors: Yuanwei Qu, Michel Perrin, Anita Torabi, Mara Abel, Martin Giese

Abstract: Geological modeling currently uses various computer-based applications. Data harmonization at the semantic level by means of ontologies is essential for making these applications interoperable. Since geo-modeling is currently part of multidisciplinary projects, semantic harmonization is required to model not only geological knowledge but also to integrate other domain knowledge at a general level.… ▽ More Geological modeling currently uses various computer-based applications. Data harmonization at the semantic level by means of ontologies is essential for making these applications interoperable. Since geo-modeling is currently part of multidisciplinary projects, semantic harmonization is required to model not only geological knowledge but also to integrate other domain knowledge at a general level. For this reason, the domain ontologies used for describing geological knowledge must be based on a sound ontology background to ensure the described geological knowledge is integratable. This paper presents a domain ontology: GeoFault, resting on the Basic Formal Ontology BFO (Arp et al., 2015) and the GeoCore ontology (Garcia et al., 2020). It models the knowledge related to geological faults. Faults are essential to various industries but are complex to model. They can be described as thin deformed rock volumes or as spatial arrangements resulting from the different displacements of geological blocks. At a broader scale, faults are currently described as mere surfaces, which are the components of complex fault arrays. The reference to the BFO and GeoCore package allows assigning these various fault elements to define ontology classes and their logical linkage within a consistent ontology framework. The GeoFault ontology covers the core knowledge of faults 'strico sensu,' excluding ductile shear deformations. This considered vocabulary is essentially descriptive and related to regional to outcrop scales, excluding microscopic, orogenic, and tectonic plate structures. The ontology is molded in OWL 2, validated by competency questions with two use cases, and tested using an in-house ontology-driven data entry application. The work of GeoFault provides a solid framework for disambiguating fault knowledge and a foundation of fault data integration for the applications and the users. △ Less

Submitted 14 February, 2023; originally announced February 2023.

Generating Sparse Counterfactual Explanations For Multivariate Time Series

Authors: Jana Lang, Martin Giese, Winfried Ilg, Sebastian Otte

Abstract: Since neural networks play an increasingly important role in critical sectors, explaining network predictions has become a key research topic. Counterfactual explanations can help to understand why classifier models decide for particular class assignments and, moreover, how the respective input samples would have to be modified such that the class prediction changes. Previous approaches mainly foc… ▽ More Since neural networks play an increasingly important role in critical sectors, explaining network predictions has become a key research topic. Counterfactual explanations can help to understand why classifier models decide for particular class assignments and, moreover, how the respective input samples would have to be modified such that the class prediction changes. Previous approaches mainly focus on image and tabular data. In this work we propose SPARCE, a generative adversarial network (GAN) architecture that generates SPARse Counterfactual Explanations for multivariate time series. Our approach provides a custom sparsity layer and regularizes the counterfactual loss function in terms of similarity, sparsity, and smoothness of trajectories. We evaluate our approach on real-world human motion datasets as well as a synthetic time series interpretability benchmark. Although we make significantly sparser modifications than other approaches, we achieve comparable or better performance on all metrics. Moreover, we demonstrate that our approach predominantly modifies salient time steps and features, leaving non-salient inputs untouched. △ Less

Submitted 4 July, 2022; v1 submitted 2 June, 2022; originally announced June 2022.

Comments: 13 pages, 7 figures. Preprint. Under review; added appendix

NICER monitoring of supersoft X-ray sources

Authors: M. Orio, K. Gendreau, M. Giese, J. G. M. Luna, J. Magdolen, S. Pei, B. Sun, E. Behar, A. Dobrotka, J. Mikolajewska, D. R. Pasham, T. E. Strohmayer

Abstract: We monitored four supersoft sources - two persistent ones, CAL 83 and MR Vel, and the recent novae YZ Ret (Nova Ret 2020) and V1674 Her (Nova Her 2021) - with NICER. The two persistent SSS were observed with unvaried X-ray flux level and spectrum, respectively, 13 and 20 years after the last observations. Short period modulations of the supersoft X-ray source (SSS) appear where the spectrum of the… ▽ More We monitored four supersoft sources - two persistent ones, CAL 83 and MR Vel, and the recent novae YZ Ret (Nova Ret 2020) and V1674 Her (Nova Her 2021) - with NICER. The two persistent SSS were observed with unvaried X-ray flux level and spectrum, respectively, 13 and 20 years after the last observations. Short period modulations of the supersoft X-ray source (SSS) appear where the spectrum of the luminous central source was fully visibl (in CAL 83 and V1674 Her) and were absent in YZ Ret and MR Vel, in which the flux originated in photoionized or shocked plasma, while the white dwarf (WD) was not observable. We thus suggest that the pulsations occur on, or very close to, the WD surface. The pulsations of CAL 83 were almost unvaried after 15 years, including an irregular drift of the $\simeq$67 s period by 2.1 s. Simulations, including previous XMM-Newton data, indicate actual variations in period length within hours, rather than an artifact of the variable amplitude of the pulsations. Large amplitude pulsations with a period of 501.53$\pm$0.30 s were always detected in V1674 Her, as long as the SSS was observable. This period seems to be due to rotation of a highly magnetized WD.We cannot confirm the maximum effective temperature of ($\simeq$145,000 K) previously inferred for this nova, and discuss the difficulty in interpreting its spectrum. The WD appears to present two surface zones, one of which does not emit SSS flux. △ Less

Submitted 4 April, 2022; originally announced April 2022.

Comments: in press in the Astrophysical Journal

Early Recognition of Ball Catching Success in Clinical Trials with RNN-Based Predictive Classification

Authors: Jana Lang, Martin A. Giese, Matthis Synofzik, Winfried Ilg, Sebastian Otte

Abstract: Motor disturbances can affect the interaction with dynamic objects, such as catching a ball. A classification of clinical catching trials might give insight into the existence of pathological alterations in the relation of arm and ball movements. Accurate, but also early decisions are required to classify a catching attempt before the catcher's first ball contact. To obtain clinically valuable res… ▽ More Motor disturbances can affect the interaction with dynamic objects, such as catching a ball. A classification of clinical catching trials might give insight into the existence of pathological alterations in the relation of arm and ball movements. Accurate, but also early decisions are required to classify a catching attempt before the catcher's first ball contact. To obtain clinically valuable results, a significant decision confidence of at least 75% is required. Hence, three competing objectives have to be optimized at the same time: accuracy, earliness and decision-making confidence. Here we propose a coupled classification and prediction approach for early time series classification: a predictive, generative recurrent neural network (RNN) forecasts the next data points of ball trajectories based on already available observations; a discriminative RNN continuously generates classification guesses based on the available data points and the unrolled sequence predictions. We compare our approach, which we refer to as predictive sequential classification (PSC), to state-of-the-art sequence learners, including various RNN and temporal convolutional network (TCN) architectures. On this hard real-world task we can consistently demonstrate the superiority of PSC over all other models in terms of accuracy and confidence with respect to earliness of recognition. Specifically, PSC is able to confidently classify the success of catching trials as early as 123 milliseconds before the first ball contact. We conclude that PSC is a promising approach for early time series classification, when accurate and confident decisions are required. △ Less

Submitted 6 July, 2021; originally announced July 2021.

Comments: Accepted by the 30th International Conference on Artificial Neural Networks (ICANN 2021)

Continuous Decoding of Daily-Life Hand Movements from Forearm Muscle Activity for Enhanced Myoelectric Control of Hand Prostheses

Authors: Alessandro Salatiello, Martin A. Giese

Abstract: State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional map** between forearm EMG activity and hand kinematics is only known… ▽ More State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional map** between forearm EMG activity and hand kinematics is only known with limited accuracy. Therefore, no robust method exists for the reliable computation of control signals for the independent and proportional actuation of more than two DOFs. A common approach to deal with this limitation is to pre-program the prostheses for the execution of a restricted number of behaviors (e.g., pinching, gras**, and wrist rotation) that are activated by the detection of specific EMG activation patterns. However, this approach severely limits the range of activities users can perform with the prostheses during their daily living. In this work, we introduce a novel method, based on a long short-term memory (LSTM) network, to continuously map forearm EMG activity onto hand kinematics. Critically, unlike previous work, which often focuses on simple and highly controlled motor tasks, we tested our method on a dataset of activities of daily living (ADLs): the KIN-MUS UJI dataset. To the best of our knowledge, ours is the first reported work on the prediction of hand kinematics that uses this challenging dataset. Remarkably, we show that our network is able to generalize to novel untrained ADLs. Our results suggest that the presented method is suitable for the generation of control signals for the independent and proportional actuation of the multiple DOFs of state-of-the-art hand prostheses. △ Less

Submitted 28 April, 2021; originally announced April 2021.

Comments: Accepted for publication in the Proceedings of the 2021 IEEE International Joint Conference on Neural Networks (IJCNN 2021)

Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data

Authors: Alessandro Salatiello, Martin A. Giese

Abstract: Recurrent Neural Networks (RNNs) are popular models of brain function. The typical training strategy is to adjust their input-output behavior so that it matches that of the biological circuit of interest. Even though this strategy ensures that the biological and artificial networks perform the same computational task, it does not guarantee that their internal activity dynamics match. This suggests… ▽ More Recurrent Neural Networks (RNNs) are popular models of brain function. The typical training strategy is to adjust their input-output behavior so that it matches that of the biological circuit of interest. Even though this strategy ensures that the biological and artificial networks perform the same computational task, it does not guarantee that their internal activity dynamics match. This suggests that the trained RNNs might end up performing the task employing a different internal computational mechanism, which would make them a suboptimal model of the biological circuit. In this work, we introduce a novel training strategy that allows learning not only the input-output behavior of an RNN but also its internal network dynamics, based on sparse neural recordings. We test the proposed method by training an RNN to simultaneously reproduce internal dynamics and output signals of a physiologically-inspired neural model. Specifically, this model generates the multiphasic muscle-like activity patterns typically observed during the execution of reaching movements, based on the oscillatory activation patterns concurrently observed in the motor cortex. Remarkably, we show that the reproduction of the internal dynamics is successful even when the training algorithm relies on the activities of a small subset of neurons sampled from the biological network. Furthermore, we show that training the RNNs with this method significantly improves their generalization performance. Overall, our results suggest that the proposed method is suitable for building powerful functional RNN models, which automatically capture important computational properties of the biological circuit of interest from sparse neural recordings. △ Less

Submitted 5 May, 2020; originally announced May 2020.

Journal ref: Artificial Neural Networks and Machine Learning - ICANN 2020. ICANN 2020. Lecture Notes in Computer Science, vol 12396. Springer, Cham.:874-86

A Unifying Framework for the Identification of Motor Primitives

Authors: Enrico Chiovetto, Andrea d'Avella, Martin Giese

Abstract: A long-standing hypothesis in neuroscience is that the central nervous system accomplishes complex motor behaviors through the combination of a small number of motor primitives. Many studies in the last couples of decades have identified motor primitives at the kinematic, kinetic, and electromyographic level, thus supporting modularity at different levels of organization in the motor system. Howev… ▽ More A long-standing hypothesis in neuroscience is that the central nervous system accomplishes complex motor behaviors through the combination of a small number of motor primitives. Many studies in the last couples of decades have identified motor primitives at the kinematic, kinetic, and electromyographic level, thus supporting modularity at different levels of organization in the motor system. However, these studies relied on heterogeneous definitions of motor primitives and on different algorithms for their identification. Standard unsupervised learning algorithms such as principal component analysis, independent component analysis, and non-negative matrix factorization, or more advanced techniques involving the estimation of temporal delays of the relevant mixture components have been applied. This plurality of algorithms has made difficult to compare and interpret results obtained across different studies. Moreover, how the different definitions of motor primitives relate to each other has never been examined systematically. Here we propose a comprehensive framework for the definition of different types of motor primitives and a single algorithm for their identification. By embedding smoothness priors and specific constraints in the underlying generative model, the algorithm can identify many different types of motor primitives. We assessed the identification performance of the algorithm both on simulated data sets, for which the properties of the primitives and of the corresponding combination parameters were known, and on experimental electromyographic and kinematic data sets, collected from human subjects accomplishing goal-oriented and rhythmic motor tasks. The identification accuracy of the new algorithm was typically equal or better than the accuracy of other unsupervised learning algorithms used previously for the identification of the same types of primitives. △ Less

Submitted 22 March, 2016; originally announced March 2016.

Comments: 33 pages, 8 figures, 1 table