Showing 1–26 of 26 results for author: Grill, J

Step** on the Edge: Curvature Aware Learning Rate Tuners

Authors: Vincent Roulet, Atish Agarwala, Jean-Bastien Grill, Grzegorz Swirszcz, Mathieu Blondel, Fabian Pedregosa

Abstract: Curvature information -- particularly, the largest eigenvalue of the loss Hessian, known as the sharpness -- often forms the basis for learning rate tuners. However, recent work has shown that the curvature information undergoes complex dynamics during training, going from a phase of increasing sharpness to eventual stabilization. We analyze the closed-loop feedback effect between learning rate tu… ▽ More Curvature information -- particularly, the largest eigenvalue of the loss Hessian, known as the sharpness -- often forms the basis for learning rate tuners. However, recent work has shown that the curvature information undergoes complex dynamics during training, going from a phase of increasing sharpness to eventual stabilization. We analyze the closed-loop feedback effect between learning rate tuning and curvature. We find that classical learning rate tuners may yield greater one-step loss reduction, yet they ultimately underperform in the long term when compared to constant learning rates in the full batch regime. These models break the stabilization of the sharpness, which we explain using a simplified model of the joint dynamics of the learning rate and the curvature. To further investigate these effects, we introduce a new learning rate tuning method, Curvature Dynamics Aware Tuning (CDAT), which prioritizes long term curvature stabilization over instantaneous progress on the objective. In the full batch regime, CDAT shows behavior akin to prefixed warm-up schedules on deep learning objectives, outperforming tuned constant learning rates. In the mini batch regime, we observe that stochasticity introduces confounding effects that explain the previous success of some learning rate tuners at appropriate batch sizes. Our findings highlight the critical role of understanding the joint dynamics of the learning rate and curvature, beyond greedy minimization, to diagnose failures and design effective adaptive learning rate tuners. △ Less

Submitted 8 July, 2024; originally announced July 2024.

In silico high-resolution whole lung model to predict the locally delivered dose of inhaled drugs

Authors: Maximilian J. Grill, Jonas Biehler, Karl-Robert Wichmann, David Rudlstorfer, Maximilian Rixner, Marie Brei, Jakob Richter, Joshua Bügel, Nina Pischke, Wolfgang A. Wall, Kei W. Müller

Abstract: The big crux with drug delivery to human lungs is that the delivered dose at the local site of action is unpredictable and very difficult to measure, even a posteriori. It is highly subject-specific as it depends on lung morphology, disease, breathing, and aerosol characteristics. Given these challenges, computational approaches have shown potential, but have so far failed due to fundamental metho… ▽ More The big crux with drug delivery to human lungs is that the delivered dose at the local site of action is unpredictable and very difficult to measure, even a posteriori. It is highly subject-specific as it depends on lung morphology, disease, breathing, and aerosol characteristics. Given these challenges, computational approaches have shown potential, but have so far failed due to fundamental methodical limitations. We present and validate a novel in silico model that enables the subject-specific prediction of local aerosol deposition throughout the entire lung. Its unprecedented spatiotemporal resolution allows to track each aerosol particle anytime during the breathing cycle, anywhere in the complete system of conducting airways and the alveolar region. Predictions are shown to be in excellent agreement with in vivo SPECT/CT data for a healthy human cohort. We further showcase the model's capabilities to represent strong heterogeneities in diseased lungs by studying an IPF patient. Finally, high computational efficiency and automated model generation and calibration ensure readiness to be applied at scale. We envision our method not only to improve inhalation therapies by informing and accelerating all stages of (pre-)clinical drug and device development, but also as a more-than-equivalent alternative to nuclear imaging of the lungs. △ Less

Submitted 11 July, 2023; v1 submitted 7 July, 2023; originally announced July 2023.

Emergent Communication: Generalization and Overfitting in Lewis Games

Authors: Mathieu Rita, Corentin Tallec, Paul Michel, Jean-Bastien Grill, Olivier Pietquin, Emmanuel Dupoux, Florian Strub

Abstract: Lewis signaling games are a class of simple communication games for simulating the emergence of language. In these games, two agents must agree on a communication protocol in order to solve a cooperative task. Previous work has shown that agents trained to play this game with reinforcement learning tend to develop languages that display undesirable properties from a linguistic point of view (lack… ▽ More Lewis signaling games are a class of simple communication games for simulating the emergence of language. In these games, two agents must agree on a communication protocol in order to solve a cooperative task. Previous work has shown that agents trained to play this game with reinforcement learning tend to develop languages that display undesirable properties from a linguistic point of view (lack of generalization, lack of compositionality, etc). In this paper, we aim to provide better understanding of this phenomenon by analytically studying the learning problem in Lewis games. As a core contribution, we demonstrate that the standard objective in Lewis games can be decomposed in two components: a co-adaptation loss and an information loss. This decomposition enables us to surface two potential sources of overfitting, which we show may undermine the emergence of a structured communication protocol. In particular, when we control for overfitting on the co-adaptation loss, we recover desired properties in the emergent languages: they are more compositional and generalize better. △ Less

Submitted 15 October, 2022; v1 submitted 30 September, 2022; originally announced September 2022.

Comments: 36th Conference on Neural Information Processing Systems (NeurIPS 2022)

Frameworks for Estimating Causal Effects in Observational Settings: Comparing Confounder Adjustment and Instrumental Variables

Authors: Roy S. Zawadzki, Joshua D. Grill, Daniel L. Gillen

Abstract: To estimate causal effects, analysts performing observational studies in health settings utilize several strategies to mitigate bias due to confounding by indication. There are two broad classes of approaches for these purposes: use of confounders and instrumental variables (IVs). Because such approaches are largely characterized by untestable assumptions, analysts must operate under an indefinite… ▽ More To estimate causal effects, analysts performing observational studies in health settings utilize several strategies to mitigate bias due to confounding by indication. There are two broad classes of approaches for these purposes: use of confounders and instrumental variables (IVs). Because such approaches are largely characterized by untestable assumptions, analysts must operate under an indefinite paradigm that these methods will work imperfectly. In this tutorial, we formalize a set of general principles and heuristics for estimating causal effects in the two approaches when the assumptions are potentially violated. This crucially requires reframing the process of observational studies as hypothesizing potential scenarios where the estimates from one approach are less inconsistent than the other. While most of our discussion of methodology centers around the linear setting, we touch upon complexities in non-linear settings and flexible procedures such as target minimum loss-based estimation (TMLE) and double machine learning (DML). To demonstrate the application of our principles, we investigate the use of donepezil off-label for mild cognitive impairment (MCI). We compare and contrast results from confounder and IV methods, traditional and flexible, within our analysis and to a similar observational study and clinical trial. △ Less

Submitted 27 April, 2023; v1 submitted 14 September, 2022; originally announced September 2022.

Asymptotically consistent and computationally efficient modeling of short-ranged molecular interactions between curved slender fibers undergoing large 3D deformations

Authors: Maximilian J. Grill, Wolfgang A. Wall, Christoph Meier

Abstract: This article proposes a novel computational modeling approach for short-ranged molecular interactions between curved slender fibers undergoing large 3D deformations, and gives a detailed overview how it fits into the framework of existing fiber or beam interaction models, either considering microscale molecular or macroscale contact effects. The direct evaluation of a molecular interaction potenti… ▽ More This article proposes a novel computational modeling approach for short-ranged molecular interactions between curved slender fibers undergoing large 3D deformations, and gives a detailed overview how it fits into the framework of existing fiber or beam interaction models, either considering microscale molecular or macroscale contact effects. The direct evaluation of a molecular interaction potential between two general bodies in 3D space would require to integrate molecule densities over two 3D volumes, leading to a sixfold integral to be solved numerically. By exploiting the short-range nature of the considered class of interaction potentials as well as the fundamental kinematic assumption of undeformable fiber cross-sections, as typically applied in mechanical beam theories, a recently derived, closed-form analytical solution is applied for the interaction potential between a given section of the first fiber (slave beam) and the entire second fiber (master beam). This novel approach based on a pre-defined section-beam interaction potential (SBIP) requires only one single integration step along the slave beam length to be performed numerically. In terms of accuracy, the total beam-beam interaction potential resulting from this approach is shown to exhibit an asymptotically consistent angular and distance scaling behavior. In addition to elementary two-fiber systems, carefully chosen to verify accuracy and asymptotic consistence of the proposed SBIP approach, a potential practical application in form of adhesive nanofiber-grafted surfaces is studied. Involving a large number of helicoidal fibers undergoing large 3D deformations, arbitrary mutual fiber orientations as well as frequent local fiber pull-off and snap-into-contact events, this example demonstrates the robustness and computational efficiency of the new approach. △ Less

Submitted 18 August, 2022; v1 submitted 5 August, 2022; originally announced August 2022.

Analytical disk-cylinder interaction potential laws for the computational modeling of adhesive, deformable (nano)fibers

Authors: Maximilian J. Grill, Wolfgang A. Wall, Christoph Meier

Abstract: The analysis of complex fibrous systems or materials on the micro- and nanoscale, which have a high practical relevance for many technical or biological systems, requires accurate analytical descriptions of the adhesive and repulsive forces acting on the fiber surfaces. While such analytical expressions are generally needed both for theoretical studies and for computer-based simulations, the latte… ▽ More The analysis of complex fibrous systems or materials on the micro- and nanoscale, which have a high practical relevance for many technical or biological systems, requires accurate analytical descriptions of the adhesive and repulsive forces acting on the fiber surfaces. While such analytical expressions are generally needed both for theoretical studies and for computer-based simulations, the latter motivates us here to derive disk-cylinder interaction potential laws that are valid for arbitrary mutual orientations in the decisive regime of small surface separations. The chosen type of fundamental point-pair interaction follows the simple Lennard-Jones model with inverse power laws for both the adhesive van der Waals part and the steric, repulsive part. We present three different solutions, ranging from highest accuracy to the best trade-off between simplicity of the expression and sufficient accuracy for our intended use. The validity of simplifying approximations and the accuracy of the derived potential laws is thoroughly analyzed, using both numerical and analytical reference solutions for specific interaction cases. Most importantly, the correct asymptotic scaling behavior in the decisive regime of small separations is achieved, and also the theoretically predicted $(1\!/\!\sin\!α)$-angle dependence (for non-parallel cylinders) is obtained by the proposed analytical solutions. As we show in the outlook to our current research, the derived analytical disk-cylinder interaction potential laws may be used to formulate highly efficient computational models for the interaction of arbitrarily curved fibers, such that the disk represents the cross-section of the first and the cylinder a local approximation to the shape of the second fiber. △ Less

Submitted 18 August, 2022; v1 submitted 5 August, 2022; originally announced August 2022.

BYOL-Explore: Exploration by Bootstrapped Prediction

Authors: Zhaohan Daniel Guo, Shantanu Thakoor, Miruna Pîslar, Bernardo Avila Pires, Florent Altché, Corentin Tallec, Alaa Saade, Daniele Calandriello, Jean-Bastien Grill, Yunhao Tang, Michal Valko, Rémi Munos, Mohammad Gheshlaghi Azar, Bilal Piot

Abstract: We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, the world dynamics, and an exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challeng… ▽ More We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, the world dynamics, and an exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually-rich 3-D environments. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore s intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents. △ Less

Submitted 16 June, 2022; originally announced June 2022.

On the role of population heterogeneity in emergent communication

Authors: Mathieu Rita, Florian Strub, Jean-Bastien Grill, Olivier Pietquin, Emmanuel Dupoux

Abstract: Populations have often been perceived as a structuring component for language to emerge and evolve: the larger the population, the more structured the language. While this observation is widespread in the sociolinguistic literature, it has not been consistently reproduced in computer simulations with neural agents. In this paper, we thus aim to clarify this apparent contradiction. We explore emerg… ▽ More Populations have often been perceived as a structuring component for language to emerge and evolve: the larger the population, the more structured the language. While this observation is widespread in the sociolinguistic literature, it has not been consistently reproduced in computer simulations with neural agents. In this paper, we thus aim to clarify this apparent contradiction. We explore emergent language properties by varying agent population size in the speaker-listener Lewis Game. After reproducing the experimental difference, we challenge the simulation assumption that the agent community is homogeneous. We first investigate how speaker-listener asymmetry alters language structure to examine two potential diversity factors: training speed and network capacity. We find out that emergent language properties are only altered by the relative difference of learning speeds between speaker and listener, and not by their absolute values. From then, we leverage this observation to control population heterogeneity without introducing confounding factors. We finally show that introducing such training speed heterogeneities naturally sort out the initial contradiction: larger simulated communities start develo** more stable and structured languages. △ Less

Submitted 27 April, 2022; originally announced April 2022.

Comments: International Conference on Learning Representations (ICLR) 2022

Generalized Section-Section Interaction Potentials in the Geometrically Exact Beam Theory: Modeling of Intermolecular Forces, Asymptotic Limit as Strain-Energy Function, and Formulation of Rotational Constraints

Authors: Christoph Meier, Maximilian J. Grill, Wolfgang A. Wall

Abstract: The present contribution proposes a universal framework to formulate generalized section-section interaction potentials (SSIP) within the geometrically exact beam theory. By exploiting the fundamental kinematic assumption of undeformable cross-sections, an objective (i.e., frame-invariant) description of SSIPs via a minimal set of six (translational and rotational) relative coordinates, either in… ▽ More The present contribution proposes a universal framework to formulate generalized section-section interaction potentials (SSIP) within the geometrically exact beam theory. By exploiting the fundamental kinematic assumption of undeformable cross-sections, an objective (i.e., frame-invariant) description of SSIPs via a minimal set of six (translational and rotational) relative coordinates, either in spatial or in material form, is proposed. Based on work-pairing, work-conjugated section-section interaction forces and moments, either in spatial or in material form, are identified that can be consistently derived from a variational principle. Interestingly, it is shown that hyperelastic stored-energy functions relating the deformation measures and stress-resultants of the well-known geometrically exact Simo-Reissner beam theory can also be identified as SSIPs when considering the asymptotic limit of small relative distances and rotations between the interacting cross-sections. Moreover, the proposed variational problem formulation is demonstrated to be of a very general nature, thus allowing for the formulation of translational and rotational constraints between arbitrarily oriented cross-sections based on either a penalty or a Lagrange multiplier potential. Possible applications include fiber-based structures and materials in technical and biological systems, where the proposed approach allows to model short- or long-ranged inter-molecular (e.g., electrostatic, van der Waals or repulsive steric) interactions between fibers in geometrically complex arrangements and to formulate translational and rotational coupling constraints between different fibers (e.g., cross-linked polymer chains) or between fibers and a matrix phase (e.g., fiber-reinforced composites). △ Less

Submitted 17 August, 2022; v1 submitted 20 May, 2021; originally announced May 2021.

Adjustment for Biased Sampling Using NHANES Derived Propensity Weights

Authors: Olivia M. Bernstein, Brian G. Vegetabile, Christian R. Salazar, Joshua D. Grill, Daniel L. Gillen

Abstract: The Consent-to-Contact (C2C) registry at the University of California, Irvine collects data from community participants to aid in the recruitment to clinical research studies. Self-selection into the C2C likely leads to bias due in part to enrollees having more years of education relative to the US general population. Salazar et al. (2020) recently used the C2C to examine associations of race/ethn… ▽ More The Consent-to-Contact (C2C) registry at the University of California, Irvine collects data from community participants to aid in the recruitment to clinical research studies. Self-selection into the C2C likely leads to bias due in part to enrollees having more years of education relative to the US general population. Salazar et al. (2020) recently used the C2C to examine associations of race/ethnicity with participant willingness to be contacted about research studies. To address questions about generalizability of estimated associations we estimate propensity for self-selection into the convenience sample weights using data from the National Health and Nutrition Examination Survey (NHANES). We create a combined dataset of C2C and NHANES subjects and compare different approaches (logistic regression, covariate balancing propensity score, entropy balancing, and random forest) for estimating the probability of membership in C2C relative to NHANES. We propose methods to estimate the variance of parameter estimates that account for uncertainty that arises from estimating propensity weights. Simulation studies explore the impact of propensity weight estimation on uncertainty. We demonstrate the approach by repeating the analysis by Salazar et al. with the deduced propensity weights for the C2C subjects and contrast the results of the two analyses. This method can be implemented using our estweight package in R available on GitHub. △ Less

Submitted 20 April, 2021; originally announced April 2021.

Comments: 20 pages, 4 figures, 1 table. Health Serv Outcomes Res Method (2022)

Broaden Your Views for Self-Supervised Video Learning

Authors: Adrià Recasens, Pauline Luc, Jean-Baptiste Alayrac, Luyu Wang, Ross Hemsley, Florian Strub, Corentin Tallec, Mateusz Malinowski, Viorica Patraucean, Florent Altché, Michal Valko, Jean-Bastien Grill, Aäron van den Oord, Andrew Zisserman

Abstract: Most successful self-supervised learning methods are trained to align the representations of two independent views from the data. State-of-the-art methods in video are inspired by image techniques, where these two views are similarly extracted by crop** and augmenting the resulting crop. However, these methods miss a crucial element in the video domain: time. We introduce BraVe, a self-supervise… ▽ More Most successful self-supervised learning methods are trained to align the representations of two independent views from the data. State-of-the-art methods in video are inspired by image techniques, where these two views are similarly extracted by crop** and augmenting the resulting crop. However, these methods miss a crucial element in the video domain: time. We introduce BraVe, a self-supervised learning framework for video. In BraVe, one of the views has access to a narrow temporal window of the video while the other view has a broad access to the video content. Our models learn to generalise from the narrow view to the general content of the video. Furthermore, BraVe processes the views with different backbones, enabling the use of alternative augmentations or modalities into the broad view such as optical flow, randomly convolved RGB frames, audio or their combinations. We demonstrate that BraVe achieves state-of-the-art results in self-supervised representation learning on standard video and audio classification benchmarks including UCF101, HMDB51, Kinetics, ESC-50 and AudioSet. △ Less

Submitted 19 October, 2021; v1 submitted 30 March, 2021; originally announced March 2021.

Comments: This paper is an extended version of our ICCV-21 paper. It includes more results as well as a minor architectural variation which improves results

A computational framework for modeling cell-matrix interactions in soft biological tissues

Authors: Jonas F. Eichinger, Maximilian J. Grill, Iman Davoodi Kermani, Roland C. Aydin, Wolfgang A. Wall, Jay D. Humphrey, Christian J. Cyron

Abstract: Living soft tissues appear to promote the development and maintenance of a preferred mechanical state within a defined tolerance around a so-called set-point. This phenomenon is often referred to as mechanical homeostasis. In contradiction to the prominent role of mechanical homeostasis in various (patho)physiological processes, its underlying micromechanical mechanisms acting on the level of indi… ▽ More Living soft tissues appear to promote the development and maintenance of a preferred mechanical state within a defined tolerance around a so-called set-point. This phenomenon is often referred to as mechanical homeostasis. In contradiction to the prominent role of mechanical homeostasis in various (patho)physiological processes, its underlying micromechanical mechanisms acting on the level of individual cells and fibers remain poorly understood, especially, how these mechanisms on the microscale lead to what we macroscopically call mechanical homeostasis. Here, we present a novel finite element based computational framework that is constructed bottom up, that is, it models key mechanobiological mechanisms such as actin cytoskeleton contraction and molecular clutch behavior of individual cells interacting with a reconstructed three-dimensional extracellular fiber matrix. The framework reproduces many experimental observations regarding mechanical homeostasis on short time scales (hours), in which the deposition and degradation of extracellular matrix can largely be neglected. This model can serve as a systematic tool for future in silico studies of the origin of the numerous still unexplained experimental observations about mechanical homeostasis. △ Less

Submitted 24 March, 2021; originally announced March 2021.

A Novel Modeling and Simulation Approach for the Hindered Mobility of Charged Particles in Biological Hydrogels

Authors: Maximilian J. Grill, Jonas F. Eichinger, Jonas Koban, Christoph Meier, Oliver Lieleg, Wolfgang A. Wall

Abstract: This article presents a novel computational model to study the selective filtering of biological hydrogels due to the surface charge and size of diffusing particles. It is the first model that includes the random 3D fiber orientation and connectivity of the biopolymer network and that accounts for elastic deformations of the fibers by means of beam theory. As a key component of the model, novel fo… ▽ More This article presents a novel computational model to study the selective filtering of biological hydrogels due to the surface charge and size of diffusing particles. It is the first model that includes the random 3D fiber orientation and connectivity of the biopolymer network and that accounts for elastic deformations of the fibers by means of beam theory. As a key component of the model, novel formulations are proposed both for the electrostatic and repulsive steric interactions between a spherical particle and a beam. In addition to providing a thorough validation of the model, the presented computational studies yield new insights into the underlying mechanisms of hindered particle mobility, especially regarding the influence of the aforementioned aspects that are unique to this model. It is found that the precise distribution of fiber and thus charge agglomerations in the network have a crucial influence on the mobility of oppositely charged particles and gives rise to distinct motion patterns. Considering the high practical significance for instance with respect to targeted drug release or infection defense, the provided proof of concept motivates further advances of the model toward a truly predictive computational tool that allows a case- and patient-specific assessment for real (biological) systems. △ Less

Submitted 17 January, 2021; originally announced January 2021.

Disordered biopolymer filament bundles: Topological defects and kinks

Authors: Valentin M. Slepukhin, Maximilian J. Grill, Qingda Hu, Elliot L. Botvinick, Wolfgang A. Wall, Alex J. Levine

Abstract: Bundles of stiff filaments are ubiquitous in the living world, found both in the cytoskeleton and in the extracellular medium. These bundles are typically held together by smaller cross-linking molecules. We demonstrate analytically, numerically and experimentally that such bundles can be kinked, i.e., have localized regions of high curvature that are long-lived metastable states. We propose three… ▽ More Bundles of stiff filaments are ubiquitous in the living world, found both in the cytoskeleton and in the extracellular medium. These bundles are typically held together by smaller cross-linking molecules. We demonstrate analytically, numerically and experimentally that such bundles can be kinked, i.e., have localized regions of high curvature that are long-lived metastable states. We propose three possible mechanisms of kink stabilization: a difference in trapped length of the filament segments between two cross links; a dislocation where the endpoint of a filament occurs within the bundle, and the braiding of the filaments in the bundle. At a high concentration of cross links, the last two effects lead to the topologically protected kinked states. Finally, we explore numerically and analytically the transition of the metastable kinked state to the stable straight bundle. △ Less

Submitted 23 November, 2020; originally announced November 2020.

Comments: 9 pages, 6 figures

BYOL works even without batch statistics

Authors: Pierre H. Richemond, Jean-Bastien Grill, Florent Altché, Corentin Tallec, Florian Strub, Andrew Brock, Samuel Smith, Soham De, Razvan Pascanu, Bilal Piot, Michal Valko

Abstract: Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach for image representation. From an augmented view of an image, BYOL trains an online network to predict a target network representation of a different augmented view of the same image. Unlike contrastive methods, BYOL does not explicitly use a repulsion term built from negative pairs in its training objective. Yet, it avoids co… ▽ More Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach for image representation. From an augmented view of an image, BYOL trains an online network to predict a target network representation of a different augmented view of the same image. Unlike contrastive methods, BYOL does not explicitly use a repulsion term built from negative pairs in its training objective. Yet, it avoids collapse to a trivial, constant representation. Thus, it has recently been hypothesized that batch normalization (BN) is critical to prevent collapse in BYOL. Indeed, BN flows gradients across batch elements, and could leak information about negative views in the batch, which could act as an implicit negative (contrastive) term. However, we experimentally show that replacing BN with a batch-independent normalization scheme (namely, a combination of group normalization and weight standardization) achieves performance comparable to vanilla BYOL ($73.9\%$ vs. $74.3\%$ top-1 accuracy under the linear evaluation protocol on ImageNet with ResNet-$50$). Our finding disproves the hypothesis that the use of batch statistics is a crucial ingredient for BYOL to learn useful representations. △ Less

Submitted 20 October, 2020; originally announced October 2020.

Monte-Carlo Tree Search as Regularized Policy Optimization

Authors: Jean-Bastien Grill, Florent Altché, Yunhao Tang, Thomas Hubert, Michal Valko, Ioannis Antonoglou, Rémi Munos

Abstract: The combination of Monte-Carlo tree search (MCTS) with deep reinforcement learning has led to significant advances in artificial intelligence. However, AlphaZero, the current state-of-the-art MCTS algorithm, still relies on handcrafted heuristics that are only partially understood. In this paper, we show that AlphaZero's search heuristics, along with other common ones such as UCT, are an approxima… ▽ More The combination of Monte-Carlo tree search (MCTS) with deep reinforcement learning has led to significant advances in artificial intelligence. However, AlphaZero, the current state-of-the-art MCTS algorithm, still relies on handcrafted heuristics that are only partially understood. In this paper, we show that AlphaZero's search heuristics, along with other common ones such as UCT, are an approximation to the solution of a specific regularized policy optimization problem. With this insight, we propose a variant of AlphaZero which uses the exact solution to this policy optimization problem, and show experimentally that it reliably outperforms the original algorithm in multiple domains. △ Less

Submitted 24 July, 2020; originally announced July 2020.

Comments: Accepted to International Conference on Machine Learning (ICML), 2020

Directed force propagation in semiflexible networks

Authors: Maximilian J. Grill, Jonathan Kernes, Valentin M. Slepukhin, Wolfgang A. Wall, Alex J. Levine

Abstract: We consider the propagation of tension along specific filament of a semiflexible filament network in response to the application of a point force using a combination of numerical simulations and analytic theory. We find the distribution of force within the network is highly heterogenous, with a small number of fibers supporting a significant fraction of the applied load over distances of multiple… ▽ More We consider the propagation of tension along specific filament of a semiflexible filament network in response to the application of a point force using a combination of numerical simulations and analytic theory. We find the distribution of force within the network is highly heterogenous, with a small number of fibers supporting a significant fraction of the applied load over distances of multiple mesh sizes surrounding the point of force application. We suggest that these structures may be thought of as tensile force chains, whose structure we explore via simulation. We develop self-consistent calculations of the point-force response function and introduce a transfer matrix approach to explore the decay of tension (into bending) energy and the branching of tensile force chains in the network. △ Less

Submitted 26 June, 2020; originally announced June 2020.

Comments: 17 pages, 13 figures

Bootstrap your own latent: A new approach to self-supervised Learning

Authors: Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko

Abstract: We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the… ▽ More We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches $74.3\%$ top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and $79.6\%$ with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub. △ Less

Submitted 10 September, 2020; v1 submitted 13 June, 2020; originally announced June 2020.

Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning

Authors: Daniel Guo, Bernardo Avila Pires, Bilal Piot, Jean-bastien Grill, Florent Altché, Rémi Munos, Mohammad Gheshlaghi Azar

Abstract: Learning a good representation is an essential component for deep reinforcement learning (RL). Representation learning is especially important in multitask and partially observable settings where building a representation of the unknown environment is crucial to solve the tasks. Here we introduce Prediction of Bootstrap Latents (PBL), a simple and flexible self-supervised representation learning a… ▽ More Learning a good representation is an essential component for deep reinforcement learning (RL). Representation learning is especially important in multitask and partially observable settings where building a representation of the unknown environment is crucial to solve the tasks. Here we introduce Prediction of Bootstrap Latents (PBL), a simple and flexible self-supervised representation learning algorithm for multitask deep RL. PBL builds on multistep predictive representations of future observations, and focuses on capturing structured information about environment dynamics. Specifically, PBL trains its representation by predicting latent embeddings of future observations. These latent embeddings are themselves trained to be predictive of the aforementioned representations. These predictions form a bootstrap** effect, allowing the agent to learn more about the key aspects of the environment dynamics. In addition, by defining prediction tasks completely in latent space, PBL provides the flexibility of using multimodal observations involving pixel images, language instructions, rewards and more. We show in our experiments that PBL delivers across-the-board improved performance over state of the art deep RL agents in the DMLab-30 and Atari-57 multitask setting. △ Less

Submitted 30 April, 2020; originally announced April 2020.

A mortar-type finite element approach for embedding 1D beams into 3D solid volumes

Authors: Ivo Steinbrecher, Matthias Mayr, Maximilian J. Grill, Johannes Kremheller, Christoph Meier, Alexander Popp

Abstract: In this work we present a novel computational method for embedding arbitrary curved one-dimensional (1D) fibers into three-dimensional (3D) solid volumes, as e.g. in fiber-reinforced materials. The fibers are explicitly modeled with highly efficient 1D geometrically exact beam finite elements, based on various types of geometrically nonlinear beam theories. The surrounding solid volume is modeled… ▽ More In this work we present a novel computational method for embedding arbitrary curved one-dimensional (1D) fibers into three-dimensional (3D) solid volumes, as e.g. in fiber-reinforced materials. The fibers are explicitly modeled with highly efficient 1D geometrically exact beam finite elements, based on various types of geometrically nonlinear beam theories. The surrounding solid volume is modeled with 3D continuum (solid) elements. An embedded mortar-type approach is employed to enforce the kinematic coupling constraints between the beam elements and solid elements on non-matching meshes. This allows for very flexible mesh generation and simple material modeling procedures in the solid, since it can be discretized without having to capture for the reinforcements, while still being able to account for complex nonlinear effects due to the embedded fibers. Several numerical examples demonstrate the consistency, robustness and accuracy of the proposed method, as well as its applicability to rather complex fiber-reinforced structures of practical relevance. △ Less

Submitted 9 December, 2019; originally announced December 2019.

Investigation of the Peeling and Pull-off Behavior of Adhesive Elastic Fibers via a Novel Computational Beam Interaction Model

Authors: Maximilian J. Grill, Christoph Meier, Wolfgang A. Wall

Abstract: This article studies the fundamental problem of separating two adhesive elastic fibers based on numerical simulation employing a recently developed finite element model for molecular interactions between curved slender fibers. Specifically, it covers the two-sided peeling and pull-off process starting from fibers contacting along its entire length to fully separated fibers including all intermedia… ▽ More This article studies the fundamental problem of separating two adhesive elastic fibers based on numerical simulation employing a recently developed finite element model for molecular interactions between curved slender fibers. Specifically, it covers the two-sided peeling and pull-off process starting from fibers contacting along its entire length to fully separated fibers including all intermediate configurations and the well-known physical instability of snap** into contact and snap** free. We analyze the resulting force-displacement curve showing a rich and highly nonlinear system behavior arising from the interplay of adhesion, mechanical contact interaction and structural resistance against (axial, shear and bending) deformation. While similar to one-sided peeling studies from the literature, a distinct initiation and peeling phase can be observed, the two-sided peeling setup considered in the present work reveals the extended final pull-off stage as third characteristic phase. Moreover, the influence of different material and interaction parameters such as Young's modulus as well as type (electrostatic or van der Waals) and strength of adhesion is critically studied. Most importantly, it is found that the maximum force occurs in the pull-off phase for electrostatic attraction, but in the initiation phase for van der Waals adhesion. In addition to the physical system behavior, the most important numerical aspects required to simulate this challenging computational problem in a robust and accurate manner are discussed. Thus, besides the insights gained into the considered two-fiber system, this study provides a proof of concept facilitating the application of the employed model to larger and increasingly complex systems of slender fibers. △ Less

Submitted 31 July, 2019; v1 submitted 30 July, 2019; originally announced July 2019.

A Computational Model for Molecular Interactions Between Curved Slender Fibers Undergoing Large 3D Deformations With a Focus on Electrostatic, van der Waals and Repulsive Steric Forces

Authors: Maximilian J. Grill, Wolfgang A. Wall, Christoph Meier

Abstract: This contribution proposes the first 3D beam-to-beam interaction model for molecular interactions between curved slender fibers undergoing large deformations. While the general model is not restricted to a specific beam formulation, in the present work it is combined with the geometrically exact beam theory and discretized via the finite element method. A direct evaluation of the total interaction… ▽ More This contribution proposes the first 3D beam-to-beam interaction model for molecular interactions between curved slender fibers undergoing large deformations. While the general model is not restricted to a specific beam formulation, in the present work it is combined with the geometrically exact beam theory and discretized via the finite element method. A direct evaluation of the total interaction potential for general 3D bodies requires the integration of contributions from molecule or charge distributions over the volumes of the interaction partners, leading to a 6D integral (two nested 3D integrals) that has to be solved numerically. Here, we propose a novel strategy to formulate reduced section-to-section interaction laws for the resultant interaction potential between a pair of cross-sections of two slender fibers such that only two 1D integrals along the fibers' length directions have to be solved numerically. This section-to-section interaction potential (SSIP) approach yields a significant gain in efficiency, which is essential to enable the simulation of relevant time and length scales for many practical applications. In a first step, the generic structure of SSIP laws, which is suitable for the most general interaction scenario (e.g. fibers with arbitrary cross-section shape and inhomogeneous atomic/charge density within the cross-section) is presented. Assuming circular, homogeneous cross-sections, in a next step, specific analytical expressions for SSIP laws describing short-range volume interactions (e.g. van der Waals or steric interactions) and long-range surface interactions (e.g. Coulomb interactions) are proposed. The validity of the SSIP laws as well as the accuracy and robustness of the general SSIP approach to beam-to-beam interactions is thoroughly verified by means of a set of numerical examples considering steric repulsion, electrostatic or van der Waals adhesion. △ Less

Submitted 31 July, 2019; v1 submitted 30 July, 2019; originally announced July 2019.

World Discovery Models

Authors: Mohammad Gheshlaghi Azar, Bilal Piot, Bernardo Avila Pires, Jean-Bastien Grill, Florent Altché, Rémi Munos

Abstract: As humans we are driven by a strong desire for seeking novelty in our world. Also upon observing a novel pattern we are capable of refining our understanding of the world based on the new information---humans can discover their world. The outstanding ability of the human mind for discovery has led to many breakthroughs in science, art and technology. Here we investigate the possibility of building… ▽ More As humans we are driven by a strong desire for seeking novelty in our world. Also upon observing a novel pattern we are capable of refining our understanding of the world based on the new information---humans can discover their world. The outstanding ability of the human mind for discovery has led to many breakthroughs in science, art and technology. Here we investigate the possibility of building an agent capable of discovering its world using the modern AI technology. In particular we introduce NDIGO, Neural Differential Information Gain Optimisation, a self-supervised discovery model that aims at seeking new information to construct a global view of its world from partial and noisy observations. Our experiments on some controlled 2-D navigation tasks show that NDIGO outperforms state-of-the-art information-seeking methods in terms of the quality of the learned representation. The improvement in performance is particularly significant in the presence of white or structured noise where other information-seeking methods follow the noise instead of discovering their world. △ Less

Submitted 1 March, 2019; v1 submitted 20 February, 2019; originally announced February 2019.

Optimistic optimization of a Brownian

Authors: Jean-Bastien Grill, Michal Valko, Rémi Munos

Abstract: We address the problem of optimizing a Brownian motion. We consider a (random) realization $W$ of a Brownian motion with input space in $[0,1]$. Given $W$, our goal is to return an $ε$-approximation of its maximum using the smallest possible number of function evaluations, the sample complexity of the algorithm. We provide an algorithm with sample complexity of order $\log^2(1/ε)$. This improves o… ▽ More We address the problem of optimizing a Brownian motion. We consider a (random) realization $W$ of a Brownian motion with input space in $[0,1]$. Given $W$, our goal is to return an $ε$-approximation of its maximum using the smallest possible number of function evaluations, the sample complexity of the algorithm. We provide an algorithm with sample complexity of order $\log^2(1/ε)$. This improves over previous results of Al-Mharmah and Calvin (1996) and Calvin et al. (2017) which provided only polynomial rates. Our algorithm is adaptive---each query depends on previous values---and is an instance of the optimism-in-the-face-of-uncertainty principle. △ Less

Submitted 15 January, 2019; originally announced January 2019.

Comments: 10 pages, 2 figures

Journal ref: Neural Information Processing Systems (NeurIPS 2018)

The conformation of a semiflexible filament in a quenched random potential

Authors: Valentin M. Slepukhin, Maximilian J. Grill, Kei W. Müller, Wolfgang A. Wall, Alex J. Levine

Abstract: Motivated by the observation of the storage of excess elastic free energy - (prestress) -- in cross linked semiflexible networks, we consider the problem of the conformational statistics of a single semiflexible polymer in a quenched random potential. The random potential, which represents the effect of cross linking to other filaments is assumed to have a finite correlation length $ξ$ and mean st… ▽ More Motivated by the observation of the storage of excess elastic free energy - (prestress) -- in cross linked semiflexible networks, we consider the problem of the conformational statistics of a single semiflexible polymer in a quenched random potential. The random potential, which represents the effect of cross linking to other filaments is assumed to have a finite correlation length $ξ$ and mean strength $V_{0}$. We examine statistical distribution of curvature in filament with thermal persistence length $\ell_{P}$ and length $L_0$ in the limit that $\ell_{P} \gg L_0$. We compare our theoretical predictions to finite element Brownian dynamics simulations. Lastly we comment on the validity of replica field techniques in addressing these questions. △ Less

Submitted 14 January, 2019; originally announced January 2019.

Journal ref: Phys. Rev. E 99, 042501 (2019)

Geometrically exact beam elements and smooth contact schemes for the modeling of fiber-based materials and structures

Authors: Christoph Meier, Maximilian J. Grill, Wolfgang A. Wall, Alexander Popp

Abstract: Recently, the authors have proposed a novel all-angle beam contact (ABC) formulation that combines the advantages of existing point and line contact models in a variationally consistent manner. However, the ABC formulation has so far only been applied in combination with a special torsion-free beam model, which yields a very simple and efficient finite element formulation, but which is restricted… ▽ More Recently, the authors have proposed a novel all-angle beam contact (ABC) formulation that combines the advantages of existing point and line contact models in a variationally consistent manner. However, the ABC formulation has so far only been applied in combination with a special torsion-free beam model, which yields a very simple and efficient finite element formulation, but which is restricted to initially straight beams with isotropic cross-sections. In order to abstain from these restrictions, the current work combines the ABC formulation with a geometrically exact Kirchhoff-Love beam element formulation that is capable of treating even the most general cases of slender beam problems in terms of initial geometry and external loads. While the neglect of shear deformation that is inherent to this formulation has been shown to provide considerable numerical advantages in the range of high beam slenderness ratios, alternative shear-deformable beam models are required for examples with thick beams. The current contribution additionally proposes a novel geometrically exact beam element based on the Simo-Reissner theory. Similar to the torsion-free and the Kirchhoff-Love beam elements, also this Simo-Reissner element is based on a C1-continuous Hermite interpolation of the beam centerline, which will allow for smooth contact kinematics. For this Hermitian Simo-Reissner element, a consistent spatial convergence behavior as well as the successful avoidance of membrane and shear locking will be demonstrated numerically. All in all, the combination of the ABC formulation with these different beam element variants (i.e.~the torsion-free element, the Kirchhoff-Love element and the Simo-Reissner element) results in a very flexible and modular simulation framework that allows to choose the optimal element formulation for any given application in terms of accuracy, efficiency and robustness. △ Less

Submitted 19 November, 2016; originally announced November 2016.