-
Domain-informed graph neural networks: a quantum chemistry case study
Authors:
Jay Morgan,
Adeline Paiement,
Christian Klinke
Abstract:
We explore different strategies to integrate prior domain knowledge into the design of a deep neural network (DNN). We focus on graph neural networks (GNN), with a use case of estimating the potential energy of chemical systems (molecules and crystals) represented as graphs. We integrate two elements of domain knowledge into the design of the GNN to constrain and regularise its learning, towards h…
▽ More
We explore different strategies to integrate prior domain knowledge into the design of a deep neural network (DNN). We focus on graph neural networks (GNN), with a use case of estimating the potential energy of chemical systems (molecules and crystals) represented as graphs. We integrate two elements of domain knowledge into the design of the GNN to constrain and regularise its learning, towards higher accuracy and generalisation. First, knowledge on the existence of different types of relations (chemical bonds) between atoms is used to modulate the interaction of nodes in the GNN. Second, knowledge of the relevance of some physical quantities is used to constrain the learnt features towards a higher physical relevance using a simple multi-task paradigm. We demonstrate the general applicability of our knowledge integrations by applying them to two architectures that rely on different mechanisms to propagate information between nodes and to update node states.
△ Less
Submitted 25 August, 2022;
originally announced August 2022.
-
Detecting Humans in RGB-D Data with CNNs
Authors:
Kaiyang Zhou,
Adeline Paiement,
Majid Mirmehdi
Abstract:
We address the problem of people detection in RGB-D data where we leverage depth information to develop a region-of-interest (ROI) selection method that provides proposals to two color and depth CNNs. To combine the detections produced by the two CNNs, we propose a novel fusion approach based on the characteristics of depth images. We also present a new depth-encoding scheme, which not only encode…
▽ More
We address the problem of people detection in RGB-D data where we leverage depth information to develop a region-of-interest (ROI) selection method that provides proposals to two color and depth CNNs. To combine the detections produced by the two CNNs, we propose a novel fusion approach based on the characteristics of depth images. We also present a new depth-encoding scheme, which not only encodes depth images into three channels but also enhances the information for classification. We conduct experiments on a publicly available RGB-D people dataset and show that our approach outperforms the baseline models that only use RGB data.
△ Less
Submitted 16 July, 2022;
originally announced July 2022.
-
A DNN Framework for Learning Lagrangian Drift With Uncertainty
Authors:
Joseph Jenkins,
Adeline Paiement,
Yann Ourmières,
Julien Le Sommer,
Jacques Verron,
Clément Ubelmann,
Hervé Glotin
Abstract:
Reconstructions of Lagrangian drift, for example for objects lost at sea, are often uncertain due to unresolved physical phenomena within the data. Uncertainty is usually overcome by introducing stochasticity into the drift, but this approach requires specific assumptions for modelling uncertainty. We remove this constraint by presenting a purely data-driven framework for modelling probabilistic d…
▽ More
Reconstructions of Lagrangian drift, for example for objects lost at sea, are often uncertain due to unresolved physical phenomena within the data. Uncertainty is usually overcome by introducing stochasticity into the drift, but this approach requires specific assumptions for modelling uncertainty. We remove this constraint by presenting a purely data-driven framework for modelling probabilistic drift in flexible environments. Using ocean circulation model simulations, we generate probabilistic trajectories of object location by simulating uncertainty in the initial object position. We train an emulator of probabilistic drift over one day given perfectly known velocities and observe good agreement with numerical simulations. Several loss functions are tested. Then, we strain our framework by training models where the input information is imperfect. On these harder scenarios, we observe reasonable predictions although the effects of data drift become noticeable when evaluating the models against unseen flow scenarios.
△ Less
Submitted 24 May, 2023; v1 submitted 12 April, 2022;
originally announced April 2022.
-
Characterization of Low Surface Brightness structures in annotated deep images
Authors:
Elisabeth Sola,
Pierre-Alain Duc,
Felix Richards,
Adeline Paiement,
Mathias Urbano,
Julie Klehammer,
Michal Bílek,
Jean-Charles Cuillandre,
Stephen Gwyn,
Alan McConnachie
Abstract:
The characterization of Low Surface Brightness (LSB) stellar structures around galaxies such as tidal debris of on-going or past collisions is essential to constrain models of galactic evolution. Our goal is to obtain quantitative measurements of LSB structures identified in deep images of samples consisting of hundreds of galaxies. We developed an online annotation tool that enables contributors…
▽ More
The characterization of Low Surface Brightness (LSB) stellar structures around galaxies such as tidal debris of on-going or past collisions is essential to constrain models of galactic evolution. Our goal is to obtain quantitative measurements of LSB structures identified in deep images of samples consisting of hundreds of galaxies. We developed an online annotation tool that enables contributors to delineate the shapes of diffuse extended stellar structures, as well as artefacts or foreground structures. All parameters are automatically stored in a database which may be queried to retrieve quantitative measurements. We annotated LSB structures around 352 nearby massive galaxies with deep images obtained with the CFHT as part of two large programs: MATLAS and UNIONS/CFIS. Each LSB structure was delineated and labeled according to its likely nature: stellar shells, streams associated to a disrupted satellite, tails formed in major mergers, ghost reflections or cirrus. From our database containing 8441 annotations, the area, size, median surface brightness and distance to the host of 228 structures were computed. The results confirm the fact that tidal structures defined as streams are thinner than tails, as expected by numerical simulations. In addition, tidal tails appear to exhibit a higher surface brightness than streams (by about 1 mag), which may be related to different survival times for the two types of collisional debris. We did not detect any tidal feature fainter than 27.5 mag.arcsec$^{-2}$, while the nominal surface brightness limits of our surveys range between 28.3 and 29 mag.arcsec$^{-2}$, a difference that needs to be taken into account when estimating the sensitivity of future surveys to identify LSB structures. Our annotation database of observed LSB structures may be used for quantitative analysis and as a training set for machine learning algorithms (abbreviated).
△ Less
Submitted 8 March, 2022;
originally announced March 2022.
-
Adaptive Neighbourhoods for the Discovery of Adversarial Examples
Authors:
Jay Morgan,
Adeline Paiement,
Arno Pauly,
Monika Seisenberger
Abstract:
Deep Neural Networks (DNNs) have often supplied state-of-the-art results in pattern recognition tasks. Despite their advances, however, the existence of adversarial examples have caught the attention of the community. Many existing works have proposed methods for searching for adversarial examples within fixed-sized regions around training points. Our work complements and improves these existing a…
▽ More
Deep Neural Networks (DNNs) have often supplied state-of-the-art results in pattern recognition tasks. Despite their advances, however, the existence of adversarial examples have caught the attention of the community. Many existing works have proposed methods for searching for adversarial examples within fixed-sized regions around training points. Our work complements and improves these existing approaches by adapting the size of these regions based on the problem complexity and data sampling density. This makes such approaches more appropriate for other types of data and may further improve adversarial training methods by increasing the region sizes without creating incorrect labels.
△ Less
Submitted 22 January, 2021;
originally announced January 2021.
-
Learnable Gabor modulated complex-valued networks for orientation robustness
Authors:
Felix Richards,
Adeline Paiement,
Xianghua Xie,
Elisabeth Sola,
Pierre-Alain Duc
Abstract:
Robustness to transformation is desirable in many computer vision tasks, given that input data often exhibits pose variance. While translation invariance and equivariance is a documented phenomenon of CNNs, sensitivity to other transformations is typically encouraged through data augmentation. We investigate the modulation of complex valued convolutional weights with learned Gabor filters to enabl…
▽ More
Robustness to transformation is desirable in many computer vision tasks, given that input data often exhibits pose variance. While translation invariance and equivariance is a documented phenomenon of CNNs, sensitivity to other transformations is typically encouraged through data augmentation. We investigate the modulation of complex valued convolutional weights with learned Gabor filters to enable orientation robustness. The resulting network can generate orientation dependent features free of interpolation with a single set of learnable rotation-governing parameters. By choosing to either retain or pool orientation channels, the choice of equivariance versus invariance can be directly controlled. Moreover, we introduce rotational weight-tying through a proposed cyclic Gabor convolution, further enabling generalisation over rotations. We combine these innovations into Learnable Gabor Convolutional Networks (LGCNs), that are parameter-efficient and offer increased model complexity. We demonstrate their rotation invariance and equivariance on MNIST, BSD and a dataset of simulated and real astronomical images of Galactic cirri.
△ Less
Submitted 5 October, 2021; v1 submitted 23 November, 2020;
originally announced November 2020.
-
VI-Net: View-Invariant Quality of Human Movement Assessment
Authors:
Faegheh Sardari,
Adeline Paiement,
Sion Hannuna,
Majid Mirmehdi
Abstract:
We propose a view-invariant method towards the assessment of the quality of human movements which does not rely on skeleton data. Our end-to-end convolutional neural network consists of two stages, where at first a view-invariant trajectory descriptor for each body joint is generated from RGB images, and then the collection of trajectories for all joints are processed by an adapted, pre-trained 2D…
▽ More
We propose a view-invariant method towards the assessment of the quality of human movements which does not rely on skeleton data. Our end-to-end convolutional neural network consists of two stages, where at first a view-invariant trajectory descriptor for each body joint is generated from RGB images, and then the collection of trajectories for all joints are processed by an adapted, pre-trained 2D CNN (e.g. VGG-19 or ResNeXt-50) to learn the relationship amongst the different body parts and deliver a score for the movement quality. We release the only publicly-available, multi-view, non-skeleton, non-mocap, rehabilitation movement dataset (QMAR), and provide results for both cross-subject and cross-view scenarios on this dataset. We show that VI-Net achieves average rank correlation of 0.66 on cross-subject and 0.65 on unseen views when trained on only two views. We also evaluate the proposed method on the single-view rehabilitation dataset KIMORE and obtain 0.66 rank correlation against a baseline of 0.62.
△ Less
Submitted 11 August, 2020;
originally announced August 2020.
-
Challenges in Annotation of useR Data for UbiquitOUs Systems: Results from the 1st ARDUOUS Workshop
Authors:
Kristina Yordanova,
Adeline Paiement,
Max Schröder,
Emma Tonkin,
Przemyslaw Woznowski,
Carl Magnus Olsson,
Joseph Rafferty,
Timo Sztyler
Abstract:
Labelling user data is a central part of the design and evaluation of pervasive systems that aim to support the user through situation-aware reasoning. It is essential both in designing and training the system to recognise and reason about the situation, either through the definition of a suitable situation model in knowledge-driven applications, or through the preparation of training data for lea…
▽ More
Labelling user data is a central part of the design and evaluation of pervasive systems that aim to support the user through situation-aware reasoning. It is essential both in designing and training the system to recognise and reason about the situation, either through the definition of a suitable situation model in knowledge-driven applications, or through the preparation of training data for learning tasks in data-driven models. Hence, the quality of annotations can have a significant impact on the performance of the derived systems. Labelling is also vital for validating and quantifying the performance of applications. In particular, comparative evaluations require the production of benchmark datasets based on high-quality and consistent annotations. With pervasive systems relying increasingly on large datasets for designing and testing models of users' activities, the process of data labelling is becoming a major concern for the community. In this work we present a qualitative and quantitative analysis of the challenges associated with annotation of user data and possible strategies towards addressing these challenges. The analysis was based on the data gathered during the 1st International Workshop on Annotation of useR Data for UbiquitOUs Systems (ARDUOUS) and consisted of brainstorming as well as annotation and questionnaire data gathered during the talks, poster session, live annotation session, and discussion session.
△ Less
Submitted 15 March, 2018;
originally announced March 2018.
-
Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home
Authors:
Lili Tao,
Tilo Burghardt,
Majid Mirmehdi,
Dima Damen,
Ashley Cooper,
Sion Hannuna,
Massimo Camplani,
Adeline Paiement,
Ian Craddock
Abstract:
We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables…
▽ More
We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. % based on per breath gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.
△ Less
Submitted 27 July, 2016;
originally announced July 2016.
-
Multiple Human Tracking in RGB-D Data: A Survey
Authors:
Massimo Camplani,
Adeline Paiement,
Majid Mirmehdi,
Dima Damen,
Sion Hannuna,
Tilo Burghardt,
Lili Tao
Abstract:
Multiple human tracking (MHT) is a fundamental task in many computer vision applications. Appearance-based approaches, primarily formulated on RGB data, are constrained and affected by problems arising from occlusions and/or illumination variations. In recent years, the arrival of cheap RGB-Depth (RGB-D) devices has {led} to many new approaches to MHT, and many of these integrate color and depth c…
▽ More
Multiple human tracking (MHT) is a fundamental task in many computer vision applications. Appearance-based approaches, primarily formulated on RGB data, are constrained and affected by problems arising from occlusions and/or illumination variations. In recent years, the arrival of cheap RGB-Depth (RGB-D) devices has {led} to many new approaches to MHT, and many of these integrate color and depth cues to improve each and every stage of the process. In this survey, we present the common processing pipeline of these methods and review their methodology based (a) on how they implement this pipeline and (b) on what role depth plays within each stage of it. We identify and introduce existing, publicly available, benchmark datasets and software resources that fuse color and depth data for MHT. Finally, we present a brief comparative evaluation of the performance of those works that have applied their methods to these datasets.
△ Less
Submitted 14 June, 2016;
originally announced June 2016.
-
Black Hole Motion as Catalyst of Orbital Resonances
Authors:
C. M. Boily,
T. Padmanabhan,
A. Paiement
Abstract:
The motion of a black hole about the centre of gravity of its host galaxy induces a strong response from the surrounding stellar population. We treat the case of a harmonic potential analytically and show that half of the stars on circular orbits in that potential shift to an orbit of lower energy, while the other half receive a positive boost and recede to a larger radius. The black hole itself…
▽ More
The motion of a black hole about the centre of gravity of its host galaxy induces a strong response from the surrounding stellar population. We treat the case of a harmonic potential analytically and show that half of the stars on circular orbits in that potential shift to an orbit of lower energy, while the other half receive a positive boost and recede to a larger radius. The black hole itself remains on an orbit of fixed amplitude and merely acts as a catalyst for the evolution of the stellar energy distribution function f(E). We show that this effect is operative out to a radius of approx 3 to 4 times the hole's influence radius, R_bh. We use numerical integration to explore more fully the response of a stellar distribution to black hole motion. We consider orbits in a logarithmic potential and compare the response of stars on circular orbits, to the situation of a `warm' and `hot' (isotropic) stellar velocity field. While features seen in density maps are now wiped out, the kinematic signature of black hole motion still imprints the stellar line-of-sight mean velocity to a magnitude ~18% the local root mean-square velocity dispersion sigma.
△ Less
Submitted 23 June, 2007; v1 submitted 18 May, 2007;
originally announced May 2007.
