-
Differentiable Rendering as a Way to Program Cable-Driven Soft Robots
Authors:
Kasra Arnavaz,
Kenny Erleben
Abstract:
Soft robots have gained increased popularity in recent years due to their adaptability and compliance. In this paper, we use a digital twin model of cable-driven soft robots to learn control parameters in simulation. In doing so, we take advantage of differentiable rendering as a way to instruct robots to complete tasks such as point reach, grip** an object, and obstacle avoidance. This approach…
▽ More
Soft robots have gained increased popularity in recent years due to their adaptability and compliance. In this paper, we use a digital twin model of cable-driven soft robots to learn control parameters in simulation. In doing so, we take advantage of differentiable rendering as a way to instruct robots to complete tasks such as point reach, grip** an object, and obstacle avoidance. This approach simplifies the mathematical description of such complicated tasks and removes the need for landmark points and their tracking. Our experiments demonstrate the applicability of our method.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Extremely weakly-supervised blood vessel segmentation with physiologically based synthesis and domain adaptation
Authors:
Peidi Xu,
Olga Sosnovtseva,
Charlotte Mehlin Sørensen,
Kenny Erleben,
Sune Darkner
Abstract:
Accurate analysis and modeling of renal functions require a precise segmentation of the renal blood vessels. Micro-CT scans provide image data at higher resolutions, making more small vessels near the renal cortex visible. Although deep-learning-based methods have shown state-of-the-art performance in automatic blood vessel segmentations, they require a large amount of labeled training data. Howev…
▽ More
Accurate analysis and modeling of renal functions require a precise segmentation of the renal blood vessels. Micro-CT scans provide image data at higher resolutions, making more small vessels near the renal cortex visible. Although deep-learning-based methods have shown state-of-the-art performance in automatic blood vessel segmentations, they require a large amount of labeled training data. However, voxel-wise labeling in micro-CT scans is extremely time-consuming given the huge volume sizes. To mitigate the problem, we simulate synthetic renal vascular trees physiologically while generating corresponding scans of the simulated trees by training a generative model on unlabeled scans. This enables the generative model to learn the map** implicitly without the need for explicit functions to emulate the image acquisition process. We further propose an additional segmentation branch over the generative model trained on the generated scans. We demonstrate that the model can directly segment blood vessels on real scans and validate our method on both 3D micro-CT scans of rat kidneys and a proof-of-concept experiment on 2D retinal images. Code and 3D results are available at https://github.com/miccai2023anony/RenalVesselSeg
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
Deep Learning-Assisted Localisation of Nanoparticles in synthetically generated two-photon microscopy images
Authors:
Rasmus Netterstrøm,
Nikolay Kutuzov,
Sune Darkner,
Maurits Jørring Pallesen,
Martin Johannes Lauritzen,
Kenny Erleben,
Francois Lauze
Abstract:
Tracking single molecules is instrumental for quantifying the transport of molecules and nanoparticles in biological samples, e.g., in brain drug delivery studies. Existing intensity-based localisation methods are not developed for imaging with a scanning microscope, typically used for in vivo imaging. Low signal-to-noise ratios, movement of molecules out-of-focus, and high motion blur on images r…
▽ More
Tracking single molecules is instrumental for quantifying the transport of molecules and nanoparticles in biological samples, e.g., in brain drug delivery studies. Existing intensity-based localisation methods are not developed for imaging with a scanning microscope, typically used for in vivo imaging. Low signal-to-noise ratios, movement of molecules out-of-focus, and high motion blur on images recorded with scanning two-photon microscopy (2PM) in vivo pose a challenge to the accurate localisation of molecules. Using data-driven models is challenging due to low data volumes, typical for in vivo experiments. We developed a 2PM image simulator to supplement scarce training data. The simulator mimics realistic motion blur, background fluorescence, and shot noise observed in vivo imaging. Training a data-driven model with simulated data improves localisation quality in simulated images and shows why intensity-based methods fail.
△ Less
Submitted 17 March, 2023;
originally announced March 2023.
-
A Hybrid Approach to Full-Scale Reconstruction of Renal Arterial Network
Authors:
Peidi Xu,
Niels-Henrik Holstein-Rathlou,
Stinne Byrholdt Søgaard,
Carsten Gundlach,
Charlotte Mehlin Sørensen,
Kenny Erleben,
Olga Sosnovtseva,
Sune Darkner
Abstract:
The renal vasculature, acting as a resource distribution network, plays an important role in both the physiology and pathophysiology of the kidney. However, no imaging techniques allow an assessment of the structure and function of the renal vasculature due to limited spatial and temporal resolution. To develop realistic computer simulations of renal function, and to develop new image-based diagno…
▽ More
The renal vasculature, acting as a resource distribution network, plays an important role in both the physiology and pathophysiology of the kidney. However, no imaging techniques allow an assessment of the structure and function of the renal vasculature due to limited spatial and temporal resolution. To develop realistic computer simulations of renal function, and to develop new image-based diagnostic methods based on artificial intelligence, it is necessary to have a realistic full-scale model of the renal vasculature. We propose a hybrid framework to build subject-specific models of the renal vascular network by using semi-automated segmentation of large arteries and estimation of cortex area from a micro-CT scan as a starting point, and by adopting the Global Constructive Optimization algorithm for generating smaller vessels. Our results show a statistical correspondence between the reconstructed data and existing anatomical data obtained from a rat kidney with respect to morphometric and hemodynamic parameters.
△ Less
Submitted 3 March, 2023;
originally announced March 2023.
-
cRedAnno+: Annotation Exploitation in Self-Explanatory Lung Nodule Diagnosis
Authors:
Jiahao Lu,
Chong Yin,
Kenny Erleben,
Michael Bachmann Nielsen,
Sune Darkner
Abstract:
Recently, attempts have been made to reduce annotation requirements in feature-based self-explanatory models for lung nodule diagnosis. As a representative, cRedAnno achieves competitive performance with considerably reduced annotation needs by introducing self-supervised contrastive learning to do unsupervised feature extraction. However, it exhibits unstable performance under scarce annotation c…
▽ More
Recently, attempts have been made to reduce annotation requirements in feature-based self-explanatory models for lung nodule diagnosis. As a representative, cRedAnno achieves competitive performance with considerably reduced annotation needs by introducing self-supervised contrastive learning to do unsupervised feature extraction. However, it exhibits unstable performance under scarce annotation conditions. To improve the accuracy and robustness of cRedAnno, we propose an annotation exploitation mechanism by conducting semi-supervised active learning with sparse seeding and training quenching in the learned semantically meaningful reasoning space to jointly utilise the extracted features, annotations, and unlabelled data. The proposed approach achieves comparable or even higher malignancy prediction accuracy with 10x fewer annotations, meanwhile showing better robustness and nodule attribute prediction accuracy under the condition of 1% annotations. Our complete code is open-source available: https://github.com/diku-dk/credanno.
△ Less
Submitted 7 November, 2022; v1 submitted 28 October, 2022;
originally announced October 2022.
-
Open-Full-Jaw: An open-access dataset and pipeline for finite element models of human jaw
Authors:
Torkan Gholamalizadeh,
Faezeh Moshfeghifar,
Zachary Ferguson,
Teseo Schneider,
Daniele Panozzo,
Sune Darkner,
Masrour Makaremi,
François Chan,
Peter Lempel Søndergaard,
Kenny Erleben
Abstract:
Develo** computational models of the human jaw acquired from cone-beam computed tomography (CBCT) scans is time-consuming and labor-intensive. Besides, a quantitative comparison is not attainable in the literature due to the involved manual tasks and the lack of surface/volumetric meshes. We share an open-access repository of 17 patient-specific finite-element (FE) models of human jaws acquired…
▽ More
Develo** computational models of the human jaw acquired from cone-beam computed tomography (CBCT) scans is time-consuming and labor-intensive. Besides, a quantitative comparison is not attainable in the literature due to the involved manual tasks and the lack of surface/volumetric meshes. We share an open-access repository of 17 patient-specific finite-element (FE) models of human jaws acquired from CBCT scans and the utilized pipeline for generating them. The proposed pipeline minimizes model generation time and potential errors caused by human interventions. It gets dense surface meshes and provides reduced conformal surface/volumetric meshes suitable for FE analysis. We have quantified the geometrical variations of developed models and assessed models' accuracy from different aspects; (1) the maximum deviations from the input meshes, (2) the mesh quality, and (3) the simulation results. Our results indicate that the developed computational models are precise and have quality meshes suitable for various FE scenarios. Therefore, we believe this dataset will pave the way for future population studies.
△ Less
Submitted 24 August, 2022;
originally announced September 2022.
-
Auto-segmentation of Hip Joints using MultiPlanar UNet with Transfer learning
Authors:
Peidi Xu,
Faezeh Moshfeghifar,
Torkan Gholamalizadeh,
Michael Bachmann Nielsen,
Kenny Erleben,
Sune Darkner
Abstract:
Accurate geometry representation is essential in develo** finite element models. Although generally good, deep-learning segmentation approaches with only few data have difficulties in accurately segmenting fine features, e.g., gaps and thin structures. Subsequently, segmented geometries need labor-intensive manual modifications to reach a quality where they can be used for simulation purposes. W…
▽ More
Accurate geometry representation is essential in develo** finite element models. Although generally good, deep-learning segmentation approaches with only few data have difficulties in accurately segmenting fine features, e.g., gaps and thin structures. Subsequently, segmented geometries need labor-intensive manual modifications to reach a quality where they can be used for simulation purposes. We propose a strategy that uses transfer learning to reuse datasets with poor segmentation combined with an interactive learning step where fine-tuning of the data results in anatomically accurate segmentations suitable for simulations. We use a modified MultiPlanar UNet that is pre-trained using inferior hip joint segmentation combined with a dedicated loss function to learn the gap regions and post-processing to correct tiny inaccuracies on symmetric classes due to rotational invariance. We demonstrate this robust yet conceptually simple approach applied with clinically validated results on publicly available computed tomography scans of hip joints. Code and resulting 3D models are available at: https://github.com/MICCAI2022-155/AuToSeg}
△ Less
Submitted 18 August, 2022; v1 submitted 17 August, 2022;
originally announced August 2022.
-
A multi-patient analysis of the center of rotation trajectories using finite element models of the human mandible
Authors:
Torkan Gholamalizadeh,
Sune Darkner,
Peter Lempel Søndergaard,
Kenny Erleben
Abstract:
Studying different types of tooth movements can help us to better understand the force systems used for tooth position correction in orthodontic treatments. This study considers a more realistic force system in tooth movement modeling across different patients and investigates the effect of the couple force direction on the position of the center of rotation (CRot). The finite-element (FE) models…
▽ More
Studying different types of tooth movements can help us to better understand the force systems used for tooth position correction in orthodontic treatments. This study considers a more realistic force system in tooth movement modeling across different patients and investigates the effect of the couple force direction on the position of the center of rotation (CRot). The finite-element (FE) models of human mandibles from three patients are used to investigate the position of the CRots for different patients' teeth in 3D space. The CRot is considered a single point in a 3D coordinate system and is obtained by choosing the closest point on the axis of rotation to the center of resistance (CRes). A force system, consisting of a constant load and a couple (pair of forces), is applied to each tooth, and the corresponding CRot trajectories are examined across different patients. To perform a consistent inter-patient analysis, different patients' teeth are registered to the corresponding reference teeth using an affine transformation. The selected directions and applied points of force on the reference teeth are then transformed into the registered teeth domains. The effect of the direction of the couple on the location of the CRot is also studied by rotating the couples about the three principal axes of a patient's premolar. Our results indicate that similar patterns can be obtained for the CRot positions of different patients and teeth if the same load conditions are used. Moreover, equally rotating the direction of the couple about the three principal axes results in different patterns for the CRot positions, especially in labiolingual direction. The CRot trajectories follow similar patterns in the corresponding teeth, but any changes in the direction of the force and couple cause misalignment of the CRot trajectories, seen as rotations about the long axis of the tooth.
△ Less
Submitted 30 May, 2022;
originally announced July 2022.
-
Reducing Annotation Need in Self-Explanatory Models for Lung Nodule Diagnosis
Authors:
Jiahao Lu,
Chong Yin,
Oswin Krause,
Kenny Erleben,
Michael Bachmann Nielsen,
Sune Darkner
Abstract:
Feature-based self-explanatory methods explain their classification in terms of human-understandable features. In the medical imaging community, this semantic matching of clinical knowledge adds significantly to the trustworthiness of the AI. However, the cost of additional annotation of features remains a pressing issue. We address this problem by proposing cRedAnno, a data-/annotation-efficient…
▽ More
Feature-based self-explanatory methods explain their classification in terms of human-understandable features. In the medical imaging community, this semantic matching of clinical knowledge adds significantly to the trustworthiness of the AI. However, the cost of additional annotation of features remains a pressing issue. We address this problem by proposing cRedAnno, a data-/annotation-efficient self-explanatory approach for lung nodule diagnosis. cRedAnno considerably reduces the annotation need by introducing self-supervised contrastive learning to alleviate the burden of learning most parameters from annotation, replacing end-to-end training with two-stage training. When training with hundreds of nodule samples and only 1% of their annotations, cRedAnno achieves competitive accuracy in predicting malignancy, meanwhile significantly surpassing most previous works in predicting nodule attributes. Visualisation of the learned space further indicates that the correlation between the clustering of malignancy and nodule attributes coincides with clinical knowledge. Our complete code is open-source available: https://github.com/diku-dk/credanno.
△ Less
Submitted 25 July, 2022; v1 submitted 27 June, 2022;
originally announced June 2022.
-
A direct geometry processing cartilage generation method using segmented bone models from datasets with poor cartilage visibility
Authors:
Faezeh Moshfeghifar,
Max Kragballe Nielsen,
José D. Tascón-Vidarte,
Sune Darkner,
Kenny Erleben
Abstract:
We present a method to generate subject-specific cartilage for the hip joint. Given bone geometry, our approach is agnostic to image modality, creates conforming interfaces, and is well suited for finite element analysis. We demonstrate our method on ten hip joints showing anatomical shape consistency and well-behaved stress patterns. Our method is fast and may assist in large-scale biomechanical…
▽ More
We present a method to generate subject-specific cartilage for the hip joint. Given bone geometry, our approach is agnostic to image modality, creates conforming interfaces, and is well suited for finite element analysis. We demonstrate our method on ten hip joints showing anatomical shape consistency and well-behaved stress patterns. Our method is fast and may assist in large-scale biomechanical population studies of the hip joint when manual segmentation or training data is not feasible.
△ Less
Submitted 20 March, 2022;
originally announced March 2022.
-
gradSim: Differentiable simulation for system identification and visuomotor control
Authors:
Krishna Murthy Jatavallabhula,
Miles Macklin,
Florian Golemo,
Vikram Voleti,
Linda Petrini,
Martin Weiss,
Breandan Considine,
Jerome Parent-Levesque,
Kevin Xie,
Kenny Erleben,
Liam Paull,
Florian Shkurti,
Derek Nowrouzezahrai,
Sanja Fidler
Abstract:
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information during image formation. Current solutions require precise 3D labels which are labor-intensive to gather, and infeasible to create for many systems such as deformable…
▽ More
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information during image formation. Current solutions require precise 3D labels which are labor-intensive to gather, and infeasible to create for many systems such as deformable solids or cloth. We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering to jointly model the evolution of scene dynamics and image formation. This novel combination enables backpropagation from pixels in a video sequence through to the underlying physical attributes that generated them. Moreover, our unified computation graph -- spanning from the dynamics and through the rendering process -- enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision, while obtaining performance competitive to or better than techniques that rely on precise 3D labels.
△ Less
Submitted 6 April, 2021;
originally announced April 2021.
-
Mandibular Teeth Movement Variations in Tip** Scenario: A Finite Element Study on Several Patients
Authors:
Torkan Gholamalizadeh,
Sune Darkner,
Paolo Maria Cattaneo,
Peter Søndergaard,
Kenny Erleben
Abstract:
Previous studies on computational modeling of tooth movement in orthodontic treatments are limited to a single model and fail in generalizing the simulation results to other patients. To this end, we consider multiple patients and focus on tooth movement variations under the identical load and boundary conditions both for intra- and inter-patient analyses. We introduce a novel computational analys…
▽ More
Previous studies on computational modeling of tooth movement in orthodontic treatments are limited to a single model and fail in generalizing the simulation results to other patients. To this end, we consider multiple patients and focus on tooth movement variations under the identical load and boundary conditions both for intra- and inter-patient analyses. We introduce a novel computational analysis tool based on finite element models (FEMs) addressing how to assess initial tooth displacement in the mandibular dentition across different patients for uncontrolled tip** scenarios with different load magnitudes applied to the mandibular dentition. This is done by modeling the movement of each patient's tooth as a nonlinear function of both load and tooth size. As the size of tooth can affect the resulting tooth displacement, a combination of two clinical biomarkers obtained from the tooth anatomy, i.e., crown height and root volume, is considered to make the proposed model generalizable to different patients and teeth.
△ Less
Submitted 11 October, 2020;
originally announced October 2020.
-
Non-Smooth Newton Methods for Deformable Multi-Body Dynamics
Authors:
Miles Macklin,
Kenny Erleben,
Matthias Müller,
Nuttapong Chentanez,
Stefan Jeschke,
Viktor Makoviychuk
Abstract:
We present a framework for the simulation of rigid and deformable bodies in the presence of contact and friction. Our method is based on a non-smooth Newton iteration that solves the underlying nonlinear complementarity problems (NCPs) directly. This approach allows us to support nonlinear dynamics models, including hyperelastic deformable bodies and articulated rigid mechanisms, coupled through a…
▽ More
We present a framework for the simulation of rigid and deformable bodies in the presence of contact and friction. Our method is based on a non-smooth Newton iteration that solves the underlying nonlinear complementarity problems (NCPs) directly. This approach allows us to support nonlinear dynamics models, including hyperelastic deformable bodies and articulated rigid mechanisms, coupled through a smooth isotropic friction model. The fixed-point nature of our method means it requires only the solution of a symmetric linear system as a building block. We propose a new complementarity preconditioner for NCP functions that improves convergence, and we develop an efficient GPU-based solver based on the conjugate residual (CR) method that is suitable for interactive simulations. We show how to improve robustness using a new geometric stiffness approximation and evaluate our method's performance on a number of robotics simulation scenarios, including dexterous manipulation and training using reinforcement learning.
△ Less
Submitted 10 July, 2019;
originally announced July 2019.
-
A Validated Physical Model For Real-Time Simulation of Soft Robotic Snakes
Authors:
Renato Gasoto,
Miles Macklin,
Xuan Liu,
Yinan Sun,
Kenny Erleben,
Cagdas Onal,
Jie Fu
Abstract:
In this work we present a framework that is capable of accurately representing soft robotic actuators in a multiphysics environment in real-time. We propose a constraint-based dynamics model of a 1-dimensional pneumatic soft actuator that accounts for internal pressure forces, as well as the effect of actuator latency and dam** under inflation and deflation and demonstrate its accuracy a full so…
▽ More
In this work we present a framework that is capable of accurately representing soft robotic actuators in a multiphysics environment in real-time. We propose a constraint-based dynamics model of a 1-dimensional pneumatic soft actuator that accounts for internal pressure forces, as well as the effect of actuator latency and dam** under inflation and deflation and demonstrate its accuracy a full soft robotic snake with the composition of multiple 1D actuators. We verify our model's accuracy in static deformation and dynamic locomotion open-loop control experiments. To achieve real-time performance we leverage the parallel computation power of GPUs to allow interactive control and feedback.
△ Less
Submitted 4 April, 2019;
originally announced April 2019.
