-
Deep learning-based detection of morphological features associated with hypoxia in H&E breast cancer whole slide images
Authors:
Petru Manescu,
Joseph Geradts,
Delmiro Fernandez-Reyes
Abstract:
Hypoxia occurs when tumour cells outgrow their blood supply, leading to regions of low oxygen levels within the tumour. Calculating hypoxia levels can be an important step in understanding the biology of tumours, their clinical progression and response to treatment. This study demonstrates a novel application of deep learning to evaluate hypoxia in the context of breast cancer histomorphology. Mor…
▽ More
Hypoxia occurs when tumour cells outgrow their blood supply, leading to regions of low oxygen levels within the tumour. Calculating hypoxia levels can be an important step in understanding the biology of tumours, their clinical progression and response to treatment. This study demonstrates a novel application of deep learning to evaluate hypoxia in the context of breast cancer histomorphology. More precisely, we show that Weakly Supervised Deep Learning (WSDL) models can accurately detect hypoxia associated features in routine Hematoxylin and Eosin (H&E) whole slide images (WSI). We trained and evaluated a deep Multiple Instance Learning model on tiles from WSI H&E tissue from breast cancer primary sites (n=240) obtaining on average an AUC of 0.87 on a left-out test set. We also showed significant differences between features of hypoxic and normoxic tissue regions as distinguished by the WSDL models. Such DL hypoxia H&E WSI detection models could potentially be extended to other tumour types and easily integrated into the pathology workflow without requiring additional costly assays.
△ Less
Submitted 21 November, 2023;
originally announced November 2023.
-
Efficient Jacobian-Based Inverse Kinematics with Sim-to-Real Transfer of Soft Robots by Learning
Authors:
Guoxin Fang,
Yingjun Tian,
Zhi-Xin Yang,
Jo M. P. Geraedts,
Charlie C. L. Wang
Abstract:
This paper presents an efficient learning-based method to solve the inverse kinematic (IK) problem on soft robots with highly non-linear deformation. The major challenge of efficiently computing IK for such robots is due to the lack of analytical formulation for either forward or inverse kinematics. To address this challenge, we employ neural networks to learn both the map** function of forward…
▽ More
This paper presents an efficient learning-based method to solve the inverse kinematic (IK) problem on soft robots with highly non-linear deformation. The major challenge of efficiently computing IK for such robots is due to the lack of analytical formulation for either forward or inverse kinematics. To address this challenge, we employ neural networks to learn both the map** function of forward kinematics and also the Jacobian of this function. As a result, Jacobian-based iteration can be applied to solve the IK problem. A sim-to-real training transfer strategy is conducted to make this approach more practical. We first generate a large number of samples in a simulation environment for learning both the kinematic and the Jacobian networks of a soft robot design. Thereafter, a sim-to-real layer of differentiable neurons is employed to map the results of simulation to the physical hardware, where this sim-to-real layer can be learned from a very limited number of training samples generated on the hardware. The effectiveness of our approach has been verified on pneumatic-driven soft robots for path following and interactive positioning.
△ Less
Submitted 5 June, 2022; v1 submitted 27 December, 2020;
originally announced December 2020.
-
Sensing and Reconstruction of 3D Deformation on Pneumatic Soft Robots
Authors:
Rob B. N. Scharff,
Guoxin Fang,
Yingjun Tian,
Jun Wu,
Jo M. P. Geraedts,
Charlie C. L. Wang
Abstract:
Real-time proprioception is a challenging problem for soft robots, which have almost infinite degrees-of-freedom in body deformation. When multiple actuators are used, it becomes more difficult as deformation can also occur on actuators caused by interaction between each other. To tackle this problem, we present a method in this paper to sense and reconstruct 3D deformation on pneumatic soft robot…
▽ More
Real-time proprioception is a challenging problem for soft robots, which have almost infinite degrees-of-freedom in body deformation. When multiple actuators are used, it becomes more difficult as deformation can also occur on actuators caused by interaction between each other. To tackle this problem, we present a method in this paper to sense and reconstruct 3D deformation on pneumatic soft robots by first integrating multiple low-cost sensors inside the chambers of pneumatic actuators and then using machine learning to convert the captured signals into shape parameters of soft robots. An exterior motion capture system is employed to generate the datasets for both training and testing. With the help of good shape parameterization, the 3D shape of a soft robot can be accurately reconstructed from signals obtained from multiple sensors. We demonstrate the effectiveness of this approach on two designs of soft robots -- a robotic joint and a deformable membrane. After parameterizing the deformation of these soft robots into compact shape parameters, we can effectively train the neural networks to reconstruct the 3D deformation from the sensor signals. The sensing and shape prediction pipeline can run at 50Hz in real-time on a consumer-level device.
△ Less
Submitted 22 December, 2020;
originally announced December 2020.
-
Gloss, Color and Topography Scanning for Reproducing a Painting's Appearance using 3D printing
Authors:
Willemijn Elkhuizen,
Tessa Essers,
Yu Song,
Jo Geraedts,
Clemens Weijkamp,
Joris Dik,
Sylvia Pont
Abstract:
High fidelity reproductions of paintings provide new opportunities to museums in preserving and providing access to cultural heritage. This paper presents an integrated system which is able to capture and fabricate color, topography and gloss of a painting, of which gloss capturing forms the most important contribution. A 3D imaging system, using fringe-encoded stereo imaging, is extended to captu…
▽ More
High fidelity reproductions of paintings provide new opportunities to museums in preserving and providing access to cultural heritage. This paper presents an integrated system which is able to capture and fabricate color, topography and gloss of a painting, of which gloss capturing forms the most important contribution. A 3D imaging system, using fringe-encoded stereo imaging, is extended to capture spatially-varying gloss, utilizing specular reflectance polarization. The gloss is measured by sampling the specular reflection around Brewster's angle, where these reflections are effectively polarized, and can be separated from the unpolarized, diffuse reflectance. Off-center gloss measurements are calibrated relative to the center measurement. Off-specular gloss measurements, following from local variation of the surface normal, are masked based on the height map and corrected. Shadowed regions, caused by the 3D relief, are treated similarly. The area of a single capture is approximately 180x90mm at a resolution of 25x25micron. Aligned color, height, and gloss tiles are stitched together, registering overlap** color areas. These maps are inputs for a 3D printer. Two paintings were reproduced to verify the effectiveness and efficiency of the proposed system. One painting was scanned four times, consecutively rotated by 90 degrees, to evaluate the influence of the scanning system geometric configuration on the gloss measurement. Experimental results show that the method is sufficiently fast for practical application. The results can well be used for the purpose of physical reproduction and other applications needing first-order estimates of the appearance. Our method to extend appearance scanning with gloss measurements is a valuable addition in the quest for realistic reproductions, in terms of its practical applicability and its perceptual added value, when added to color and topography.
△ Less
Submitted 23 October, 2019;
originally announced October 2019.
-
Planning Jerk-Optimized Trajectory with Discrete-Time Constraints for Redundant Robots
Authors:
Chengkai Dai,
Sylvain Lefebvre,
Kai-Ming Yu,
Jo M. P. Geraedts,
Charlie C. L. Wang
Abstract:
We present a method for effectively planning the motion trajectory of robots in manufacturing tasks, the tool-paths of which are usually complex and have a large number of discrete-time constraints as waypoints. Kinematic redundancy also exists in these robotic systems. The jerk of motion is optimized in our trajectory planning method at the meanwhile of fabrication process to improve the quality…
▽ More
We present a method for effectively planning the motion trajectory of robots in manufacturing tasks, the tool-paths of which are usually complex and have a large number of discrete-time constraints as waypoints. Kinematic redundancy also exists in these robotic systems. The jerk of motion is optimized in our trajectory planning method at the meanwhile of fabrication process to improve the quality of fabrication.
△ Less
Submitted 15 February, 2020; v1 submitted 14 September, 2019;
originally announced September 2019.
