-
Guiding the Last Centimeter: Novel Anatomy-Aware Probe Servoing for Standardized Imaging Plane Navigation in Robotic Lung Ultrasound
Authors:
Xihan Ma,
Mingjie Zeng,
Jeffrey C. Hill,
Beatrice Hoffmann,
Ziming Zhang,
Haichong K. Zhang
Abstract:
Navigating the ultrasound (US) probe to the standardized imaging plane (SIP) for image acquisition is a critical but operator-dependent task in conventional freehand diagnostic US. Robotic US systems (RUSS) offer the potential to enhance imaging consistency by leveraging real-time US image feedback to optimize the probe pose, thereby reducing reliance on operator expertise. However, determining th…
▽ More
Navigating the ultrasound (US) probe to the standardized imaging plane (SIP) for image acquisition is a critical but operator-dependent task in conventional freehand diagnostic US. Robotic US systems (RUSS) offer the potential to enhance imaging consistency by leveraging real-time US image feedback to optimize the probe pose, thereby reducing reliance on operator expertise. However, determining the proper approach to extracting generalizable features from the US images for probe pose adjustment remain challenging. In this work, we propose a SIP navigation framework for RUSS, exemplified in the context of robotic lung ultrasound (LUS). This framework facilitates automatic probe adjustment when in proximity to the SIP. This is achieved by explicitly extracting multiple anatomical features presented in real-time LUS images and performing non-patient-specific template matching to generate probe motion towards the SIP using image-based visual servoing (IBVS). This framework is further integrated with the active-sensing end-effector (A-SEE), a customized robot end-effector that leverages patient external body geometry to maintain optimal probe alignment with the contact surface, thus preserving US signal quality throughout the navigation. The proposed approach ensures procedural interpretability and inter-patient adaptability. Validation is conducted through anatomy-mimicking phantom and in-vivo evaluations involving five human subjects. The results show the framework's high navigation precision with the probe correctly located at the SIP for all cases, exhibiting positioning error of under 2 mm in translation and under 2 degree in rotation. These results demonstrate the navigation process's capability to accomondate anatomical variations among patients.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
Simultaneous Estimation of Hand Configurations and Finger Joint Angles using Forearm Ultrasound
Authors:
Keshav Bimbraw,
Christopher J. Nycz,
Matt Schueler,
Ziming Zhang,
Haichong K. Zhang
Abstract:
With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, augmented/virtual reality (AR/VR) interfaces, and physical robotic systems. Hand motion recognition is widely used to enable these interactions. Hand configuration classification and MCP joint angle detection is important for a comprehensive reconstruction o…
▽ More
With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, augmented/virtual reality (AR/VR) interfaces, and physical robotic systems. Hand motion recognition is widely used to enable these interactions. Hand configuration classification and MCP joint angle detection is important for a comprehensive reconstruction of hand motion. sEMG and other technologies have been used for the detection of hand motions. Forearm ultrasound images provide a musculoskeletal visualization that can be used to understand hand motion. Recent work has shown that these ultrasound images can be classified using machine learning to estimate discrete hand configurations. Estimating both hand configuration and MCP joint angles based on forearm ultrasound has not been addressed in the literature. In this paper, we propose a CNN based deep learning pipeline for predicting the MCP joint angles. The results for the hand configuration classification were compared by using different machine learning algorithms. SVC with different kernels, MLP, and the proposed CNN have been used to classify the ultrasound images into 11 hand configurations based on activities of daily living. Forearm ultrasound images were acquired from 6 subjects instructed to move their hands according to predefined hand configurations. Motion capture data was acquired to get the finger angles corresponding to the hand movements at different speeds. Average classification accuracy of 82.7% for the proposed CNN and over 80% for SVC for different kernels was observed on a subset of the dataset. An average RMSE of 7.35 degrees was obtained between the predicted and the true MCP joint angles. A low latency (6.25 - 9.1 Hz) pipeline has been proposed for estimating both MCP joint angles and hand configuration aimed at real-time control of human-machine interfaces.
△ Less
Submitted 28 November, 2022;
originally announced November 2022.
-
Actuated Reflector-Based Three-dimensional Ultrasound Imaging with Adaptive-Delay Synthetic Aperture Focusing
Authors:
Yichuan Tang,
Ryosuke Tsumura,
Jakub T. Kaminski,
Haichong K. Zhang
Abstract:
Three-dimensional (3D) ultrasound (US) imaging addresses the limitation in field-of-view (FOV) in conventional two-dimensional (2D) US imaging by providing 3D viewing of the anatomy. 3D US imaging has been extensively adapted for diagnosis and image-guided surgical intervention. However, conventional approaches to implement 3D US imaging require either expensive and sophisticated 2D array transduc…
▽ More
Three-dimensional (3D) ultrasound (US) imaging addresses the limitation in field-of-view (FOV) in conventional two-dimensional (2D) US imaging by providing 3D viewing of the anatomy. 3D US imaging has been extensively adapted for diagnosis and image-guided surgical intervention. However, conventional approaches to implement 3D US imaging require either expensive and sophisticated 2D array transducers, or external actuation mechanisms to move a one-dimensional array mechanically. Here, we propose a 3D US imaging mechanism using actuated acoustic reflector instead of the sensor elements for volume acquisition with significantly extended 3D FOV, which can be implemented with simple hardware and compact size. To improve image quality on the elevation plane, we introduce an adaptive-delay synthetic aperture focusing (AD-SAF) method for elevation beamforming. We first evaluated the proposed imaging mechanism and AD-SAF with simulated point targets and cysts targets. Results of point targets suggested improved image quality on the elevation plane, and results of cysts targets demonstrated a potential to improve 3D visualization of human anatomy. We built a prototype imaging system that has a 3D FOV of 38 mm (lateral) by 38 mm (elevation) by 50 mm (axial) and collected data in imaging experiments with phantoms. Experimental data showed consistency with simulation results. The AD-SAF method enhanced quantifying the cyst volume size in the breast mimicking phantom compared to without elevation beamforming. These results suggested that the proposed 3D US imaging mechanism could potentially be applied in clinical scenarios.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
Prediction of Metacarpophalangeal joint angles and Classification of Hand configurations based on Ultrasound Imaging of the Forearm
Authors:
Keshav Bimbraw,
Christopher Julius Nycz,
Matt Schueler,
Ziming Zhang,
Haichong K. Zhang
Abstract:
With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, AR/VR interfaces, and physical robotic systems. Hand movement recognition is widely used to enable this interaction. Hand configuration classification and Metacarpophalangeal (MCP) joint angle detection are important for a comprehensive reconstruction of the…
▽ More
With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, AR/VR interfaces, and physical robotic systems. Hand movement recognition is widely used to enable this interaction. Hand configuration classification and Metacarpophalangeal (MCP) joint angle detection are important for a comprehensive reconstruction of the hand motion. Surface electromyography and other technologies have been used for the detection of hand motions. Ultrasound images of the forearm offer a way to visualize the internal physiology of the hand from a musculoskeletal perspective. Recent work has shown that these images can be classified using machine learning to predict various hand configurations. In this paper, we propose a Convolutional Neural Network (CNN) based deep learning pipeline for predicting the MCP joint angles. We supplement our results by using a Support Vector Classifier (SVC) to classify the ultrasound information into several predefined hand configurations based on activities of daily living (ADL). Ultrasound data from the forearm was obtained from 6 subjects who were instructed to move their hands according to predefined hand configurations relevant to ADLs. Motion capture data was acquired as the ground truth for hand movements at different speeds (0.5 Hz, 1 Hz, & 2 Hz) for the index, middle, ring, and pinky fingers. We were able to get promising SVC classification results on a subset of our collected data set. We demonstrated a correspondence between the predicted MCP joint angles and the actual MCP joint angles for the fingers, with an average root mean square error of 7.35 degrees. We implemented a low latency (6.25 - 9.1 Hz) pipeline for the prediction of both MCP joint angles and hand configuration estimation aimed at real-time control of digital devices, AR/VR interfaces, and physical robots.
△ Less
Submitted 26 September, 2021; v1 submitted 22 September, 2021;
originally announced September 2021.
-
Tele-operative Robotic Lung Ultrasound Scanning Platform for Triage of COVID-19 Patients
Authors:
Ryosuke Tsumura,
John W. Hardin,
Keshav Bimbraw,
Olushola S. Odusanya,
Yihao Zheng,
Jeffrey C. Hill,
Beatrice Hoffmann,
Winston Soboyejo,
Haichong K. Zhang
Abstract:
Novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become a pandemic of epic proportions and a global response to prepare health systems worldwide is of utmost importance. In addition to its cost-effectiveness in a resources-limited setting, lung ultrasound (LUS) has emerged as a rapid noninvasive imaging tool for the diagnosis of COVID-19 infected patients. Concerns surroundin…
▽ More
Novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become a pandemic of epic proportions and a global response to prepare health systems worldwide is of utmost importance. In addition to its cost-effectiveness in a resources-limited setting, lung ultrasound (LUS) has emerged as a rapid noninvasive imaging tool for the diagnosis of COVID-19 infected patients. Concerns surrounding LUS include the disparity of infected patients and healthcare providers, relatively small number of physicians and sonographers capable of performing LUS, and most importantly, the requirement for substantial physical contact between the patient and operator, increasing the risk of transmission. Mitigation of the spread of the virus is of paramount importance. A 2-dimensional (2D) tele-operative robotic platform capable of performing LUS in for COVID-19 infected patients may be of significant benefit. The authors address the aforementioned issues surrounding the use of LUS in the application of COVID- 19 infected patients. In addition, first time application, feasibility and safety were validated in three healthy subjects, along with 2D image optimization and comparison for overall accuracy. Preliminary results demonstrate that the proposed platform allows for successful acquisition and application of LUS in humans.
△ Less
Submitted 11 November, 2020; v1 submitted 23 October, 2020;
originally announced October 2020.
-
Effect of He on the Order-Disorder Transition in Ni3Al under Irradiation
Authors:
Peyman Saidi,
Pooyan Changizian,
Eric Nicholson,
He Ken Zhang,
Yu Luo,
Zhongwen Yao,
Chandra Veer Singh,
Mark R. Daymond,
Laurent Karim Beland
Abstract:
The order-disorder transition in Ni-Al alloys under irradiation represents an interplay between various re-ordering processes and disordering due to thermal spikes generated by incident high energy particles. Typically, ordering in enabled by diffusion of thermally-generated vacancies, and can only take place at temperatures where they are mobile and in sufficiently high concentration. Here, in-si…
▽ More
The order-disorder transition in Ni-Al alloys under irradiation represents an interplay between various re-ordering processes and disordering due to thermal spikes generated by incident high energy particles. Typically, ordering in enabled by diffusion of thermally-generated vacancies, and can only take place at temperatures where they are mobile and in sufficiently high concentration. Here, in-situ transmission electron micrographs reveal that the presence of He, usually considered to be a deleterious immiscible atom in this material, promotes reordering in Ni3Al at temperatures where vacancies are not effective ordering agents. A rate-theory model is presented, that quantitatively explains this behavior, based on parameters extracted from atomistic simulations. These calculations show that the V2He complex is an effective agent through its high stability and mobility. It is surmised that immiscible atoms may stabilize reordering agents in other materials undergoing driven processes, and preserve ordered phases at temperature where the driven processes would otherwise lead to disorder.
△ Less
Submitted 26 February, 2020;
originally announced February 2020.
-
n-Alkyl Thiol Head Group Interactions with the Au(111) Surface
Authors:
Y. Yourdshahyan,
H. K. Zhang,
A. M. Rappe
Abstract:
State-of-the-art first principles calculations based on density functional theory were performed on CH_3(CH_2)_{n-1}S-Au(111) systems. We show that the adsorption site of methylthiolate at low coverage on the Au(111) surface is the fcc site, not the hcp site as has been recently reported. Further, we report results for chain length dependency and the electronic structure of the system.
State-of-the-art first principles calculations based on density functional theory were performed on CH_3(CH_2)_{n-1}S-Au(111) systems. We show that the adsorption site of methylthiolate at low coverage on the Au(111) surface is the fcc site, not the hcp site as has been recently reported. Further, we report results for chain length dependency and the electronic structure of the system.
△ Less
Submitted 22 August, 2000;
originally announced August 2000.
