-
Occlusion-Robust Multi-Sensory Posture Estimation in Physical Human-Robot Interaction
Authors:
Amir Yazdani,
Roya Sabbagh Novin,
Andrew Merryweather,
Tucker Hermans
Abstract:
3D posture estimation is important in analyzing and improving ergonomics in physical human-robot interaction and reducing the risk of musculoskeletal disorders. Vision-based posture estimation approaches are prone to sensor and model errors, as well as occlusion, while posture estimation solely from the interacting robot's trajectory suffers from ambiguous solutions. To benefit from the advantages…
▽ More
3D posture estimation is important in analyzing and improving ergonomics in physical human-robot interaction and reducing the risk of musculoskeletal disorders. Vision-based posture estimation approaches are prone to sensor and model errors, as well as occlusion, while posture estimation solely from the interacting robot's trajectory suffers from ambiguous solutions. To benefit from the advantages of both approaches and improve upon their drawbacks, we introduce a low-cost, non-intrusive, and occlusion-robust multi-sensory 3D postural estimation algorithm in physical human-robot interaction. We use 2D postures from OpenPose over a single camera, and the trajectory of the interacting robot while the human performs a task. We model the problem as a partially-observable dynamical system and we infer the 3D posture via a particle filter. We present our work in teleoperation, but it can be generalized to other applications of physical human-robot interaction. We show that our multi-sensory system resolves human kinematic redundancy better than posture estimation solely using OpenPose or posture estimation solely using the robot's trajectory. This will increase the accuracy of estimated postures compared to the gold-standard motion capture postures. Moreover, our approach also performs better than other single sensory methods when postural assessment using RULA assessment tool.
△ Less
Submitted 12 August, 2022;
originally announced August 2022.
-
Impactful Robots: Evaluating Visual and Audio Warnings to Help Users Brace for Impact in Human Robot Interaction
Authors:
Nathaniel G. Luttmer,
Takara E. Truong,
Alicia M. Boynton,
Andrew S. Merryweather,
David R. Carrier,
Mark A. Minor
Abstract:
Wearable robotic devices have potential to assist and protect their users. Toward design of a Smart Helmet, this article examines the effectiveness of audio and visual warnings to help participants brace for impacts. A user study examines different warnings and impacts applied to users while running. Perturbation forces scaled to user mass are applied from different directions and user displacemen…
▽ More
Wearable robotic devices have potential to assist and protect their users. Toward design of a Smart Helmet, this article examines the effectiveness of audio and visual warnings to help participants brace for impacts. A user study examines different warnings and impacts applied to users while running. Perturbation forces scaled to user mass are applied from different directions and user displacement is measured to characterize effectiveness of the warning. This is accomplished using the TreadPort Active Wind Tunnel adapted to deliver forward, rearward, right, or left perturbation forces at precise moments during the locomotor cycle. The article presents an overview of the system and demonstrates the ability to precisely deliver consistent warnings and perturbations during gait. User study results highlight effectiveness of visual and audio warnings to help users brace for impact, resulting in guidelines that will inform future human-robot warning systems.
△ Less
Submitted 27 July, 2022;
originally announced July 2022.
-
DULA and DEBA: Differentiable Ergonomic Risk Models for Postural Assessment and Optimization in Ergonomically Intelligent pHRI
Authors:
Amir Yazdani,
Roya Sabbagh Novin,
Andrew Merryweather,
Tucker Hermans
Abstract:
Ergonomics and human comfort are essential concerns in physical human-robot interaction applications. Defining an accurate and easy-to-use ergonomic assessment model stands as an important step in providing feedback for postural correction to improve operator health and comfort. Common practical methods in the area suffer from inaccurate ergonomics models in performing postural optimization. In or…
▽ More
Ergonomics and human comfort are essential concerns in physical human-robot interaction applications. Defining an accurate and easy-to-use ergonomic assessment model stands as an important step in providing feedback for postural correction to improve operator health and comfort. Common practical methods in the area suffer from inaccurate ergonomics models in performing postural optimization. In order to retain assessment quality, while improving computational considerations, we propose a novel framework for postural assessment and optimization for ergonomically intelligent physical human-robot interaction. We introduce DULA and DEBA, differentiable and continuous ergonomics models learned to replicate the popular and scientifically validated RULA and REBA assessments with more than 99% accuracy. We show that DULA and DEBA provide assessment comparable to RULA and REBA while providing computational benefits when being used in postural optimization. We evaluate our framework through human and simulation experiments. We highlight DULA and DEBA's strength in a demonstration of postural optimization for a simulated pHRI task.
△ Less
Submitted 6 May, 2022;
originally announced May 2022.
-
Ergonomically Intelligent Physical Human-Robot Interaction: Postural Estimation, Assessment, and Optimization
Authors:
Amir Yazdani,
Roya Sabbagh Novin,
Andrew Merryweather,
Tucker Hermans
Abstract:
Ergonomics and human comfort are essential concerns in physical human-robot interaction. Common practical methods in the area either fail in estimating the correct posture due to occlusion or suffer from inaccurate ergonomics models in performing postural optimization. We propose a novel alternative framework for posture estimation, assessment, and optimization for ergonomically intelligent physic…
▽ More
Ergonomics and human comfort are essential concerns in physical human-robot interaction. Common practical methods in the area either fail in estimating the correct posture due to occlusion or suffer from inaccurate ergonomics models in performing postural optimization. We propose a novel alternative framework for posture estimation, assessment, and optimization for ergonomically intelligent physical human-robot interaction. We show that we can estimate human posture solely from the trajectory of the interacting robot with median deviation of 5 deg from motion capture. We propose DULA, a differentiable ergonomics assessment tool with 99.73% accuracy comparing to RULA. We use DULA in postural optimization for physical human-robot interaction tasks such as co-manipulation and teleoperation. We evaluate our framework through human and simulation experiments.
△ Less
Submitted 7 October, 2021; v1 submitted 12 August, 2021;
originally announced August 2021.
-
DULA: A Differentiable Ergonomics Model for Postural Optimization in Physical HRI
Authors:
Amir Yazdani,
Roya Sabbagh Novin,
Andrew Merryweather,
Tucker Hermans
Abstract:
Ergonomics and human comfort are essential concerns in physical human-robot interaction applications. Defining an accurate and easy-to-use ergonomic assessment model stands as an important step in providing feedback for postural correction to improve operator health and comfort. In order to enable efficient computation, previously proposed automated ergonomic assessment and correction tools make a…
▽ More
Ergonomics and human comfort are essential concerns in physical human-robot interaction applications. Defining an accurate and easy-to-use ergonomic assessment model stands as an important step in providing feedback for postural correction to improve operator health and comfort. In order to enable efficient computation, previously proposed automated ergonomic assessment and correction tools make approximations or simplifications to gold-standard assessment tools used by ergonomists in practice. In order to retain assessment quality, while improving computational considerations, we introduce DULA, a differentiable and continuous ergonomics model learned to replicate the popular and scientifically validated RULA assessment. We show that DULA provides assessment comparable to RULA while providing computational benefits. We highlight DULA's strength in a demonstration of gradient-based postural optimization for a simulated teleoperation task.
△ Less
Submitted 14 July, 2021;
originally announced July 2021.
-
Optimizing Hospital Room Layout to Reduce the Risk of Patient Falls
Authors:
Sarvenaz Chaeibakhsh,
Roya Sabbagh Novin,
Tucker Hermans,
Andrew Merryweather,
Alan Kuntz
Abstract:
Despite years of research into patient falls in hospital rooms, falls and related injuries remain a serious concern to patient safety. In this work, we formulate a gradient-free constrained optimization problem to generate and reconfigure the hospital room interior layout to minimize the risk of falls. We define a cost function built on a hospital room fall model that takes into account the suppor…
▽ More
Despite years of research into patient falls in hospital rooms, falls and related injuries remain a serious concern to patient safety. In this work, we formulate a gradient-free constrained optimization problem to generate and reconfigure the hospital room interior layout to minimize the risk of falls. We define a cost function built on a hospital room fall model that takes into account the supportive or hazardous effect of the patient's surrounding objects, as well as simulated patient trajectories inside the room. We define a constraint set that ensures the functionality of the generated room layouts in addition to conforming to architectural guidelines. We solve this problem efficiently using a variant of simulated annealing. We present results for two real-world hospital room types and demonstrate a significant improvement of 18% on average in patient fall risk when compared with a traditional hospital room layout and 41% when compared with randomly generated layouts.
△ Less
Submitted 8 January, 2021;
originally announced January 2021.
-
Risk-Aware Decision Making in Service Robots to Minimize Risk of Patient Falls in Hospitals
Authors:
Roya Sabbagh Novin,
Amir Yazdani,
Andrew Merryweather,
Tucker Hermans
Abstract:
Planning under uncertainty is a crucial capability for autonomous systems to operate reliably in uncertain and dynamic environments. The concern of safety becomes even more critical in healthcare settings where robots interact with human patients. In this paper, we propose a novel risk-aware planning framework to minimize the risk of falls by providing a patient with an assistive device. Our appro…
▽ More
Planning under uncertainty is a crucial capability for autonomous systems to operate reliably in uncertain and dynamic environments. The concern of safety becomes even more critical in healthcare settings where robots interact with human patients. In this paper, we propose a novel risk-aware planning framework to minimize the risk of falls by providing a patient with an assistive device. Our approach combines learning-based prediction with model-based control to plan for the fall prevention task. This provides advantages compared to end-to-end learning methods in which the robot's performance is limited to specific scenarios, or purely model-based approaches that use relatively simple function approximators and are prone to high modeling errors. We compare various risk metrics and the results from simulated scenarios show that using the proposed cost function, the robot can plan interventions to avoid high fall score events.
△ Less
Submitted 25 March, 2021; v1 submitted 15 October, 2020;
originally announced October 2020.
-
Development of a Novel Computational Model for Evaluating Fall Risk in Patient Room Design
Authors:
Roya Sabbagh Novin,
Ellen Taylor,
Tucker Hermans,
Andrew Merryweather
Abstract:
Objectives: The aims of this study are to identify factors in physical environments that contribute to patient falls in hospitals and to propose a computational model to evaluate patient room designs.
Background: The existing fall risk assessment tools have an acceptable level of sensitivity and specificity, however, they only consider intrinsic factors and medications, making the prediction ver…
▽ More
Objectives: The aims of this study are to identify factors in physical environments that contribute to patient falls in hospitals and to propose a computational model to evaluate patient room designs.
Background: The existing fall risk assessment tools have an acceptable level of sensitivity and specificity, however, they only consider intrinsic factors and medications, making the prediction very limited in terms of how the physical environment contributes to fall risk.
Methods: We provide a computational model for risk of fall based on physical-environment and patient-motion factors. We use a trajectory optimization approach for patient motion prediction.
Results: We run the proposed model on four room designs as examples of various room design categories. Results show the capabilities of the proposed model in identifying risky locations within the room.
Conclusions: Our study shows the potential capabilities of the proposed model. Due to lack of enough evidence for the examined factors, it is not possible at this point to gain robust confidence in the final evaluations. More studies using quantitative, relational, or causal designs are recommended to inform the proposed model for patient falls.
Application: Develo** a comprehensive fall risk model is a significant step in understanding and solving the problem of patient falls in hospitals. It can provide guidance for healthcare decision makers to optimize effective interventions to reduce risk of falls while promoting safe patient mobility in the hospital room environment. We can also use it in healthcare technologies such as assistive robots to provide informed assistance.
△ Less
Submitted 28 August, 2020; v1 submitted 20 August, 2020;
originally announced August 2020.
-
Is The Leader Robot an Adequate Sensor for Posture Estimation and Ergonomic Assessment of A Human Teleoperator?
Authors:
Amir Yazdani,
Roya Sabbagh Novin,
Andrew Merryweather,
Tucker Hermans
Abstract:
Ergonomic assessment of human posture plays a vital role in understanding work-related safety and health. Current posture estimation approaches face occlusion challenges in teleoperation and physical human-robot interaction. We investigate if the leader robot is an adequate sensor for posture estimation in teleoperation and we introduce a new probabilistic approach that relies solely on the trajec…
▽ More
Ergonomic assessment of human posture plays a vital role in understanding work-related safety and health. Current posture estimation approaches face occlusion challenges in teleoperation and physical human-robot interaction. We investigate if the leader robot is an adequate sensor for posture estimation in teleoperation and we introduce a new probabilistic approach that relies solely on the trajectory of the leader robot for generating observations. We model the human using a redundant, partially-observable dynamical system and we infer the posture using a standard particle filter. We compare our approach with postures from a commercial motion capture system and also two least-squares optimization approaches for human inverse kinematics. The results reveal that the proposed approach successfully estimates human postures and ergonomic risk scores comparable to those estimates from gold-standard motion capture.
△ Less
Submitted 19 March, 2021; v1 submitted 24 February, 2020;
originally announced February 2020.
-
A Model Predictive Approach for Online Mobile Manipulation of Nonholonomic Objects using Learned Dynamics
Authors:
Roya Sabbagh Novin,
Amir Yazdani,
Andrew Merryweather,
Tucker Hermans
Abstract:
A particular type of assistive robots designed for physical interaction with objects could play an important role assisting with mobility and fall prevention in healthcare facilities. Autonomous mobile manipulation presents a hurdle prior to safely using robots in real life applications. In this article, we introduce a mobile manipulation framework based on model predictive control using learned d…
▽ More
A particular type of assistive robots designed for physical interaction with objects could play an important role assisting with mobility and fall prevention in healthcare facilities. Autonomous mobile manipulation presents a hurdle prior to safely using robots in real life applications. In this article, we introduce a mobile manipulation framework based on model predictive control using learned dynamics models of objects. We focus on the specific problem of manipulating legged objects such as those commonly found in healthcare environments and personal dwellings (e.g. walkers, tables, chairs). We describe a probabilistic method for autonomous learning of an approximate dynamics model for these objects. In this method, we learn dynamic parameters using a small dataset consisting of force and motion data from interactions between the robot and object. Moreover, we account for multiple manipulation strategies by formulating the manipulation planning as a mixed-integer convex optimization. The proposed framework considers the hybrid control system comprised of i) choosing which leg to grasp, and ii) control of continuous applied forces for manipulation. We formalize our algorithm based on model predictive control to compensate for modeling errors and find an optimal path to manipulate the object from one configuration to another. We show results for several objects with various wheel configurations. Simulation and physical experiments show that the obtained dynamics models are sufficiently accurate for safe and collision-free manipulation. When combined with the proposed manipulation planning algorithm, the robot successfully moves the object to a desired pose while avoiding collision.
△ Less
Submitted 10 November, 2020; v1 submitted 19 December, 2019;
originally announced December 2019.
