-
Design Principles Developed through User-Centered and Socio-Technical Methods Improve Clinician Satisfaction, Speed, and Confidence in Pharmacogenomic Clinical Decision Support
Authors:
Timothy M. Herr,
Therese A. Nelson,
Luke V. Rasmussen,
Yinan Zheng,
Nicola Lancki,
MPH,
Justin B. Starren
Abstract:
OBJECTIVE: To design and evaluate new pharmacogenomic (PGx) clinical decision support (CDS) alerts, built to adhere to PGx CDS design principles developed through socio-technical approaches.
MATERIALS AND METHODS: Based on previously identified design principles, we created 11 new PGx CDS alert designs and developed an interactive web application containing realistic clinical scenarios and user…
▽ More
OBJECTIVE: To design and evaluate new pharmacogenomic (PGx) clinical decision support (CDS) alerts, built to adhere to PGx CDS design principles developed through socio-technical approaches.
MATERIALS AND METHODS: Based on previously identified design principles, we created 11 new PGx CDS alert designs and developed an interactive web application containing realistic clinical scenarios and user workflows that mimicked a real-world EHR system. We recruited General Internal Medicine and Cardiology clinicians from Northwestern Medicine and recorded their interactions with the original and new designs. We measured clinician response, satisfaction, speed, and confidence through questionnaires and analysis of the recordings.
RESULTS: The study included 12 clinicians. Participants were significantly more satisfied (p=0.0000001), faster (p=0.009), and more confident (p<.05) with the new designs than the original ones. The study lacked statistical power to determine whether prescribing accuracy was improved, but participants were no less accurate, and clinical actions were more concordant with alert interactions (p=0.004) with the new designs. We found a significant learning curve associated with the original designs, which was eliminated with the new designs.
DISCUSSION: This study successfully demonstrates that socio-technical and user-centered design techniques can improve PGx CDS alert designs. Best practices for PGx CDS design are limited in the literature, with few effectiveness studies available. These results can help guide future PGx CDS implementations to be more clinician friendly and less time-consuming.
CONCLUSION: The results of this study support the PGx CDS design principles we proposed in previous work. As a next step, the new designs should be implemented in a live setting for further validation.
△ Less
Submitted 31 January, 2020;
originally announced February 2020.
-
Design Principles and Clinician Preferences for Pharmacogenomic Clinical Decision Support Alerts
Authors:
Timothy M. Herr,
Therese A. Nelson,
Justin B. Starren
Abstract:
OBJECTIVE: To better understand clinician needs and preferences for the display of pharmacogenomic (PGx) information in clinical decision support (CDS) tools.
MATERIALS AND METHODS: We developed a semi-structured interview to collect feedback and preferences in six key areas of PGx CDS design, from clinicians who had prior experience with live PGx CDS tools. Eight clinicians from Northwestern Me…
▽ More
OBJECTIVE: To better understand clinician needs and preferences for the display of pharmacogenomic (PGx) information in clinical decision support (CDS) tools.
MATERIALS AND METHODS: We developed a semi-structured interview to collect feedback and preferences in six key areas of PGx CDS design, from clinicians who had prior experience with live PGx CDS tools. Eight clinicians from Northwestern Medicine's (NM) General Internal Medicine clinic participated in the study.
RESULTS: Clinicians expressed preference for interruptive pop-up alerts during order entry, brief descriptions of relevant drug-gene interactions, and a clear and specific recommended alternative course of action when a medication is contraindicated. They did not wish to see detailed genetic data, preferring phenotypic information predicted from the genotype. Nor did they wish to be interrupted when genetic test results do not indicate a change in treatment plan. Clinicians reported little familiarity with Clinical Pharmacogenetic Implementation Consortium prescribing recommendations but reported trusting recommendations of their professional societies and resources like UpToDate. Analysis of unstructured comments concurred with structured results, indicating a general uncertainty among participants around how to interpret and apply PGx information in practice.
DISCUSSION: Results point to several underlying principles that can inform future PGx CDS alert designs: Be Specific and Actionable; Be Brief; Display Phenotypes not Genotypes; Rely on Sources Clinicians Already Trust; and, Be Adaptable to Learning Effects.
CONCLUSION: This study is part of a broader socio-technical design approach to PGx CDS design underway at NM and provides a baseline for future PGx CDS development. Designs based on these results have the potential to improve clinician education and adherence levels, and to improve patient outcomes.
△ Less
Submitted 31 January, 2020;
originally announced February 2020.
-
Characterizing Design Patterns of EHR-Driven Phenotype Extraction Algorithms
Authors:
Yizhen Zhong,
Luke Rasmussen,
Yu Deng,
Jennifer Pacheco,
Maureen Smith,
Justin Starren,
Wei-Qi Wei,
Peter Speltz,
Joshua Denny,
Nephi Walton,
George Hripcsak,
Christopher G Chute,
Yuan Luo
Abstract:
The automatic development of phenotype algorithms from Electronic Health Record data with machine learning (ML) techniques is of great interest given the current practice is very time-consuming and resource intensive. The extraction of design patterns from phenotype algorithms is essential to understand their rationale and standard, with great potential to automate the development process. In this…
▽ More
The automatic development of phenotype algorithms from Electronic Health Record data with machine learning (ML) techniques is of great interest given the current practice is very time-consuming and resource intensive. The extraction of design patterns from phenotype algorithms is essential to understand their rationale and standard, with great potential to automate the development process. In this pilot study, we perform network visualization on the design patterns and their associations with phenotypes and sites. We classify design patterns using the fragments from previously annotated phenotype algorithms as the ground truth. The classification performance is used as a proxy for coherence at the attribution level. The bag-of-words representation with knowledge-based features generated a good performance in the classification task (0.79 macro-f1 scores). Good classification accuracy with simple features demonstrated the attribution coherence and the feasibility of automatic identification of design patterns. Our results point to both the feasibility and challenges of automatic identification of phenoty** design patterns, which would power the automatic development of phenotype algorithms.
△ Less
Submitted 15 November, 2018;
originally announced November 2018.
-
Predictive Analytics Using Smartphone Sensors for Depressive Episodes
Authors:
Taeheon Jeong,
Diego Klabjan,
Justin Starren
Abstract:
The behaviors of patients with depression are usually difficult to predict because the patients demonstrate the symptoms of a depressive episode without a warning at unexpected times. The goal of this research is to build algorithms that detect signals of such unusual moments so that doctors can be proactive in approaching already diagnosed patients before they fall in depression. Each patient is…
▽ More
The behaviors of patients with depression are usually difficult to predict because the patients demonstrate the symptoms of a depressive episode without a warning at unexpected times. The goal of this research is to build algorithms that detect signals of such unusual moments so that doctors can be proactive in approaching already diagnosed patients before they fall in depression. Each patient is equipped with a smartphone with the capability to track its sensors. We first find the home location of a patient, which is then augmented with other sensor data to identify sleep patterns and select communication patterns. The algorithms require two to three weeks of training data to build standard patterns, which are considered normal behaviors; and then, the methods identify any anomalies in day-to-day data readings of sensors. Four smartphone sensors, including the accelerometer, the gyroscope, the location probe and the communication log probe are used for anomaly detection in slee** and communication patterns.
△ Less
Submitted 24 March, 2016;
originally announced March 2016.
