Description
Week 7 DiscussionSelect one (1) of the three (3) published articles that was approved in week 4.Post the title of the article, authors, purpose, and type of study: Quantitative, Qualitative, or Systematic Review.Discuss how it might influence your practice. What changes to your practice would you recommend based on the article?
Unformatted Attachment Preview
JAMIA Open, 6(1), 2023, ooad015
https://doi.org/10.1093/jamiaopen/ooad015
Research and Applications
Research and Applications
Christopher Ryan King
1
, Ayanna Shambe1,2, and Joanna Abraham
1,3
1
Department of Anesthesiology, Washington University School of Medicine, Washington University in St. Louis, St. Louis, Missouri, USA, 2Saint Louis University School of Medicine, St. Louis, Missouri, USA and 3Institute for Informatics, Washington University in St. Louis, St. Louis, Missouri, USA
Corresponding Author: Christopher Ryan King, Department of Anesthesiology, Washington University School of Medicine,
Washington University in St. Louis, 660 S. Euclid Ave, MSC 8054-50-02, St. Louis, MO 63110, USA;
christopherking@wustl.edu
Received 6 September 2021; Revised 22 February 2023; Editorial Decision 25 February 2023; Accepted 27 February 2023
ABSTRACT
Objective: Situational awareness and anticipatory guidance for nurses receiving a patient after surgery are keys
to patient safety. Little work has defined the role of artificial intelligence (AI) to support these functions during
nursing handoff communication or patient assessment. We used interviews to better understand how AI could
work in this context.
Materials and Methods: Eleven nurses participated in semistructured interviews. Mixed inductive-deductive
thematic analysis was used to extract major themes and subthemes around roles for AI supporting postoperative nursing.
Results: Five themes were generated from the interviews: (1) nurse understanding of patient condition guides
care decisions, (2) handoffs are important to nurse situational awareness, but multiple barriers reduce their
effectiveness, (3) AI may address barriers to handoff effectiveness, (4) AI may augment nurse care decision
making and team communication outside of handoff, and (5) user experience in the electronic health record
and information overload are likely barriers to using AI. Important subthemes included that AI-identified
problems would be discussed at handoff and team communications, that AI-estimated elevated risks would
trigger patient re-evaluation, and that AI-identified important data may be a valuable addition to nursing
assessment.
Discussion and Conclusion: Most research on postoperative handoff communication relies on structured checklists. Our results suggest that properly designed AI tools might facilitate postoperative handoff communication
for nurses by identifying specific elevated risks faced by a patient, triggering discussion on those topics. Limitations include a single center, many participants lacking of applied experience with AI, and limited participation
rate.
Key words: artificial intelligence, postoperative nursing, PACU, handoffs, situational awareness
Lay Summary
Nurses caring for patients after surgery make many decisions about what complications to look for and how to treat issues
that arise. They rely on handoffs from prior clinicians to understand the patient’s background, relevant events, and care
plans so far. We interviewed nurses to ask if and how artificial intelligence (AI) might help them focus their handoff
C The Author(s) 2023. Published by Oxford University Press on behalf of the American Medical Informatics Association.
V
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
1
Downloaded from https://academic.oup.com/jamiaopen/article/6/1/ooad015/7079073 by guest on 13 September 2023
Potential uses of AI for perioperative nursing handoffs: a
qualitative study
2
JAMIA Open, 2023, Vol. 6, No. 1
communication on likely problems and generally understand the patient. Our participants stated that if AI identified likely
issues, they would discuss those topics in handoff, communicate about those problems with physicians, and modify their
monitoring and treatment to the level of risk faced by the patient. This finding runs against most research on improving
communication, which focuses on fixed checklists of topics to discuss. Most uses of AI for nurses focus on making specific
to-do recommendations and documentation reminders, but we find that nurses would benefit from AI which focuses more
on their understanding of the patient’s condition.
BACKGROUND AND SIGNIFICANCE
Downloaded from https://academic.oup.com/jamiaopen/article/6/1/ooad015/7079073 by guest on 13 September 2023
Inpatient handoffs are the transfer of responsibility, information,
and control between clinicians or teams. Incomplete or inaccurate
handoffs are a source of subsequent medical errors and patient
injury,1–3 particularly for patients undergoing major surgery.4–7 We
focus on postoperative nurse handoffs during surgical patient transfers from the operating room (OR) to the postanesthesia care unit
(PACU) and from the PACU to inpatient ward. Handoffs are important for receiving nurses to understand the patient’s situation
because residual sedation, pain, delirium, fatigue, and surgical injuries can make patient-nurse communication difficult. Additionally,
the patient’s context changes; surgery eliminates some concerns
and creates the opportunity for new complications. The data
surrounding surgical patients are voluminous and diverse while
simultaneously incomplete, which strains the ability of receiving
nurses to review and assimilate it de novo.8–10 Two functions of
handoff are of special interest to us: situational awareness and
anticipatory guidance. Situational awareness is the combination of
perceiving critical factors in the environment, understanding what
those factors mean for the clinician’s goals, and understanding what
will happen next.11 Anticipatory guidance is the communication of
likely patient status changes and plans for how to address them.12,13
These 2 functions support early recognition and coordinated treatment of complications, which have substantial effects reducing postoperative mortality and morbidity. 14 Major handoff quality
improvement projects have integrated both of these concepts.15–17
Protocols and checklists are employed to ensure that key information is transmitted during handoffs throughout healthcare.18–20
Some electronic health records (EHRs) have integrated standardized
handoffs,21 including nurse-to-nurse handoffs15,22 and perioperative
nursing handoffs specifically.23,24 Nevertheless, handoff-related
information gaps are common for postoperative patients.10,25–28
The EHR has promise for mitigating and reducing these information gaps. EHRs place an enormous amount of data at the fingertips
of all clinicians. In theory, this ought to allow a nurse to prepare for
handoff and recover from an incomplete handoff. Dashboard-type
displays can be used during handoffs for this summary function. 29
Despite this promise, most handoff-EHR integration work does not
focus on the critical functions of situational awareness and anticipatory guidance.30 Staggers et al31 found that existing EHR handoff
summaries were too rigid and incomplete to be useful; additionally,
they interfered with the receiving nurse’s encoding of information
via note taking. They subsequently found that nurses made little use
of EHR handoff support due to these limitations.32 Calculations and
displays of EHR data can be viewed as sense-making, with tension
between different purposes and users.33
Artificial intelligence (AI) integrated into EHRs is an exciting,
related development. AI is a broad term, including all computer programing which replicates or imitates cognitive functions. The most
common approach applying AI to EHR data for nursing is supervised machine learning (ML), in which algorithms use EHR data as
inputs to predict unknown or unrecorded characteristics of a
patient, such as future adverse events, current patient condition, or
undocumented comorbidities.34 Although often discussed
exchangeably, ML (an approach to pattern recognition) and clinical
decision support (CDS) (applying pattern recognition to suggest
actions or documentation) are conceptually different. For a given
AI/ML pattern recognition tool, a wide variety of uses cases, visualizations, and user interfaces are possible. AI using EHR data has
become much more general and accurate in the last few years,35,36
allowing prediction of perioperative events37–41 and learning effective treatment strategies.42 AI is able to interpret nursing documentation to recognize patient types and predict clinical
deterioration.43–47 Research has explored AI/ML in several roles to
augment the capabilities of bedside nurses, including identifying
care needs or predicting adverse events based on EHR data, scheduling and equipment management, patient activity tracking, processing nursing documentation for transitions of care, quantifying risks
in family discussions, and interactive patient education.34,48–50 For
example, ML identification of patients with a high risk of pressure
ulcers51,52 or falls53 can trigger CDS for nursing interventions. The
related CDS literature for nurses has focused on recommending specific actions based on scoring systems and expert-devised rules.54 In
addition to predicting adverse events, AI/ML models can flag important data for review. While information dashboards have long been
integrated into EHRs with expert-driven rules for abnormal
data,31,32,55–57 contemporary systems include AI/ML models to
identify “relevant” patient data.58–60
Very few AI studies have gone beyond initial development phases
or shown benefits to stakeholders,49,50 and the more developed usecases are often highly specialized, such as rapid-response-team
alarms. 48 Expanding nursing engagement in design of AI projects is
a recognized priority,61 as very few AI or information system studies
involve nurses at early stages.50,62
A handful of studies have considered the impact of AI
prediction in augmenting handoff communication. In the neonatal
ICU context, Hunter et al63 used natural-language generation to
summarize EHR data and generate potential problems and
care plans in a dynamic shift-change report. Forbes and colleagues56,64 envisioned a dynamic EHR integrated shift-report summary for nurses including key data, diagnoses, and predicted
adverse events. Hunter and Forbes’s work56,63,64 suggests a distinct
role for AI prediction from traditional CDS: facilitating problembased report and assessment during handoffs. Although clinician
assessment of the patient’s condition is a key part of all structured
handoffs, AI identification of likely complications and important
data integrated into dynamic “handoff sheets” could supplement
handoff assessment more flexibly than traditional checklist-based
protocols.
We previously explored related ideas at the OR to intensive care
unit handoff, which often has a brief nurse-to-nurse component due
to the multidisciplinary nature of the handoff.65,66 Key findings of
that study were the difficulty of making EHR information universally accessible, the need to focus on AI with direct relevance to
patient care, and general acceptance of blending AI risk prediction
with current summaries of patient data into a handoff tool.
JAMIA Open, 2023, Vol. 6, No. 1
However, the ICU shift-change and OR-ICU handoffs previously
studied are quite different from the OR-PACU-ward transition.
3
Preop holding
OR
OBJECTIVES
MATERIALS AND METHODS
Our research included 2 activities: direct observation of handoffs to
establish context in the research team and interviews with postoperative nurses to directly address the research questions.
Setting
Barnes-Jewish Hospital is a 1400-bed academic medical center in St
Louis, Missouri. We focused on the Acute and Critical Care Surgery
(ACCS) division, which performs approximately 1600 inpatient surgeries annually, primarily trauma, and acute abdominal surgery. All
postoperative patients (other than those directly admitted to intensive care) recover from anesthesia in the PACU, a 30-bed area. Four
hospital units subsequently care for ACCS patients: 2 dedicated hospital wards and 2 high-dependency units. The high-dependency units
are shared with otolaryngology, abdominal organ transplant, and
hepatobiliary services.
Observations
Researchers selected surgical cases for direct observation from the
OR schedule based on the primary surgery service (ACCS). We also
included patients likely to be admitted to high-dependency units
based on their procedures. We attempted observation on all cases
meeting these criteria between 9 AM and 5 PM on weekdays.
Researchers conducted direct observations under Washington University IRB approval (#201812137 and #202009066) with the consent of the PACU nurse to shadow their interactions with other
clinicians (OR circulator nurse, anesthesia clinician, surgery clinician, and wards nurse) and recorded notes following a structured
outline.67 The IRB approved verbal consents with electronic provision of study information as a replacement for written consents during the coronavirus disease 2019 pandemic. Because we performed
these observations to provide interpretative context for interview
analysis rather than directly answer study questions, we do not separately report findings from observations. We include this description
only to report the nurse participant recruitment process.
Intraop team (anesthesia, surgery, circulator)
gives report and preop nursing sheet to PACU
nurse. Detailed protocol.
PACU
Wards
PACU nurse gives phone or bedside handoff to
wards nurse. Preop sheet included in paper
chart. Partial protocol.
Figure 1. Illustration of perioperative handoff stages.
Description of perioperative handoff processes and
care teams
Figure 1 illustrates the handoff process. Prior to surgery, a preoperative holding area nurse completes a health status inventory in the
Epic EHR and on a paper record (Supplementary Appendix S1)
which is passed to PACU. The preoperative nurse and OR circulating nurse complete an informal handoff. After surgery, a surgery resident or fellow, the OR circulating nurse, and an anesthesia clinician
transport the patient to PACU. OR to PACU handoff follows a protocol (Supplementary Appendix S1), where the circulating nurse,
surgeon, and anesthetist each give handoff to the PACU nurse. The
handoff sheet (Supplementary Appendix S1), consent documents,
and backup records from surgical implants, and blood transfusions
are the only common paper records. All other documentation is
electronic.
Once PACU staff and the supervising anesthesiologist deem a
patient ready to leave the PACU, the PACU nurse gives handoff to
the ward nurse either at the bedside (high dependency unit) or by
phone call (ward units). A guideline addresses the handoff between
PACU and the wards nurses (Supplementary Appendix S1). Fellows,
resident physicians, nurse practitioners, and the attending surgeon
jointly manage postoperative patients. The nurse practitioner or resident physician implementing ward care is not directly involved in
the surgery. We refer to that resident or nurse practitioner as the
midlevel clinician.
Interview participants and data collection
Concurrently with our direct observations, we recruited a convenience sample of nurses from the PACU, ACCS wards, and highdependency units. We chose interviews instead of focus groups to
allow us to hear multiple independent perspectives, and for pragmatic reasons. During the study period, nurse participants faced
high workloads, making scheduling focus groups difficult. We conducted interviews under Washington University IRB approval
(#201812137 and #202009066) with the consent of the participant.
Authors King and Shambe conducted interviews using the same
guide (Supplementary Appendix S2). The content of the interviews
focused on handoff communication, patient assessment, physician
communication, and potential roles for AI. We conducted interviews
over the phone or voice application with audio recording, which
was transcribed verbatim.
Analysis
Two researchers (King and Shambe) double-coded interviews using
a mixed inductive-deductive reflexive thematic analysis approach.
First, we familiarized ourselves with the data by reviewing the transcripts and fragmenting them into topical sections. Second,68 we
Downloaded from https://academic.oup.com/jamiaopen/article/6/1/ooad015/7079073 by guest on 13 September 2023
Although direct experimentation with implementing AI support for
perioperative handoffs would be informative, we set out to establish
a use-case with clinicians and refine what content would be useful
for clinicians prior to implementation. We identified 3 unanswered
preliminary questions in prior research about postoperative bedside
nurses as givers or receivers of handoff which we aim to address: (1)
would postoperative nurses accept AI recommendations for handoff
topics? (2) would nurses find AI-based predictions of adverse events
useful and relevant? (3) would a single presentation of AI-based predictions be acceptable to most nurses? The goal of this single-center
qualitative study was to explore these topics and how AI added to a
handoff workflow might fit into the situational awareness, assessment, monitoring, and communication goals of postanesthesia care
unit (PACU) and postoperative ward nurses. We intend these findings to guide subsequent design and implementation efforts, but we
did not evaluate a specific AI product or technical implementation.
Preop nurse assessment recorded to paper,
provided to circulating nurse. No protocol.
4
RESULTS
We conducted 11 total interviews: 7 PACU nurses and 4 ward
nurses. Supplementary Table S5 (Supplementary Appendix S5) displays the 5 major themes in our findings, subthemes, and exemplar
quotations of each subtheme: (1) nurse understanding of patient
condition guides care decision; (2) handoffs are important to nurse
situational awareness, but multiple barriers reduce their effectiveness; (3) AI may address barriers to handoff effectiveness; (4) AI
may augment nurse care decision making and team communication
outside of handoff; and (5) EHR user experience and information
overload are likely barriers to using AI during handoffs. These
themes had substantial interactions, and with each subtheme, we
note closely related subthemes. Supplementary Table S5 shows the
relevance to OR-PACU, PACU-ward, or both handoffs of each subtheme along with number of interviews referencing each.
Nurse understanding of patient condition guides care
decisions
Participants stressed that their bedside presence allowed rapid detection and hopefully mitigation of complications. They universally
agreed that their understanding of the issues facing a patient modified what signs and symptoms they were alert for (Subtheme 1.b),
what issues they communicated to the PACU or midlevel clinician
(Subtheme 1.c), and what treatments they recommended. Several
participants stated that although almost all treatment changes
required a team discussion, their recommendations were likely to be
considered or acted on.
Handoffs are important to nurse situational awareness,
but multiple barriers reduce their effectiveness
Participants stressed that accurate handoff was a critical way to
learn about the patient’s state, expectations for recovery, and needs
in the high-turnover environment of PACU (Subtheme 2.a).
However, they acknowledged barriers where the documentation
they relied on was incomplete (Subtheme 2.d), the handoff-giver did
not know the relevant information, or they did not understand what
needed to be conveyed. Participants agreed that problem-focused
handoffs with anticipatory guidance were extremely useful, but that
many topics in handoffs were not relevant or recited data without
context (Subtheme 2.b). Closely related to this concern was a lack of
shared priorities between the handoff giver and receiver. It was frequent for participants to describe receiving handoffs focusing on
details they found to be irrelevant or unintelligible, and for handoff,
participants to not value topics on which their counterparty asked
questions (Subtheme 2.c).
AI may address barriers to handoff effectiveness
Several participants commented on how AI risk prediction at handoff might mitigate mismatch between handoff givers and receivers.
First, almost all participants agreed that if AI identified a patient at
high risk for a complication, that this topic would be prioritized for
discussion at handoff, and that those receiving handoff would ask
follow-up questions regarding the patient state and the current plan
(Subtheme 3.a). Second, a high calculated risk could alert them that
a known comorbidity was more severe than they expected (Subtheme 3.b), which was information frequently absent from documentation. Third, awareness that a patient was overall high-risk
would prompt nurses to closely review all available data and prioritize shared careful patient evaluation (Subtheme 3.c). Finally, automatic identification of EHR data elements which increased the
patients’ risk could mitigate data omissions, especially if that data
was in an unusual location (Subtheme 3.d). Although several participants gave examples of how they might relate data given at handoff
to specific AI-identified problems (ameliorating the laundry-list type
handoff of Subtheme 2.c), none explicitly identified using the AIidentified problems to organize data.
AI may augment nurse care decision-making and team
communication outside of handoff
PACU handoff is a critical time for establishing joint plans and midlevel clinician communication needs; however, posthandoff communication was also regarded as important. Ward participants noted
that midlevel clinicians rarely proactively contacted them, leaving
nurses to deduce what issues required communication or nursing
action (Subtheme 4.a). Some participants noted that AI could help
target posthandoff nurse-midlevel communication in 2 ways. First,
if a patient had been identified as high risk, the resistance to contacting the midlevel clinicians to discuss that topic would be lowered
(Subtheme 4.b). Second, the nurse’s holistic view of patient risk
might be difficult to communicate, and AI-based pattern matching
would make this more concrete and easier to request midlevel clinicians act on or personally evaluate.
Participants noted incomplete midlevel clinician documentation
and other EHR information negatively affected their independent
assessment of the patient (Subtheme 4.c). AI identification of alternative key data would then be valuable. Additionally, AI-identified
risks for adverse events would allow the nurse to better target their
assessment and monitoring independent of any effect on handoff
(Subtheme 4.d). Participants noted that AI-identified elevated risks
could allow them to target interventions within their scope of practice, such as fall prevention, delirium prevention, and pneumonia
prevention (Subtheme 4.d). Multiple participants endorsed the
desire for more accurate prediction of patients likely to require
Downloaded from https://academic.oup.com/jamiaopen/article/6/1/ooad015/7079073 by guest on 13 September 2023
organically generated open codes after the first review. We applied
deductive codes based on relevance to major study questions (listed
in Supplementary Appendix S3). We labeled each statement as relevant to OR-PACU or PACU-ward handoffs based on the surrounding context. Next, the coders discussed the set of open codes and
resolved conflicts by consensus. We generated initial subthemes
from groups of related codes. We then compared OR-PACU and
PACU-ward coded data for similar subthemes that could be coalesced. We did not formalize a codebook, but we returned to the
raw statements for consistency with the subthemes and examined
them for relationships to other identified subthemes. We then jointly
refined subthemes based on recoded data and clustered subthemes
into themes based on connecting stories. At each stage, coders compared codes and resolved disagreements. The coders and a third
researcher (Abraham) reviewed and revised themes. After the construction of the coding tree, coders checked statements to validate
their applicability to the higher-level themes. After 10 interviews, we
completed a first round of coding, and we found that most topics
were addressed by multiple participants, meaning that saturation
was likely; we found no new topics during analysis of the 11th interview and stopped recruitment.
Supplementary Appendix S4 is a consolidated criterion for
reporting qualitative research (COREQ) checklist,69 a qualitative
research reporting framework, along with some additional methods
details.
JAMIA Open, 2023, Vol. 6, No. 1
JAMIA Open, 2023, Vol. 6, No. 1
higher nursing workload or ICU transfer, which they could use to
allocate their resources.
EHR user experience and information overload are
likely barriers to using AI
DISCUSSION
Our interviews highlighted the importance of team communication
and anticipatory guidance at and around postoperative handoffs for
nurses to optimize patient care. The data gave consistent answers to
our knowledge-gap questions:
1. Would postoperative nurses accept AI recommendations for
handoff topics? Yes, participants believed that AI which identified
patients at elevated risk would lead to focused handoff communication and physician-nurse team communication on those topics,
increasing anticipatory guidance and situational awareness.
Nurses overall expressed little hesitance to include AI-estimated
risks in their handoff assessments.
2. Would nurses find AI-based predictions of adverse events useful
and relevant? Yes, participants believed that well-functioning AI
risk assessment would lead to activating nurse-driven interventions, allocating resources (such as high-dependency beds) more
efficiently, and prioritizing monitoring for higher-risk outcomes.
To accomplish this, participants desired both overall measures of
acuity and estimation of a broad collection of risks.
3. Would a single presentation of AI-based predictions be acceptable
to most nurses? No, participants acknowledged diverse methods
of using the EHR, and diverse preferences for information presentation. While our participants were enthusiastic for AI identification of relevant information in the EHR, they also acknowledged
barriers surrounding the user experience of adding AI to their
workflows and the potential for information overload. The ability
to easily integrate AI into multiple EHR workflows and choose a
personalized presentation will be necessary for it to succeed.
Our work contrasts with much of the development of EHR AI
support for nurses,54 which largely focuses on medication documentation, medication administration, and very simple rule-based systems to identify specific nursing needs. Our work also highlights the
need for handoff communication to adapt to the patient’s condition,
contrasting with the dominant theme of the literature for improving
handoffs: standardized communication and checklists.70 Several
small studies from other nursing contexts have found similar
themes. Home care nurses in a prior study expressed a similar use
case for AI to modify the intensity of their services but did not discuss its role in transitions of care.71 User-design work for EHR-
integrated shift-change handoff support had similar ideas, arriving
at a design which blended data and predictive risks.56,64 Although
their work stemmed from interactions with nurses and nursing students, their manuscripts do not give enough methods details to further explore similarities with our work. Nurse users largely accepted
a prototype system for shift change in the neonatal intensive care
unit which focused on summarizing data in natural language and
included expert decision rules as a minor component.63
Our findings can also be related to work with dashboards
intended to detect change in patient status which lack explicit AI
predictions.54 In our work on OR to ICU handoffs,66 participants
endorsed similar desires to integrate AI into summaries of patient
data like laboratory results and vital signs and the need to focus on
actionability. In contrast to ICU participants, our participants felt
that AI augmentation of handoff topics could be useful, AI assessment of risks for midlevel clinician communication would be valuable, and that AI could assist their selection of necessary patient
assessment steps. Very recently, experience with risk-predicting AI
suggests that it facilitates a shared mental model and coordination
across disciplines by providing a reference point for patient status,72
including using this shared reference point for escalation of care.73
Our participants echoed this idea in Subtheme 4.b.
Similar to others,73–75 we found that extraction of directly interpretable patient data and actionable needs was a high priority (Subtheme 4.d). Prior work has also found that nurses more frequently
use a “bottom-up” (data and needs first) approach to patient summarization,76 which agrees with our finding of specific riskincreasing data and conditions being important for handoff support
(Subtheme 3.c). Physicians and nurses rate explainability in terms of
patient data and personal understanding as highly related to trust in
AI;73,77 however, current methods of AI explainability have been
found to have limited usefulness in practice.78 Some implementation
studies have found that AI-based alerts are relatively more salient to
nurses than physicians in this regard.79 Imperative AI-based CDS
has been effective in some direct use cases, supporting this
approach,80 but it runs the risk of automation bias.81,82 Similar to
the findings of others,73 our participants indicated that they would
consider the AI as a suggestion of where to start an evaluation rather
than a prescriptive mandate (Subtheme 4.d).
Taken together, our findings and these prior studies suggest that
AI can support nurses in their more general cognitive tasks, and that
future AI design efforts should (1) target critical moments of evaluation like shift change and handoff and (2) incorporate estimates of
acuity, condition severity, and influential data outside narrow
“nursing related” problems. We anticipate that an adaptive handoff
sheet design like Hunter and Forbes’s work56,63,64 containing automated identification of problems relevant to each patient and data
pertinent to those problems will emerge from further research with
this population and ongoing technical testing. This optimism is
restrained by the many practical implementation difficulties that
plague clinical AI,83 which was echoed in the concerns of our participants (Subtheme 5).
Limitations
Our study drew participants from a single center, which limits the
range of experiences and exposure to alternative EHRs. The ward
nurses worked in a small number of units, limiting the generalizability. The number of participants and recruitment rate from those
potentially eligible were both low. The participants had limited
experience with AI, which limits the reliability of the findings. The
Downloaded from https://academic.oup.com/jamiaopen/article/6/1/ooad015/7079073 by guest on 13 September 2023
Participants identified several barriers for nursing use of AI, largely
centered around the user experience and the potential for excessive
information volume. First, because of the large number of different
methods for accomplishing most tasks in Epic, participants did not
recommend the same locations for viewing AI risk prediction. Second, preferred visualizations also differed between participants,
with participants variously endorsing absolute risk estimates, relative risks, simplified high-medium-low risk flags, and plots. Several
participants noted that existing clinical decision support and alerts
already generate alarm fatigue, and that additional flags would
likely be ignored unless they had high value (Subtheme 5.c). Finally,
participants noted the potential for information overload with more
complex outputs (Subtheme 5.d).
5
6
setting was an academic medical center, so the views may not reflect
the experiences of those outside this type of setting. Our interview
was semistructured, and participants were informed on the nature of
our study. They may have endorsed ideas to be agreeable, but participants seemed to feel free to disagree.
CONCLUSION
FUNDING
The National Center for Advancing Translational Sciences of the
National Institutes of Health under Award Number KL2 TR002346
(PI: Victoria J. Fraser). The content is solely the responsibility of the
authors and does not necessarily represent the official views of the
Nationa
