Description
4-5 paragraphing answering question below include questions in written literature. See uploaded documents and example; read and review all four journal articles/ literature provided. formulate a nursing evidence based PICOT (population/patient/problem, Intervention, Comparison, Outcome.
PICOT Question
What is the PICOT question being answered?
Background
Provide some background information about why this is important. Statistical findings.
Review of Literature
Outline the method for finding the evidence.
What key terms were identified in the literature research?
What criteria were used to determine the inclusion in the review?
Who Critiqued the articles?
Synthesis of Research Finding
Make some comparisons across the evidence.
What levels of evidence of present?
What kinds of samples and designs were used by the researchers?
What were the overall findings in relationship to the PICOT question?
Decision to Change Practice
What is the decision for practice based on the research in light of the PICOT question?
Is the sufficient evidence?
If not describe the implication for future studies.
Unformatted Attachment Preview
Evaluating the Effectiveness of a Fall Risk Screening Tool Implemented in an
Electronic Medical Record System
Yokota, Shinichiroh RN; Tomotaki, Ai MS, RN; Mohri, Ohmi RN; Endo, Miyoko
MS, CNA, RN; Ohe, Kazuhiko PhD, MD
Author Information
Departments of Healthcare Information Management (Messrs Yokota and Mohri) and Nursing (Ms Endo), The
University of Tokyo Hospital, Tokyo, Japan; National College of Nursing Japan, Tokyo, Japan (Ms Tomotaki); and
Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan (Dr
Ohe).
Correspondence: Shinichiroh Yokota, RN, Department of Healthcare Information Management, The University of
Tokyo Hospital, Hongo 7-3-1, Bunkyo-ku, Tokyo 113-8655, Japan (yokotas@hcc.h.u-tokyo.ac.jp).
Parts of this study were supported by JSPS KAKENHI grant no. 16K20977. The authors thank Dr Kosuke Kashiwabara
(University of Tokyo), for his advice about the analysis, and members of the Medical Safety Management Center
(University of Tokyo Hospital), for their cooperation.
The authors declare no conflicts of interest.
Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is
provided in the HTML and PDF versions of this article on the journal’s Web site (www.jncqjournal.com).
Accepted for publication: November 4, 2017
Published ahead of print: December 21, 2017
Abstract
We investigated the effect of using a fall risk screening tool in an electronic medical record system by using data for
25 039 patients in 24 general wards of a single institution. The probability of the occurrence of falls decreased after
the tool was implemented, but using the tool did not reduce the actual occurrence of falls. This indicates that we
must improve not only the assessment of the risk of falls but also the interventions to prevent falls.
FALLS can cause serious injuries, including fractures and brain contusions,1 and are the most common cause of
hospital admission, directly resulting in 14 million admissions in the United States.2 In addition, falls are associated
with a high cost per patient or per fall.3 It is therefore important that medical institutions take adequate fall
prevention measures.
One step in such fall prevention measures is the use of a fall risk screening tool, many of which have been
developed.4-7 In 2002 in Japan, the Japanese Nursing Association recommended the development of fall risk
screening tools for individual institutions. There was an existing tool in our institution, but in 2014, we improved
this conventional tool by developing a fall risk screening formula for inpatients based on the nursing assessment
records of about 10 000 inpatients stored in the electronic medical record (EMR) system of our institution. The
sensitivity of the tool was 72.1%, the specificity was 69.6%, and the area under the receiver operator curve was 0.777
(95% confidence interval [CI], 0.743-0.812).8 We implemented this improved tool in our EMR system and started
using it in March 2014.8 By using the information entered by nurses (eg, history of fall, risks regarding disease, and
risks regarding activities of daily living) and patient information stored in the EMR (eg, age and gender), the system
predicts whether the fall risk for an individual is high or low.
Several studies have evaluated the performance of existing fall risk screening tools in clinical settings,9-11 and one
study reported the sensitivity and specificity of a tool implemented in an EMR system.12 The primary objective of
introducing and using such fall risk screening tools is to reduce the occurrence of falls; however, there are few
reports on the effectiveness of using these tools on the actual occurrence of falls. One such study used simple
tabulation and a comparison of the occurrence of falls after introducing a screening tool.13 The aim of the present
study was to evaluate whether implementing and using the fall risk screening tool in our EMR system contributed to
a reduction in the occurrence of falls.
Back to Top
METHODS
Back to Top
Study design
We adopted a retrospective controlled before-after study design. The Japanese Ministry of Health, Labour and
Welfare guidelines state that the target of analysis in information systems related to medical safety must be evaluated
on the basis of the frequency of incidents and accidents.14 Therefore, we retrospectively evaluated the effectiveness
of our fall risk screening tool by analyzing whether it reduced the occurrence of falls. To define fall, we used the
following definition from the World Health Organization: “A fall is an event which results in a person coming to rest
inadvertently on the ground or floor or other lower level.”15 We excluded cases in which medical staff judged that a
patient had fallen from a piece of furniture such as a chair or bed, as well as cases with duplicate records. The
research ethics committee of the authors’ institution approved this study.
Back to Top
Data
This study used data from a single institution. We investigated the period before the screening tool was implemented
(pre-period) and the period after implementation (post-period). The pre-period was defined as from April 2013 to
February 2014, and the post-period was defined as from April 2014 to February 2015. These dates were chosen to
avoid any effect due to seasonal factors. We targeted patients who were admitted to and discharged from 24 general
wards during both the pre- and post-periods. The number of beds and the focus of each ward are provided as
Supplemental Digital Content (Table, available at: http://links.lww.com/JNCQ/A410). All wards were acute care
wards and had similar characteristics. One nurse is deployed per every 7 patients, and the nursing practice model is a
mixture of team nursing and primary nursing in these wards. We excluded pediatric, maternity, intensive care unit,
emergency, and psychiatric wards from analysis.
We had to take the risk factors of inpatients into consideration when analyzing the data to reduce the possibility that
a reduction in the occurrence of falls was due to an increase in the hospitalization of patients with mild conditions.
Previous studies have confirmed that “falls have diverse causes,”16 and “many complex patient characteristics,
circumstances, and activities may contribute to inpatient falls.”17 Risk factors adopted in 2 of the most commonly
used tools include The Morse Fall Scale 4-history of falls, secondary diagnosis, ambulatory aid, intravenous therapy,
gait, and mental status-and St Thomas’s Risk Assessment Tool in Falling Elderly Inpatients 5-history of falls,
agitation, visual impairment, frequent toileting, and transfer and mobility scores of 3 or 4. Ideally, we would have
adopted the aforementioned factors as control variables to adjust for patient factors in our analyses; however, we
could not get sufficient data retroactively, so we used another data source in the EMR for control variables.
The data source we used is called the Intensity of Nursing Care Needs (INCN).18 INCN data are used in Japan to
record patients’ daily nursing care, medical treatment, and activities of daily living. Because INCN is an important
tool in the medical fee system in Japan, many institutions use INCN data, including ours. The patients whose scores
are over the cutoff points are considered serious cases, and the amount of reimbursement the medical institution
receives from patients and insurers changes according to the proportion of serious cases. INCN data present patients’
conditions fairly accurately because nurses record INCN data every day, and the data are validated by multiple
nurses. To maintain the quality of INCN data, strict evaluation standards and guidelines are decided at the national
level. In our institution, there is an education program for staff based on these standards and guidelines.
The Cronbach [alpha] coefficient for INCN data in this study, which is an index of internal consistency reliability,
was 0.86. Because the score was high, the operation was deemed valid. Therefore, we judged INCN data to be
adequate for use as control variables to adjust for patient factors in this study. Because the items and rules of INCN
are revised every other year, we selected the following 12 items that did not change during the target term for use in
this study: wound care required, concurrent use of more than 3 intravenous drip infusions, use of an
electrocardiogram, use of a syringe driver, blood transfusion or products required, able to change posture, able to sit
up in bed, able to maintain a sitting position, able to move from a bed or chair, able to maintain oral hygiene, able to
take meals, and able to change clothes.
We created the data set for analysis in the following steps. First, we gathered INCN data of the target patients and
matched it with the data for age and sex. Next, we applied a label to each daily INCN record, stating whether it was
before or after the tool was implemented by using nursing record data. Even after the tool was implemented in
clinical settings, the decisions to use the tool or not and when to use the tool were at the discretion of the nurse in
charge of the patient. Therefore, the post-period contains records in which nurses used the tool (with the tool) and
those in which nurses did not use the tool (without the tool). We excluded INCN records that could not be matched
with nursing record data. Finally, we added a label to each daily INCN record to state whether there was a fall on the
following day or not by using fall report data.
Back to Top
Statistical analysis
We analyzed differences in the probability of falling
between patient records (1) in the pre- and post-periods,
and (2) in the post-period with and without the tool. The
probability of falling was defined as the probability of a
patient falling in a single day. We adopted a logistic
regression model in which the response variable was the
Table 1Opens a
popup window
Opens a
popup
window
Opens a
popup
window
logit of the probability of falling. We performed
multilevel analysis 19 of repeated measures in
consideration of the structure of the data, which were
hierarchical and were measured repeatedly. The range of
data, explanatory variables, and random effect in analyses
1 and 2 are shown in Table 1. For analysis, we used R
3.2.4 (R Foundation for Statistical Computing, Vienna,
Austria).
Back to Top
RESULTS
We created a data set consisting of data for 573 216
patient-days. Logistic regression analysis required 10-fold
the number of events as the number of explanatory
variables 20; our analysis was able to fulfill this condition.
Descriptive statistic values of patients and the occurrence
Table 2Opens a
popup window
Opens a
popup
Opens a
popup
window
window
of falls are shown in Table 2. The results of analysis 1
provided an odds ratio for the post-period of 0.83 (95%
CI, 0.72-0.95). The results of analysis 2 provided an odds
ratio for the post-period with the tool of 1.12 (95% CI,
0.91-1.37).
Back to Top
DISCUSSION
On the basis of the results of analysis 1, the probability of the occurrence of falls was significantly reduced during the
post-period. As a previous study 21 states, implementation of a fall risk screening tool may have caused behavioral
changes in nurses in relation to patient care regarding falls, and these behavioral changes reduced the probability of
the occurrence of falls. The results of analysis 2 showed the probability of the occurrence of falls increased during
the post-period with the tool, but this was not statistically significant. This may suggest that using a fall risk
screening tool alone does not reduce the occurrence of falls. According to the Cameron et al 22 systematic review,
only multifactorial interventions in hospitals reduce the rate of falls.23In Japan, fall risk assessment is advocated, but
as our study showed, using a fall risk screening tool alone does not reduce the occurrence of falls; therefore, we have
to promote a care system that includes fall prevention interventions. However, this tool can help identify patients at
risk of falls so that appropriate measures to prevent future falls can be implemented.
There are 2 future challenges. First, nurses in our institution did not record information about interventions with
high-risk patients in a standardized format. Future studies should focus on the effects of interventions with high-risk
patients, and the methods of providing and recording data for nursing intervention should be standardized. Second,
there may be an effect of the level of experience of the nurse in charge. For example, the possibility of falling might
be lower under the care of nurses with more nursing experience or a higher clinical position. We should investigate
the effects of such nurse career attributes 24,25 for a more precise analysis.
This study had the following limitations. We used INCN data as control variables to adjust for patient attributes
instead of using patient risk factors. However, this was not enough to adjust for actual patient risk factors.
Performing a randomized control trial is ideal but difficult because the probability of the occurrence of falls was very
low (1.52 cases/1000 patient-days) in this study period, as shown in Table 2. We were thus unable to avoid potential
underreporting 26 while depending on staff’s voluntary reporting as a data source. It is unclear whether
implementation of this tool affected the number of fall reports.
Back to Top
CONCLUSION
We investigated the effect of implementing a fall risk screening tool in an EMR system by using data stored in the
EMR. We found the probability of the occurrence of falls decreased after implementation, but using the tool did not
reduce the actual occurrence of falls. This indicated that we have to not only improve the system for assessing the
risk of falls but also intervene to prevent falls.
CIN: Computers, Informatics, Nursing
Issue: Volume 35(1), January 2017, p 18-28
Copyright: Copyright (C) 2017 Wolters Kluwer Health, Inc. All rights reserved.
Publication Type: [FEATURES]
DOI: 10.1097/CIN.0000000000000290
ISSN: 1538-2931
Accession: 00024665-201701000-00005
Keywords: Clinical decision support, Electronic health records, Informatics, Meaning use, Nursing informatics competencies,
Satisfaction, Usability, Workflow
Hide Cover
[FEATURES]« Previous Article Table of Contents Next Article »
Statewide Study to Assess Nurses’ Experiences With Meaningful Use-Based
Electronic Health Records
McBride, Susan PhD, RN-BC, CPHIMS, FAAN; Tietze, Mari PhD, RN-BC,
FHIMSS; Hanley, Mary Anne PhD, RN; Thomas, Laura PhD, RN, CNE
Author Information
Author Affiliations: Texas Tech University Health Science Center, School of Nursing, Lubbock (Dr McBride), Texas
Woman’s University, College of Nursing, Dallas (Dr Tietze); Sul Ross State University Rio Grand College, Alpine (Dr.
Hanley); and Texas Tech Health University Sciences Center, School of Nursing, Lubbock (Dr Thomas), Texas.
The authors have disclosed that they have no significant relationship with, or financial interest in, any commercial
companies pertaining to this article.
Corresponding author: Mari Tietze, PhD, RN-BC, FHIMSS, Texas Woman’s University, Houston J. and Florence A.
Doswell College of Nursing, T. Boone Pickens Institute of Health Sciences-Dallas Center, 5500 Southwestern Medical
Ave, Dallas, TX 75235 (mtietze@twu.edu).
Abstract
Nursing professionals are at the frontline of the health information technology revolution. The Texas Nurses
Association and Texas Organization of Nurse Executives partnered to evaluate the changing health technology
environment in Texas, in particular the nurses’ satisfaction with the use of clinical information systems. A
descriptive exploratory study using the Clinical Information System Implementation Evaluation Scale and a newly
developed Demographic Survey and the Meaningful Use Maturity-Sensitive Index, with a narrative component, was
conducted in 2014 and 2015. Nurses across Texas received an electronic invitation to participate in the survey,
resulting in 1177 respondents. Exploratory factor analysis revealed that variables of the Meaningful Use MaturitySensitive Index and Clinical Information System Implementation Evaluation Scale show strong interrater reliability,
with Cronbach’s [alpha] scores of .889 and .881, respectively, and thereby inform the survey analysis, indicating and
explaining variations in regional and institutional trends with respect to satisfaction. For example, the maturity of a
clinical information system within an organization and age of the nurse significantly influence the probability of
nurse satisfaction (P < .05). Qualitative analysis of nurses' narratives further explained the nurses' experiences.
Recommendations for future research and educational were identified.
The healthcare industry is undergoing a major transformation to establish an interoperable health information
technology (HIT) infrastructure to connect the nation with electronic health records (EHRs) and health information
exchanges (HIEs). This informatics revolution is affecting all aspects of the nursing profession.1-3 With the passage
of the Health Information Technology for Economic and Clinical Health Act in 2009,3 the Texas Nurses Association
(TNA) Board of Directors formed an advisory committee to evaluate the changing health technology environment
and make recommendations to the TNA Board regarding steps needed to prepare Texas nurses for the rapid uptake of
technology in healthcare settings required by this legislation. The TNA Board joined with the Texas Organization of
Nurse Executives (TONE) to create a statewide partnership with nursing leaders to address the impact of HIT on
nursing in the state of Texas. The TNA and TONE Boards formulated the TNA-TONE HIT Task Force. In 2014, this
task force became an official joint committee of both Boards, with recognition of the importance of HIT to nursing
practice and the importance of both organizations partnering to address nursing impact.
The TNA-TONE HIT committee was charged with examining how Texas nurses were affected by newly
implemented EHRs across the state and to establish baseline measures of nurses' satisfaction to inform evidencebased improvement strategies. To accomplish this goal, a statewide study was initiated to evaluate the perceptions of
nurses about their clinical information systems (CISs), defined as EHRs. The full intent of the study was to evaluate
the nurses' perspectives related to satisfaction with the usability of the institutions' CIS used in patient care delivery
across Texas. This information was intended to inform statewide efforts in Texas to improve the use of HIT for
nurses and other associated stakeholders. The purpose of this article is to summarize the development of a statewide
Texas HIT study, describe the methods used, report the results of the study, and outline the next steps for a statewide
improvement effort to address Texas nurses' satisfaction with their EHRs.
Back to Top
BACKGROUND AND SIGNIFICANCE OF THE STUDY
The quality of healthcare and patient safety in the United States has become a national concern in recent years,
motivated by the release of several Institute of Medicine (IOM) reports. The first IOM report noted that medical
errors were a national public health problem.4 The same report suggested that substantive improvements in
information technology were necessary to support clinical and administrative decision making about healthcare
systems. A common theme in all of the IOM reports is that broad safety and quality improvement efforts require the
development of innovative, electronic health information systems.4-6
Healthcare organizations in Texas are in the process of adopting and integrating information systems to meet the
requirements of the Office of the National Coordinator (ONC).7 The goal is to move toward universal adoption of
CISs, achieve meaningful use (MU) of EHRs, and establish interoperability through HIEs. With 73.2% of office-based
physicians using a certified EHR system, HIEs could provide a significant improvement in communication between
providers and hospitals. However, literature regarding the evaluation of the benefits that accrue from the adoption of
EHRs for nursing appears limited. For example, in a search of the major electronic databases, the past 5 years yielded
44 articles using electronic health record as a key term and the word nursing in the title. None of the articles
systematically addressed satisfaction.8-10 Within Texas, there has been little effort to systematically evaluate the
experience of nurses who use information technology.
The role of information technology is complex and dependent on the systems and processes in which it is embedded.
Furthermore, health information systems implementation is confounded by human factors and barriers that impede
user acceptance and use of the systems.11-13 If end users believe that the technology is easy to use and is beneficial
in supplying the information they need for decision-making purposes, adoption has a higher likelihood of success. If
end users perceive there is no relative benefit of the new system compared with what had been available to them in
the past, it is reasonable to assume that adoption will be resisted.10,13
Two theoretical models for the successful implementation of information systems were appraised. The DeLone and
McLean Model of Information Systems Success used in a number of information system evaluation studies identified
three dimensions important to systems success: system quality, information quality, and service quality.14 These
dimensions may be measured by user satisfaction, intention to use, and measurable net benefits of the system. The
second model, Rogers' innovation diffusion theory, identified constructs about technology that influence adoption as
well as aspects of the adopters and the adoption process.15 In addition, the innovation diffusion theory considers
organizational factors that influence technology adoption.15,16
Attributes of technological innovation that affect adoption of CISs, also known as EHRs include perceived relative
advantages, compatibility, complexity, trial-ability, benefits realization, adaptability, risk, task performance
improvement, and knowledge. Characteristics of individual adopters that influence the adoption of innovation
include tolerance of ambiguity, intellectual ability, motivation, values, learning style, and organizational or social
position.15 Given the relative newness of the use of CISs, several instruments were considered. The Clinical
Information System Implementation Evaluation Scale (CISIES), developed by Gugerty et al,17 was one that assesses
user satisfaction across organizations and reflects attributes of technological innovation. The instrument is sensitive
to the adopter's characteristics across organizations in both formative and summative evaluations of CIS
implementation. More details about the CISIES will follow.
Assessing end-user perceptions of specific aspects of system functionality, usability, and usefulness is essential to
identify approaches that can be used to make strategic improvements in CIS adoption, implementation, and
optimization of the system. The objective of the TNA/TONE HIT study was to inform a strategy that improves Texas
nurses' satisfaction with their EHRs. To accomplish this goal, baseline measures of satisfaction were needed.
Back to Top
STUDY METHODS
The statewide multiorganization study was a descriptive and exploratory study to identify key issues with the
current deployment of EHRs in the practice setting and to identify characteristics associated with satisfaction to
inform improvements. To address this aim, several research questions directed the design of the study:
1. What is the relationship between health setting characteristics and the nurses' satisfaction with their EHRs?
2. What is the relationship between the nurses' characteristics and the nurses' satisfaction with their EHRs?
3. What is the relationship between CIS characteristics and the nurses' satisfaction with their EHRs?
4. What are the themes and subthemes that emerge from the narrative comments (qualitative data) by nurses about
using their EHRs that may inform improvement strategies?
The study design was supported by multiple partners, including TNA, TONE, and the Texas Tech University Health
Sciences Center School of Nursing (TTUHSC SON). The TTUHSC provided research oversight and the internal
review board approval for the study. A research subgroup consisted of TNA-TONE HIT committee members
working with two principal investigators to develop the study design, methods, data collection, and analysis
processes.
Back to Top
Sampling and Survey Response
In this statewide study, nursing staff members, who are end users of a CIS, employed by Texas healthcare
organizations, represent the study population, including RNs, APRNs, LVNs, and support staff such as nursing
assistants. The target population was derived from a representative sampling of nurses employed in private, public,
and federal acute care facilities and their associated ambulatory/episodic care and long-term care units, referred to
collectively as the healthcare organization (HCO).
To determine sample size, an a priori power analysis was conducted to ensure 0.8 power and a Cronbach's [alpha] of
.05. A small effect size revealed the need for 1092 respondents.18 This sample size was deemed to provide adequate
representation of descriptive statistics and statistical modeling using logistic regression to examine factors associated
with predicting satisfaction.
The TNA/TONE online survey was deployed through an e-
Opens a
popup
window
mail sent to staff nurses from the chief nurse officer of the
HCO inviting voluntary and anonymous participation
using a stratified sampling method. The survey was
launched statewide on September 23, 2014. The initial
sampling strategy yielded approximately 250 survey
responses. An improved secondary sampling strategy
utilized a snowball approach with distribution of the
invitation to participate, the survey link, and
communication of purpose through e-mails sent by TNA
and TONE leadership to the membership of both
organizations. This secondary strategy resulted in 1177
total survey responses. Final data analysis to inform the
recommended strategies was reflective of comprehensive
responses from 987 participants. Figure 1 demonstrates the
volume increase from the initial launch in September 2014
Figure 1Opens
a popup
window
Opens a
popup
window
and the revised sampling strategy that resulted in the total
responses concluding in February 2015.
Back to Top
STUDY DESIGN AND INSTRUMENTATION
Two instruments were used to meet the study objectives. The first of the instruments was the Demographic Survey
and EHR Meaningful Use Maturity-Sensitive Index (MUMSI) designed by McBride and Tietze with a group of
content experts.19 The second instrument was the CISIES designed by Gugerty et al.17
Back to Top
Demographic Survey
The TNA/TONE research subgroup members designed the Demographic Survey consisting of two sections. The first
section was composed of questions descriptive of key characteristics of each participant such as practice setting, shift,
and experience with computers. These characteristics were used to control for differences in respondents and their
respective organizations. The second section of the Demographic Survey addressed the maturity of the organization's
EHR with respect to federal guidelines for MU 20 and nursing use. Methods to adjust for the maturity of the EHR are
important to fully understand the nurses' experiences, particularly given rapid deployment of EHRs to meet federal
certification guidelines across the state. The Demographic Survey was used to explore the relationships between
CISIES responses, the participant and institutional demographic characteristics such as practice setting, shift,
experience with computers, and type of institution, while controlling for the maturity of the EHR in the institution.
Back to Top
Meaningful Use Maturity-Sensitive Index
The maturity of an EHR was defined as the level of sophistication of the nurse's knowledge and use of EHR in daily
practice. The research subgroup explored various mechanisms to evaluate the maturity of the EHR in an institution.
The subgroup decided to utilize the ONC's Stage 1 MU measures as the model for capturing different levels of
functional status about the maturity of the EHR. This also reflected the status of MU maturity within the state of
Texas at the time the study was initiated. Content validity of the MUMSI was established with a two-round Delphi
method with quantitative instrument design strategies defined by Lynn.21
The MUMSI was deployed within the demographic
information in the online survey. Figure 2 reflects a
sample from the 24-item instrument and the manner in
which the questions were presented to the participants.
The participants were asked to indicate if the MU
Figure 2Opens a
popup window
functionality was present and used, present and not used,
not present, or they did not know if the functionality was
present.
Back to Top
Clinical Information System Implementation Evaluation Scale
Opens a
popup
window
Opens a
popup
window
The CISIES is a 37-item survey designed to measure the participants' satisfaction with their CIS. The CISIES 17 uses
response choices ranging from strongly agree to strongly disagree on a six-point Likert scale. The survey has been
tested for reliability and validity and has a Cronbach's [alpha] ranging from .94 to .96.17 In addition to achieving
many of the study goals, the research subgroup determined that the CISIES provided the most robust data to inform
academic and practice settings with plans needed to address HIT use by nurses.
Back to Top
Combined Instrumentation of Clinical Information System Implementation Evaluation Scale and
Meaningful Use Maturity-Sensitive Index
Both the CISIES 17 and the MUMSI 19 yielded strong
interrater reliability, with Cronbach's [alpha] scores of .881
and .889, respectively.22 Nunnally et al 23 indicated that .7
Opens a
popup
Opens a
popup
window
window
to .8 was an acceptable [alpha] level. Additionally, the
CISIES and MUMSI were examined using an exploratory
factor analysis. The subscales identified within both the
CISIES and the MUMSI can be used to detect further details
Figure 3Opens
a popup
window
that might inform improvement plans long-term. The
initial examination was performed to assure the research
subgroup that there was no immediate overlap in the
variables explaining the CISIES and the MUMSI. Figure
3 reflects the performance of these two surveys when
combined and analyzed using an exploratory factor analysis
methodology. Note that the rectangles in Figure 3, although
too difficult to interpret, represent each of the items on the
two instruments (for further details of the factor analysis,
contact the corresponding author). The research team
draws the reader's attention to the patterns distinguished
noting the cluster identified as MUMSI, CISIES, and clinical
decision support (CDS).
Back to Top
Qualitative Narrative Content Analysis
An open-ended question related to nurses' experiences with the use of technology and EHRs in the clinical setting
was included with the survey to gain in-depth textual information from the nurses beyond the constraints of the
instruments utilized. The question was stated as "We are interested in your feedback and anything we might not
have covered, please provide any additional information you care to provide." Of the 1177 surveys completed, 344
respondents provided free text or narrative responses. NVIVO 19 (QSR International, Doncaster, Victoria, Australia)
was used to conduct a word frequency analysis. This analysis detected key terms that could be easily dropped back
into the quantitative data. This result did not fully inform the improvement strategy, so further analysis of the
qualitative data was needed. These narratives were analyzed and coded for themes by the research subgroup.
The confidentiality of participants was maintained through use of an electronic survey administration system
designed and deployed by TTUHSC SON Information Technology Department, which included a Secure Sockets
Layer, a protocol that works through a cryptographic system that secures a connection between a client and a server.
An off-the-shelf software was used to design the online survey including the demographic section, 24-item MUMSI,
and 37-item CISIES questionnaires. No attempt was made to
