Post on 06-Jan-2022
Use of EHR big data to predict patient safety events
Hyeoun-Ae Park, RN, PhD, FAAN, IAHSI
Professor
College of Nursing
Seoul National University
Goal and Scope of the Study
• The goal of this study is to learn from data collected at the point of care and use that learning to promote patient safety or prevent patient safety events.
• Four patient safety problems (fall, pressure ulcer, unplanned extubation and adverse drug reaction of chemotherapy) were studied
Electronic
Health System
Clinical Data
Repository
Prediction system
Clinical Decision
Support System for
Prevention
Literature
Big data analytics
& Prediction model fitting
Knowledge
Database
Development of Computer
Interpretable Guideline
Big data Platform
Clinical
Practice
Guideline
Selecting features
Mapping features
Patient Safety
Reporting system
Extracting knowledge
Phase 1 Phase 2
Phase 3
Target: High Risk Patients Target: Patients with EventsTarget: Inpatients
Extracting data
Patient Safety Prediction and Prevention System
Study Subjects
• Case group: XXX patients who fell during hospital stay in neurology or hemato-oncology departments from Jan 1, 2015 to May 31, 2016
• Control group: XXXXX patients who did not fall during hospital stay in the same departments during the same period
Identifying Fall cases
• XXX fallers (XX%) were identified with ‘fall’ as a reason for safety event from the patient safety reports
• Additional XXX fallers (XX%) were identified with fall-related narratives from nursing notes
낙상함 (fall) 주저앉음 (drop) 넘어짐 (tumble) 쓰러짐 (collapse) 미끄러져 넘어짐
(slip down)
바닥에 몸을 던짐(throw oneself onto the floor
침대에서 떨어짐 (fall from bed) 침대에서 휠체어로 이동 중 fall down
됨 (fall from bed during transferring to wheelchair)
침대에서 휠체어로 이동 중 slip down 됨 (slip down from bed during transferring to wheelchair)
Feature Selection and Extraction
• 160 features identified from literature review
• 160 features were identified in 10 different
data sources1. Clinical observation
sheet; 2. Initial nursing
assessment; 3. Patient acuity
classification system; 4. Laboratory results report; 5. Doctor’s medication
order;
6. Fall risk assessment form;
7. Nursing activity sheet; 8. Surgical information
sheet; 9. Doctor’s progress
note; 10. Narrative nursing
notes
Data Sources by FeatureFeatures (example)
Type of data sources
A B C D E F G H I J
IV therapy ˅ ˅ ˅ ˅
Depressed, confusion, altered elimination ˅ ˅ ˅
Tube/drainage, restraints, bed rails ˅ ˅ ˅
Pain, anxiety, hyperthermia, ˅ ˅
Dizziness/vertigo ˅ ˅
Enema ˅ ˅
Age, sex ˅
Fall-risk score ˅
Drug (by types) ˅
Operation ˅
Hematocrit ˅
Diagnosis ˅
Walking aids, anorexia, fatigue, caregiver ˅
A. clinical observation sheet; B. initial nursing assessment; C. patient acuity classification system; D. laboratory results report; E. doctor’s medication order; F. fall risk assessment form; G. nursing activity sheet; H. surgical information sheet; I. doctor’s progress note; J. narrative nursing notes
Features in Different PresentationFeature Source Presentation ID 세부항목 ID 세세부항목 ID
우울 임상관찰기록 정신신체평가 21037 기분 21660 depression 30625
낙상사정도구 Symptomaticdepression
간호진술문 우울증 있음
우울해 함
어지러움/현기증 간호진술문 어지러움증 있음
어지러움증 증가함
어지러움증 호소함
어지럽고 식은땀이 난다고 호소함
일어서니 어지럽다 함
현기증 있음
Dizziness 있음
Dizziness 호소함
Vertigo 있음
Feature Source 표준코드 ATC코드 한글상품명
항고혈압제 복용 약품/투여정보 8806559015401 C02AC01 센트랄염산클로니딩주사액
8806559040212 C02CA04 카데리나정
Dizziness
Admission Discharge
“Stoppd inhalation of oxygen 2L by nasal cannula” in Nurses’ note
“Oxygen administration by nasal cannula” in Nursing activity sheet
“Oxygen therapy, method: nasal cannula, oxygen flow rate: 2L” in Clinical observation record
(occurrence of fall)
1/1 15:001/1 09:00 1/1 13:00
Example: oxygen therapy
Features with more than one data points
Feature Normalization
• Since the same feature can be extracted from more than one data source and the same feature is documented in many different ways (structured or unstructured) and more than once in a data source, we need to normalize the data.
• Principles used for feature normalization– Usability: used structured data over unstructured data– Currency: used data documented closer to the event
(case group: fall, control group: discharge) within allowed time frame
– Completeness: used a data source with higher recoding rate
– Granularity: used data documented in more detail.
Admission Discharge
Gender, route of admission, smoking..
Edema, weakness, pain..
(occurrence of patient safety event)
Surgery, diagnosis
Grouping Risk Factors by Data Extraction
Time① Documented once at
admission
② Documented once between admission and patient safety event
③ Documented more than once per day or per shift
Maximum data extraction allowance time
Patient No.Last time the PAIN
was recorded
Time of fall
occurrence. Time difference
1 Jan 1 Jan 2 2
2 Feb 15 Feb 15 1
3 Nov 10 Nov 12 3
4 May 7 May 8 2
5 Jun 22 Jun 22 1
6 Sep 3 Sep 4 2
7 Aug 27 Aug 29 3
Mean time difference = 2 days
If pain was recorded within 1 day before the onset of a pressure ulcer,
a patient has pain; otherwise, the patient does not have pain.
Maximum Data Extraction Time Allowed
Maximum data extraction time allowed by feature
CASE
CONTROL
event date
-1day-2day-3day-5day -4day-7day
-7day -3day -2day -1day
drainage
iv-therapy
heparin lock
acuity score
Foley cath.
restraints
mental status
pain
23 medicine
lab results
fall risk
score
-5day -4day
bed rails
guardianc-line
SurgeryDiagnosis
SurgeryDiagnosis
sex, age
history of diseases
admission route
functional disorder
(type)
dysuresia
assistant device
anorexia
sensory disorder
admission
admission
discharge
Fall discharge
P
Study subjects
Any pain documentation
in clinical observation
sheet?
Pain degree documented?
Pain?
Any painDocumentation
in nursing notes?
Clinical observation
sheet
Nursing Notes
Pain with degree
Pain without degree
Not documented
No pain
yes no
no
no
yes
yes
yes
yes
Pain degree documented?
no
no
Feature Normalization • Ex: Pain and degree of pain
documented within 24 hours before a fall event
Diffe
rent fo
rms
Pain?
yes
no
Statistical Analyses
• Descriptive statistics and visualization for the characteristics of case and control groups
• Univariate analyses (t-test/Mann-Whitney U test, one-way ANOVA/Kruskal-Wallis test, simple linear regression) to select significant features to be included in the prediction models
Fall Prediction Models
• Three prediction models were tested:
– A logistic regression model
– A Cox proportional hazard model,
– A decision tree model
• Training data set : Test data set = 7:3
Clinical Characteristics of Study Subjects
Fallers
(n=XXX cases)
Non-faller
(n=XXX cases) p-value
mean(±sd) or n(%) mean(±sd) or n(%)Age 62.9 (±14.7) 58.4 (±15.3) <.001
Sex M XXX (60.0) XXXX (53.7) .031
F XXX (40.0) XXXX (46.3)
Medical
Department
Neurology/Neurosurgery
XXX (47.7) XXXX (55.4) .007
Hematology/
OncologyXXX (52.3) XXXX (44.6)
LOS in hospital 23.0 (±19.3) 8.7 (±11.6) <.001
Time to fall from admission 10.9 (±12.9) -Fall-risk score (by Hendrich II) 6.4 (±2.6) 3.2 (±2.9) <.001
Time to fall from admission (n=XXX)
0
5
10
15
20
25
30
0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252729303435373839454648515962636572758183
Median = 7 daysMean = 10.9 daysIQR = 11(3.0-14.0) days
48.7% within 1 week after admission
0
5
10
15
20
25
30
35
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Num
ber
of
fallers
Time to fall/LOS
On discharge date
Distribution of time to fall from admission
length of stayof fall patients (n=XXX)
Median LOS = 19 daysMean LOS = 23 daysIQR = 20.7(9.3-30.3) days
sex
age
Admission route
Administration of vasodilator
Administration of nitrates
Administration of diuretics
Administration of cardiac stimulant
Administration of antidepressant
Administration of benzodiazepine
Administration of antipsychotics
Mental status
depression
Sleep disorder
Dysuria
Operatrion_neurology
Operation_cardiology
Operation_musculoskeletal
• Fever
• Dizziness/vertigo
• Communication disorder
• Guardian
• Other drainage
• Heparin lock
• IV therapy
• Diabetes mellitus
• Musculoskeletal disease
• Cardiovascular disease
• Unspecified cancer
• Respiratory cancer
• Walking disorder
• Balance disorder
• Restricted activity
• Total fall risk score
• Patient acuity score
34 Significant Features Selected by Univariate Analyses
-1.000
-0.800
-0.600
-0.400
-0.200
0.000
0.200
0.400
0.600
0.800
1.000
낙상
위험
군_위
험군
보행
보조
기구
사용
보행
장애
여부 TIA
배뇨
장애
헤파
린락
보유
통증
침상
난간
관장
의식
상태
인지
/사고
장애
신경
계수
술여
부
운동
장애
-보행
제외
마약
성진
통제
투약
감각
저하
기타
항우
울제
(ssir, t
ca제
외)
기타
신경
계약
물투
약
활동
수준
제한
vaso
dila
tor
투약
기타
근골
격계
질환
a-lin
e 보
유
배변
장애
e-t
ube 보
유
항당
뇨제
투약
nsa
ids
투약
불안
기타
배액
관
항파
킨슨
제투
약
균형
장애
식욕
부진
피로
/허약
감
항히
스타
민제
투약
c-lin
e 보
유
레닌
안지
오텐
신투
약
뇌졸
중
호흡
기, 흉
곽내
암
과거
수면
장애
음주
여부
항응
고제
투약
Feature Profiling
fall non-fall average
Variables CoefficientOddsratio(OR)
95% CI(OR) z value Pr(>|z|)
Cardiac_stimulant 0.311 1.364 0.761-2.329 1.093 0.275
Antipsychotic drug 0.556 1.744 1.151-2.592 2.690 <0.01
Mentalstate_confused/drowsy
-1.355 0.258 0.157-0.418 -5.449 <0.001
Mentalstate_coma/semi coma
-16.345 0.000 - -0.049 0.961
Depression -1.092 0.336 0.221-0.500 -5.266 <0.001
Sleep disorder 0.452 1.572 1.033-2.331 2.185 0.029
Dysuresia 0.911 2.488 1.691-3.596 4.744 <0.001
Operation_neurology 0.782 2.185 1.433-3.269 3.723 <0.001
Operation_muscular 1.543 4.678 2.277-8.946 4.444 <0.001
Dizziness 0.473 1.605 1.180-2.170 3.045 <0.01
Guardian -1.836 0.159 0.115-0.220 -11.159 <0.001
Heparin_lock -0.409 0.664 0.489-0.906 -2.605 <0.01
IV_therapy 0.690 1.994 1.468-2.723 4.383 <0.001
Unspecified cancer 0.636 1.889 1.334-2.645 3.651 <0.001
Respiratory cancer 0.509 1.664 1.092-2.477 2.444 0.015
Walking disorder 0.923 2.516 1.461-4.167 3.464 <0.001
Restricted activity 0.435 1.545 1.097-2.170 2.499 0.012
Fall risk score 0.400 1.492 1.378-1.617 9.799 <0.001
Acuity score 0.036 1.036 1.015-1.058 3.336 <0.001
Result: Logistic Regression Analysis
Predicted
Observed
Fall No fall Total
Fall
No fall
Total
Sensitivity = 75.0%Specificity = 76.2%Positive predicted value = 5.25%Negative predicted value = 99.43%Accuracy =76.15%AUC = 0.836
Result: Logistic Regression
Predicted
Observed
Fall No fall Total
Fall
No fall
Total
Sensitivity = 78.6%Specificity = 76.6%Positive predicted value= 7.11%Negative predicted value = 99.37%Accuracy = 76.66%AUC = 0.839
Result: Decision Tree
Variables CoefficientHazard
ratio(HR)95% CI(HR) z value Pr(>|z|)
Age 0.013 1.013 1.004-1.023 2.904 0.004
Diuretics 0.402 1.494 1.060-2.107 2.290 0.022
Mentalstate_confused/drowsy
-1.073 0.342 0.220-0.531 -4.771 <0.001
Mental state_coma/semi coma
-17.870 0.000 - -0.013 0.990
Depression -0.845 0.429 0.297-0.620 -4.505 <0.001
sleep_ disorder 0.449 1.566 1.086-2.259 2.400 0.017
Dysuria 0.832 2.297 1.647-3.203 4.900 <0.001
Fever -0.926 0.396 0.220-0.715 -3.076 0.002
Dizziness 0.413 1.511 1.141-2.000 2.884 0.004
Guardian -2.030 0.131 0.098-0.175 -13.827 <0.001
Other drainage -0.403 0.668 0.459-0.972 -2.110 0.035
IV therapy 0.858 2.358 1.788-3.111 6.070 <0.001
Unspecified cancer 0.459 1.583 1.158-2.164 2.882 0.004
Walking disorder 1.414 4.111 2.603-6.494 6.061 <0.001
Fall risk score 0.288 1.334 1.245-1.430 8.145 <0.001
Acuity score 0.036 1.037 1.017-1.057 3.690 <0.001
Result: Cox Proportional Hazard Model
Predicted
Observed
Fall No fall Total
Fall
No fall
Total
Sensitivity =71.3%Specificity = 71.9%Positive predicted value = 4.27%Negative predicted value = 99.3%Accuracy =71.87 (%AUC = 0.798
Result: Cox Proportional Hazard Model
On the 7th day
Comparison of three prediction models with Hendrich II fall risk assessment tool
Logistic reg. Decision tree Cox-hazard reg.
Hendrich II
Sensitivity 75.0% 78.6% 71.3% 67.4%
Specificity 76.2% 76.6% 71.9% 65.1%
PPV 5.3% 7.1% 4.3% 3.8%
NPV 99.4% 99.4% 99.3% 99.0%
Accuracy 76.2% 76.7% 71.9% 65.2%
AUC 0.836 0.839 0.798 0.701
Study Subjects
• Case group:XXXpatients with pressure ulcer*
during hospital stay in neurology or hemato-oncology departments from Jan 1, 2015 to May 31, 2016
* “pressure ulcer” occurrences were extactedfrom pressure ulcer incident reports.
• Control group: XXXXX patients who did not have pressure ulcer during hospital stay in the same departments during the same period
Features
• Extraction: 76 features from 9 different data sources were extracted
• Normalization: Features were normalized based on completeness, currency (recent>old), type of data (structured>unstructured, and granularity (detailed>general)
• Selection: 42 features were selected using visual analytics (Tableau) and univariate analyses
Maximum Time Allowed for Data Collection
Admission Discharge-5day -3day -1day
2. No pressure ulcer group
1. Pressure ulcer group
Gender, age, smoking, route of admission…
Pain, friction and shear, E-tube…
Edema, weakness, ventilator..
Malnutrition, skin moisture, medication…
(occurrence of pressure ulcer)
1day before discharge
Surgery, diagnosis
The features are same as those of ‘pressure ulcer group’
AdmissionDischarge-5day -3day -1day
Test of Pressure Ulcer Risk Prediction Models
• Three different types of prediction models were tested: Logistic regression, Cox-hazard model, and Decision Tree
• Training data set : Test data set = 7:3
– Training data set: XXX pressure patients and XXXXX no pressure patients
– Test data set: XX pressure ulcer patients as case group and XXXX with no pressure ulcer as control group
Clinical Characteristics of Study Subjects
PU group (n=XXX)Non-PU group
(n=XXXXX)p value
mean(± sd)
or n(%)
mean(± sd)
or n(%)
Age 61.71 (± 15.81) 58.49 (± 15.30) .004
Gender M XXX (58.9) XXXX (53.7) .143
F XX (41.1) XXXX (46.3)
Route of
admission
ER XX 48.0 XXXX 32.5<.001Clinic XXX 52.0 XXXXX 67.5
LOS in hospital, (day) 37.08 (± 31.99) 8.59 (± 11.05) <.001
Braden scale
score on
admission
Mild to very
high riskXX 41.1 XXXX 12.1
<.001No risk XXX 58.9 XXXXX 87.9
Distribution of Time to Pressure Ulcer
0
5
10
15
20
25
30
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 32 34 36 39 43 49 56 61 71 76 97
Time to Pressure Ulcer in Days
1/3 of pressure ulcers occur between 0-3 days in hospital
Rate of Pressure Ulcersby Department
Chest Cheek VertebralCoccyx
24%
11% 8% 8%
47%
17%
4%
Coccyx Hip Vertebral
Operating room
Wards & ICU
Wards(52%)
Operation room(34%)
ICU(14%)
Body side of Pressure Ulcers by Department
42 Significant Features Selected by Univariate Analyses
• Age
• Route of admission
• Braden scale score on admission
• Cardiac stimulants
• Sedative-hypnotics
• Analgesics
• Anti-inflammatory drugs
• Anticoagulants
• Diabetes mellitus
• Cancer
• Cardiovascular disease
• Respiratory disease
• Renal failure
• Head injury
• Spinal cord injury
• Fracture
• Hb
• Albumin
• Temperature
• Surgery
• Incontinence
• Diarrhea
• Malnutrition
• Nothing by mouth
• Nutrition consultation
• Gastrointestinal tube
• Foley catheter
• Ventilator
• Oxygen therapy
• Endotracheal tube
• Tracheostomy tube
• Restraint
• Immobilization devices
• Hygiene management
• Skin moisture
• Edema
• Skin condition
• Friction and shear
• Decreased consciousness
• Cognitive impairment
• Activity disorder
• Motor disorder
Comparison of Pressure Ulcer Predictability of Three Models
Indicator Logistic Reg. Decision TreeCox
Proportional Hazard
Sensitivity 0.901 0.855 0.881
Specificity 0.890 0.888 0.881
PositivePredictive Value
0.098 0.094 0.090
NegativePredictive Value
0.999 0.998 0.998
Youden’s index 0.791 0.743 0.742
AUC 0.968 0.900 0.951
Comparison of Pressure Ulcer Prediction Models
ᅳ Logistic regression model
ᅳ Cox proportional hazard regression model
ᅳ Decision tree modelᅳ Reference line
ROC curve
sensi
tivit
y
1- specificity
Study Subjects
• XXXX ICU patients with data on ‘time of endotracheal tube intubation’ and ‘time of endotracheal tube extubation’ between July 1, 2013 and June 30, 2016 (36 months)
• XX cases were categorized as an unplanned extubation by reviewing nursing narratives
Narrative Attribute Value Group
Endotracheal Tube was removed
Reason Self Case group
‘self’ or ‘unplanned’ in the narrative
Case group
Dr’s Order Control group
Obstruction
Refusal of patient
Characteristics of Study Subjects
Planned
(n=XXXX cases)
Unplanned
(n=XX cases) t or χ2 (p)
mean(±sd) or n(%) mean(±sd) or n(%)
Age 61.94 (±15.68) 68.58 (±15.01) -3.26(.001)
SexM XXXX (98.8) XX (1.2) .71(.398)
F XXXX (99.0) XX (1.0)
APACHE 26.16 (±9.52) 29.32 (±7.00) -3.07(.004)
Duration of intubation,
day3.29 (±24.50) 3.87 (±4.92) -0.18(.856)
LOS in ICU, day 6.53 (±12.13) 12.40 (±11.48) -3.73(<.001)
LOS in hospital, day 22.87 (±26.45) 31.30 (±23.67) -2.40(.017)
Feature Selection
• Step 1: 487 features were identified in the critical care flow sheet during 24 hours prior to unplanned extubation
• Step 2: Features with recording rate above 90% (to minimize missing rate) were selected 23 out of 487 features were selected
• Step 3: 23 features were grouped into ‘features with numbers of recording >3 (documented more than once per shift )’ and ‘features with numbers of recording ≤3’
Feature selection and data preparation
• Step 4: Data identification and data preparation, 3 different datasets of the features were prepared after univariate analyses. – Data set with value recorded at the nearest point of
time to the event for all features,– Data set with either nearest, maximum, minimum
or mean value of each feature which differentiate the case and control groups the most for the features recorded more than 3 times per day
– Data set with number of times each feature was recorded during 24 hrs for the features recorded more than 3 times per shift.
Nearest Maximum Mean MinimumRecording Frequency
Featureswithrecordingfrequency≤ 3
GCS (eye+motor) -8.56RASS -6.09Motor power (arm) -7.06Motor power (leg) -5.03Fio2 -0.53PEEP -1.65weight 2.61E-tube depth -2.397E-tube ID 0.8
Featureswithrecordingfrequency> 3
Systolic BP -1.990 0.144 -2.220 -3.008 1.87Mean BP -0.660 1.661 -1.214 -2.651 2.08Diastolic BP -0.340 1.357 -0.338 -1.552 1.37Pulse rate -5.160 -1.710 -4.878 -6.122 4.28Respiration rate -3.160 -0.313 -4.720 -6.236 1.27Body temperature -3.410 -2.330 -6.971 -3.030 3.72Minute volume -3.912 -0.881 -0.422 -0.56 -5.21Urine amount 3.010 2.300 2.525 1.522 -4.63SPO2 -0.556 -0.073 -0.348 -0.364 -1.61Peak pressure -2.750 -1.742 -2.631 -2.984 -1.82BST -1.060 0.293 1.543 3.101 -2.67Position change 4.204 -6.11Presence of restraint 0.283 -3.09Ventilator mode 2.699 -3.16
▶ Positive t value : Case < Control▶ Negative t value : Case > ControlUnivariate analysis
Recording
frequency
dataset
Nearest, Min,
Max or Mean
with
Highest effect
size
dataset
Nearest vaue
dataset
Recording
frequency
dataset
Model 1 Model 2 Model 3
Features withrecording frequency ≤ 3
Features withrecording frequency > 3
Nearest value
dataset
Nearest value
dataset
Comparison of Models
Model 1 Model 2 Model 3
AUC 0.82 0.88 0.90
Sensitivity 31.71 46.34 56.10
Specificity 94.13 91.40 92.74
Positive Predictive Value 6.74 6.86 9.54
Negative Predictive Value 99.04 99.20 99.36
Study Subjects
- All Patients who received the four most prescribed chemotherapy regimens (FOLFOX, FOLFIRI, GP, and Paclitaxel) from Jan 1, 2015 to June 30, 2016
- Chemotherapy-induced ADRs were identified from Clinical observation sheet, Blue sheet, and Narrative nursing notes
- ADRs include hypersensitivity, stomatitis, peripheral neuropathy, constipation, diarrhea, hand-foot syndrome, nausea & vomiting and weakness & anorexia
Nausea & Vomiting
38%
Weakness & Anorexia
28%
Peripheral Neuropathy
11%
Diarrhea
8%
Stomatitis
5%
Constipation
4%Hand-foot syndrome
3%Hypersensitivity
3%
Chemotherapy-induced ADR
Regimen
FOLFIRI FOLFOX GP Paclitaxel
Nausea & Vomiting 44.1% 39.7% 25.9% 24.5%
Weakness & Anorexia 26.1% 25.9% 41.2% 35.8%
Peripheral Neuropathy 7.1% 12.0% 10.0% 17.4%
Diarrhea 10.0% 7.6% 6.5% 6.4%
Stomatitis 5.4% 4.0% 5.3% 6.0%
Constipation 3.2% 2.9% 7.1% 4.6%
Hand-foot syndrome 1.2% 4.9% 0.6% 1.1%
Hypersensitivity 2.9% 2.9% 3.5% 4.3%
Prevalence of Chemotherapy-induced ADRs by Regimen
Feature Selection
Literature review 64
Feature extraction & normalization
31
Feature selection using Chi-square
test
Logistic regression with
stepwise selection
Hypersensitivity
StomatitisPeripheral
neuropathyConstipation Diarrhea
Hand-foot syndrome
Nausea &Vomiting
Weakness &Anorexia
11 11 16 10 16 8 22 23
Hypersensitivity
StomatitisPeripheral
neuropathyConstipation Diarrhea
Hand-foot syndrome
Nausea &Vomiting
Weakness &Anorexia
5 7 8 4 13 5 19 12
Feature Extraction
Admission Discharge- 3 Day
Age, Sex
Chemo cycle, regimen, a history of nausea & vomiting, etc.
bone marrow depression, infection, etc.
Diagnosis: diabetes mellitus, heart disease, etc.
Chemotherapy starts
• Data were extracted from Clinical observation sheet, Blue sheet, Initial nursing assessment sheet, Narrative nursing records, Laboratory results reports, Medication orders and diagnosis sheets.
Comparison of Chemotherapy-induced ADRs Prediction Models
Algorithms IndicatorsHyper
sensitivityStomatitis
Peripheral neuropathy
Constipation DiarrheaHand-foot syndrome
Nausea &Vomiting
Weakness &Anorexia
DecisionTree
AUC 0.8004 0.6493 0.7033 0.7711 0.7505 0.7070 0.8752 0.7145
Accuracy 0.6030 0.9802 0.8630 0.8433 0.8127 0.3878 0.8087 0.7271
Sensitivity 0.8412 0.3800 0.3914 0.6800 0.6547 0.9090 0.7765 0.6310
Specificity 0.6007 0.8977 0.8798 0.8451 0.8166 0.3827 0.8129 0.7363
Logistic Regression
AUC 0.8009 0.6643 0.7278 0.6462 0.7161 0.7066 0.8472 0.6742
Accuracy 0.7495 0.6014 0.5853 0.5828 0.6662 0.6301 0.7481 0.6408
Sensitivity 0.7500 0.6176 0.7435 0.6111 0.6666 0.6315 0.7480 0.6402
Specificity 0.7495 0.6010 0.5790 0.5825 0.6661 0.6301 0.7480 0.6408
Neuralnetwork
AUC 0.9298 0.6618 0.6875 0.6632 0.6944 0.7609 0.8520 0.6916
Accuracy 0.5845 0.6462 0.5960 0.6262 0.6296 0.6974 0.7724 0.6550
Sensitivity 0.6500 0.6470 0.6025 0.6111 0.6250 0.6842 0.7716 0.6524
Specificity 0.5841 0.6461 0.5957 0.6263 0.6297 0.6974 0.7725 0.6551
• We developed the models using 5 fold-cross validation
What we have learned…..
• Usefulness of nursing note as data source for identification of patient safety events and extraction of features– This was possible because electronic nursing records system
was developed using standardized terminology and data models.
• Creative way of using EHR data – Multiple sources, multiple types of data, and multiple points
of time, – Not only data documented at the nearest point to the event,
but also maximum, minimum, or mean value of the features can be used.
• Usefulness of current patient safety event risk assessment tools. – Need to introduce different assessment tool