Development and Clinical Validity of a Mild … Article Development and Clinical Validity of a Mild...

Research Article

Development and Clinical Validity of a Mild Vascular CognitiveImpairment Assessment Tool for Korean Stroke Patients

Hyun Soo Oh, PhD, RN, 1 Ji Sun Kim, RN, 1 Eun Bi Shim, RN, 2 Wha Sook Seo, PhD, RN 1, *

1 Department of Nursing, Inha University, Incheon, South Korea2 Incheon Regional Cardio-Cerebro-Vascular Center, Inha University Hospital, Incheon, South Korea

a r t i c l e i n f o

Article history:Received 23 September 2014Received in revised form1 April 2015Accepted 8 April 2015

Keywords:mild cognitive impairmentnursing assessmentreliability and validitystroke

s u m m a r y

Purpose: The present study was conducted to develop a mild vascular cognitive impairment (MVCI)assessment tool for patients with stroke and to examine its validity, reliability, and clinical adequacy.Methods: Items of this tool were developed based on previously verified cognitive assessment tools. Face,content, and criterion (concurrent) validities, optimal cut-off score for differentiation of MVCI andnormal cognitive function, clinical adequacy, internal consistency, and inter-rater reliability of theassessment tool were determined in 60 stroke patients at a university hospital located in Incheon, SouthKorea.Results: The devised MVCI assessment tool contains 20 items which were designed to assess sevencognitive domains: orientation, memory, language, attention, reasoning/abstraction, visuospatialperception, and executive function/problem solving. Content, face, and construct validities were wellsupported. Clinical adequacy testing revealed that the overall probability of correctly discriminatingMVCI using the MVCI assessment tool for stroke was 90.0%, which was statistically significant.Furthermore, a score of 23 was found to be the optimal cut-off score for MVCI. Internal consistency andinter-rater reliability were also well supported.Conclusions: The findings of this study indicate that the developed MVCI assessment tool for stroke couldserve as a clinically useful tool for detecting MVCI and for properly assessing degree of cognitiveimpairment in stroke patients.

Copyright © 2015, Korean Society of Nursing Science. Published by Elsevier. All rights reserved.

Introduction

Vascular cognitive impairment is a spectrum of cognitive im-pairments caused by various diseases, such as hypertension, dia-betes, arteriosclerosis, transient ischemic attack, and stroke [1].Stroke is one of the most common causes of vascular cognitiveimpairment [2]. The term cognitive impairment covers a wide va-riety of conditions ranging from the mildest form of cognitiveimpairment to overt dementia [1,3e5]. Poststroke cognitiveimpairment may lead to dementia (vascular dementia), but it oftenexhibits mild deficiencies in cognitive function (mild vascularcognitive impairment; MVCI) [6], which is referred to as “vascularcognitive impairment, no dementia”. Despite great differences be-tween studies, the highest incidence rate of 64.0% for MVCI among

stroke survivors was reported by Jin, Di Legge, Ostbye, Feightner,and Hachinski [7]. Few studies have been conducted on the MVCIincidence rate in Korea. Only one study reported an incidence rateof 39.0% (n ¼ 156) for mild cognitive impairment among 396Korean stroke patients [8].

According to Wentzel et al [9], approximately half of patientswithMVCI develop dementia within 5 years of stroke. The commoncognitive domains associated with MVCI in stroke patients areshort-term memory (31.0%), long-term memory (23.0%), visuo-spatial ability (37.0%), executive functions (25.0%), and language(14.0%) [10]. In addition, MVCI following stroke has been shown tobe significantly associated with early death and decrease in qualityof life for patients and their families [6].

The assessment of MVCI is important because the early detec-tion of cognitive impairment may facilitate intervention targetingthe prevention of dementia in stroke patients [11,12]. In addition,proper assessment can provide precise information to stroke pa-tients and their families, and enable the development of effectivecognitive rehabilitation plans that improve outcomes and quality of

* Correspondence to: Wha Sook Seo, RN, PhD, Department of Nursing, College ofMedicine, Inha University, YongHyun Dong 253, Incheon, 22212, South Korea.

E-mail address: [email protected]

Contents lists available at ScienceDirect

Asian Nursing Research

journal homepage: www.asian-nursingresearch.com

http://dx.doi.org/10.1016/j.anr.2015.04.005p1976-1317 e2093-7482/Copyright © 2015, Korean Society of Nursing Science. Published by Elsevier. All rights reserved.

Asian Nursing Research 9 (2015) 226e234

life. Therefore, development of a well-validated cognitive assess-ment tool is important for the establishment of nursing care andrehabilitation plans for stroke patients.

Backgrounds

It has been consistently reported that MVCI occurs frequently inpatients with stroke [13,14]. According to Nys, van Zandvoort, deKort, Jansen, de Haan, and Kappelled [15], a high proportion ofstroke survivors develop MVCI within 3 months of stroke, espe-cially if the cerebral cortex is involved (74.0%). Patients sufferingfrom vascular dementia often present with cognitive decline andmemory impairment severe enough to interfere with daily activ-ities [1]. On the other hand, MVCI is characterized by slightcognitive impairments in only a few cognitive domains and doesnot significantly impact daily functioning [16e18]. Nevertheless,MVCI may lead to dependence on others for daily activities,adversely affects patient quality of life, and increases family burden[6].

Of the cognitive domains, executive functioning and visualperception are most commonly associated with MVCI, and short-term and long-term memory, attention, and language are alsofrequently involved [15]. Furthermore, it has been reported that theseverities of deficits in executive functioning, language, verbalmemory, and abstract reasoning are more pronounced followingleft than right cortical stroke [15].

Investigators have consistently asserted no consensus has beenreached regarding the definition of mild cognitive impairment, andthat the boundary between normal cognitive function and mildcognitive impairment has not been clearly defined [18,19]. How-ever, distinguishing between pathological mild cognitive impair-ment and normal cognitive function is important whendetermining therapeutic options [18].

The Mini-Mental State Examination (MMSE) is the test mostwidely applied to screen cognitive impairment in clinics [20].However, although the MMSE has been shown to be useful forscreening Alzheimer's type dementia, it is inadequate for evalu-ating mild cognitive impairment because of its insensitivity to vi-suospatial and executive functional deficits [21e23].

The Cambridge Examination for Mental Disorders of the Elderly(CAMCOG) was originally designed to diagnose primary degener-ative dementia. This test has the advantage that it covers a broaderrange of cognitive functions than other tests do [24], and has beenshown to be a more accurate screening tool than the MMSE,especially for elderly stroke patients [24]. However, little is knownabout the diagnostic value of the CAMCOG for MVCI screening.

The Telephone Interviews for Cognitive Status (TICS) wasinitially developed to assess cognitive function in elderly patientswith Alzheimer's type dementia. The TICS is a brief test (takes 5e10minutes) and known to be sensitive and specific [25]. However, thistest was considered unsuitable for screening MVCI following strokebecause it covers only orientation, memory, attention, and lan-guage domains of cognitive function [25].

The Montreal Cognitive Assessment (MoCA) tool, which wasrecently introduced, was developed to detect mild cognitiveimpairment, and has been shown to be more sensitive for detectingmild cognitive impairment than the MMSE [11,26]. The MoCA in-cludes several cognitive domains that have been reported to bemost commonly associated with MVCI following stroke in clinicalresearch studies, and several studies have confirmed its highsensitivity and specificity and determined optimal cut-off values byreceiver operating characteristic (ROC) curve analysis [27]).Furthermore, the MoCA has been translated into 36 languagesincluding Korean and Chinese, and is now the most popularcognitive screening tool for detecting mild cognitive impairment

[28]. The greatest advantage of the MoCA is its ability to discrimi-nate between patients with mild cognitive impairment and pa-tients with dementia or healthy controls [29]. In addition, theMoCAis a brief, simple tool that can be applied within 10 minutes andplacesmore emphasis on visuo-motor-speed and executive abilitiesthan other tools do [30].

Based on a review of related literature, we considered the MoCAto be the most suitable measure for screening mild type cognitiveimpairment. However, the MoCA was unlikely to be used for long-term follow-up examinations and repeated measures because itincludes many items that require evaluations of animal pictures ornaming cards in a face-to-face manner. In addition, the MoCA doesnot contain items that assess problem-solving ability, one of themost common cognitive domains associated with MVCI in strokepatients [31]. Accordingly, we recognized the need for MVCIassessment tool which addresses major cognitive domains associ-ated with stroke and can be administered using diverse approachesincluding face-to-face and telephone-based administrations.

Study purpose

The present study was conducted to develop and verify a MVCIassessment tool for Korean stroke patients. The specific study aimswere (1) to review other cognitive assessment tools and developqualified items sensitive to MVCI that are capable of identifying thespecific cognitive domains involved and applicable using differentapproaches (face-to-face and telephone interviews) for long-termfollow-up examinations; (2) to evaluate the content and criterionvalidities of the MVCI assessment tool for stroke; (3) to conductROC curve analysis to examine construct validity and clinical ade-quacy by determining an optimal cut-off for differentiation of MVCIand normal cognitive function, and (4) to evaluate the reliability(internal and inter-rater reliability) of the MVCI assessment tool forstroke.

Methods

Study design

The present study adopted a nonexperimental, cross-sectionalcorrelation design to develop a MVCI assessment tool for strokeand examine its validity and reliability.

Item development procedure

Developing a brief, simple MVCI assessment tool that can beadministered in different ways was the prime goal of the presentstudy. As a preliminary step, we reviewed the literature regardingcognitive impairment after stroke. In particular, an extensive liter-ature review on previously verified cognitive assessment tools ableto assess diverse cognitive domains was performed. Finally fourcognitive assessment tools of fair quality were identified withsimple and brief features (completed within 5e10 minutes withoutprofessional aid). These were MMSE (Korean version), CAMCOG,TICS, and MoCA (Korean version).

The MMSE uses simple tasks to assess a number of cognitivedomains: orientation, memory, attention, language, and complexdemands. Orientation tasks measure sense of current year, month,date, day of the week, and season. Memory tasks assess ability torepeat the names of three objects, which are spoken by an exam-iner at a rate of one per second, immediately (registered memory)and after a set time (delayed memory). Attention tasks measure theability to count by subtracting 7 from 100. Language tasks assessthe ability to name an object, repeat a given phrase/sentence,follow a three-stage command as written, read and repeat a given

H.S. Oh et al. / Asian Nursing Research 9 (2015) 226e234 227

sentence, and make up and write a sentence about anything.Complex demand tasks measure the ability to copy two intersect-ing pentagons.

The CAMCOG assesses orientation, language (comprehensionand expression), memory (remote, recent, and new learningmemory), attention and calculation, praxis, abstract thinking, andperception (tactile and visual recognition, unusual view, and per-son recognition). Individual tasks in each domain of the CAMCOGare similar to those in the MMSE.

The TICS was developed for use in situations unsuitable for in-person cognitive screening. This test was originally designed foradministration by telephone, but it can also be administered face-to-face. The cognitive domains tested by the TICS are orientation,memory, attention, and language. Orientation tasks consist of ques-tions about current time, age, and address. Memory tasks measureabilities to repeat 10 words immediately and after a set time, and torecall recent and long-termmemories. Attention tasks assess abilityto count by subtracting 7 from100, and to repeat numbers spokenbyan examiner in reverse order. Language tasks measure sentencerepetition, verbal fluency, and comprehension abilities.

The MoCA assesses visuospatial perception, executive functions,language, memory, attention, abstraction, and orientation. Visuo-spatial perceptions and executive functions are assessed simulta-neously using a trail making task (drawing a line from letters tonumbers), a clock-drawing task (drawing the contour of a clock andpositioning numbers and hands to tell the commanded time), and acube coping task. The language tasks consist of questions asking thenames of animals on drawing cards (vocabulary), to repeat a sen-tence (repetition), and to recite more than 11 objects that can bepurchased at a market (fluency). Memory tasks assess ability torepeat five words spoken by an examiner immediately (no pointawarded) and after a set time. Attention tasks assess the ability torepeat five numbers in the order named, to repeat another threenumbers in reverse order, to count by subtracting 7 from 100, andto tap every time an the examiner says the letter “A”. Abstractiontasks assess ability to explainwhat pairs of words have in common.Orientation tasks measure the sense of current year, month, date,day of week, and place.

Taken together, it was found that cognitive domains and tasksincluded in the four assessment tools were fairly similar (Table 1):orientation, memory, language, attention, reasoning/abstract

thinking, and visuospatial perception. Furthermore, they covermost of the common cognitive domains known to be associatedwith MVCI, that is short-term and long-term memory, attention,language, executive functioning, and visuospatial perception[15e17]. However, it was noted only the MoCA contains items thatassess executive functioning.

Based on the findings of the review, we developed the MVCIassessment tool for stroke. The tool comprises seven cognitivedomains of orientation, memory, language, attention, reasoning/abstraction, visuospatial perception, and executive function/prob-lem solving (Table 2). In particular, we devised an item to assessexecutive function/problem solving based on empirical evidencethat the severities of deficits in more complicated cognitive abili-ties, such as, problem solving, are more pronounced followingstroke [31].

The orientation domain of the MVCI assessment tool for strokeconsists of 6 items that request current date, day of the week,season, phone number, address, and age (1 point for each; 6possible points). The memory domain consists of 5 items (imme-diate, remote/recent, prospect, and delayed memory), based onother cognitive assessment tools. No score was given for the itemaddressing immediate memory (repeat five words spoken by anexaminer after 30 seconds), as is the case for other cognitiveassessment tools. The possible maximum score of the memorydomain is 8 points: 1 point for remote; 1 point for recent; 1 pointfor prospect; and 5 points for delayed memory (1 point for eachword of five which were spoken by an examiner during immediatememory testing). Three items were devised for language (repeti-tion, verbal fluency, and comprehension) based on the othercognitive assessment tools (1 point for each; 3 possible points). Toassess attention, 3 items were devised (6 possible points):repeating five numbers spoken by an examiner (1 point), countingbackwards from 20 (1 point), and counting by subtracting 7 from100 (4 possible points; 1 point for each number, i.e., 93, 86, 79, and72 in sequence). Two items were devised to assess reasoning/abstraction (1 point for each) and 1 item (1 point) was devised toassess visuospatial perception.

For executive function/problem solving ability domain, 1 itemwas devised. This item was designed to assess ability to manage agiven situation using a 4-point scale where 1 represented no ideawhat to do to solve the problem (0 points), 2 represented a

Table 1 Cognitive Domains and Tasks Included in Previously Verified Cognitive Assessment Tools.

Domains Assessment tools Tasks or assessment items

Orientation MMSECAMCOGTICMoCA

e Ask to tell current date, day of the week, season, age, telephone number, & address

Memory MMSECAMCOGTICSMoCA

e Ask to immediately repeat five words spoken by an examinere Ask to repeat the five words after all other tests are completed

Language MMSECAMCOGTICSMoCA

e Ask to recite more than three words that start with the letter “A”e Ask to repeat a sentence that is difficult to pronouncee Ask to tell names of animals on drawing cards or after listening to descriptions of the animalse Ask to listen to short sentences & answer questions

Attention MMSECAMCOGTICSMoCA

e Ask to repeat five numbers spoken by an examinere Ask to count backwards from 20e Ask to clap whenever an examiner says days of the week in the correct sequence

Reasoning/Abstraction CAMCOGMoCA

e Ask to explain what each pair of words has in common (bicycles-train or rose-lily)

Visuospatial perception CAMCOGMoCA

e Ask to copy a drawing of a cubee Ask to correctly draw a clock showing the time stated by an examinere Ask to draw a line from letters to numbers (trail making)

Executive function MoCA e Ask to count by subtracting 7 from 100

Note. CAMCOG ¼ the cambridge examination for mental disorders of the elderly; MMSE ¼ mini-mental state examination; MoCA ¼ montreal cognitive assessment;TICS ¼ telephone interviews for cognitive status.

H.S. Oh et al. / Asian Nursing Research 9 (2015) 226e234228

disorganized or unreasonable approach (1 point), 3 represented asimplistic approach in a childish manner (2 points), 4 representedan immature approach considering his/her age (3 points), and 5represented a systematic and reasonable approach (4 points). Tworesearch assistants had practiced presenting all items to strokepatients over a 2-week period before data collection. During thisperiod, the two assistants were given two typical example situa-tions, and practiced how to categorize expected responses into thefive response options. Such pre-trainings were set up simulta-neously for the two assistants to decrease individual differencesrelated to response category selection. Because the example situ-ations were made of having definite answers, no difficulties in theselection of response category were found during data collection.To increase accuracy in assessing executive function/problemsolving ability, the two example situations were given to each studyparticipant and both responses were comprehensively assessed.

Pretest

A pretest was conducted on five stroke patients to ensure that,(1) it was understandable, (2) respondents could complete it within5e10 minutes, and (3) that cognitive function scores obtainedwhen the MVCI assessment tool was administered face-to-face orby telephone were similar. In general, the time period requiredbetween two assessments is 2 weeks to avoid recall bias in healthyadult participants. However, 1-week interval between testing viaface-to-face and telephone interview seemed to be appropriateconsidering the characteristics of our participants (elderly strokepatients who might have cognitive impairments) in the presentstudy.

During the pretesting, we noticed that five respondents weresufficient for pretesting purposes. The pretest was conducted bytwo research assistants. One was a nursing graduate student andthe other had participated in several clinical research projects.

Pretesting indicated that the devised tool was understandableand had no potential problems regarding the assessment of MVCI instroke patients. In addition, the cognitive function scores whenadministered face-to-face or by telephone interview were similar.The time taken to administer this tool during pretesting was 5e13minutes.

Study participants

The study participants comprised 60 stroke patients treated at auniversity hospital located in Incheon, South Korea. Only partici-pants who satisfied the following criteria were recruited: (1) strokediagnosed by a neurologist or neurosurgeon based on neuro-imaging findings within 3 months of stroke onset, (2) an age ofgreater than 19 years, (3) no history of previous cognitive disability,and (4) no difficulty regarding voluntary handmovement (requiredfor drawing). Stroke patients with expressive/receptive dysphasiawere included in the present study because language abilityassessment is one of the most important purposes of a cognitiveassessment tool. The MVCI assessment tool was also designed totest language ability.

We realized that study participants drawn from one hospitalmight limit the generalizability of the results obtained. However,the university hospital at which the data collection was conductedhad been appointed as a regional Cardio-Cerebro-Vascular Center

Figure 1. ROC curve of the MVCI assessment tool for stroke. The area under curve b is0.90 (Table 4), indicating that the probability of correctly discriminating cognitiveimpairment using the MVCI assessment tool is 90.0%. Note. MVCI ¼ mild vascularcognitive impairment; ROC ¼ receiver operating characteristic. aRandom chance line;bROC curve.

Table 2 MVCI Assessment Tool for Stroke Devised in Present Study.

Domains Subdomains Score Items/tasks

Orientation Time & place 6 e Ask to tell current date, day of the week, season, phone number, address, & ageMemory Immediate memory 0 e Ask to repeat five words spoken by an examiner after 30 seconds

Remote/recent memory 2 e Ask to tell past historye Ask to recall the latest events

Prospect memory 1 e Ask to recall any event (doctor's appointment, family affair, or personal affairs)scheduled on the next day or week

Delayed memory 5 e Ask to repeat five words spoken by an examiner after a set length of timeLanguage Repetition 1 e Ask to repeat a sentence that can be difficult to follow

Verbal fluency 1 e Ask to tell more than three words start with the Korean letter “ㄱ”

Comprehension 1 e Ask to answer questions about common senseAttention Counting numbers 2 e Ask to repeat five numbers spoken by an examiner in sequence

Calculation task 4 e Ask to count backwards from 20e Ask to count by subtracting 7 from 100

Reasoning/abstraction Similarities 2 e Ask to explain what each pair of objects or living things has in common (animals,flowers, instruments, or modes of transportation)

e Ask to make a pair of objects (pencil & notebook)Visuospatial ability 1 e Ask to tell the time indicated on a wall clockExecutive function/Problem solving Situation play 4 e Ask to explain how to manage a given situation commonly confronted in daily life

(4-score scale)Total score 30

Note. MVCI ¼ mild vascular cognitive impairment.


by the Korean Ministry of Health and Welfare. Thus, as a leadinghospital with respect to the treatment of acute/sub-acute stroke,this hospital was considered appropriate in terms of regionalrepresentativeness.

Theminimum sample size required for testing the validity of theMVCI assessment tool for strokewas computed using theMediCalc-13.0 program (MediCalc Software, Ostend, Belgium). The findingsobtained indicated a minimum sample size of 28 with an a value of.05, a power (1eb) of 0.80, an area under curve (AUC) of 0.80, and aratio of cognitive impairment to normal cognitive function of 1.28based on a study that reported amaximumprevalence rate of 64.0%for vascular cognitive impairment after stroke [7]. In the presentstudy, AUC was used for ROC curve analysis and the value of 0.80was derived based on an average AUC value of the MoCA(0.75e0.85) reported by Pendlebury et al [32].

Although the minimum sample size required for the presentstudy was found to be 28, data was actually collected from 60participants based on expectations of missing data and erratic re-sponses due to cognitive impairment.

Ethical consideration

This study was approved by the human research committee ofthe university the authors were affiliated with (IRB No.140213-1A).We made it clear to all participants that they were free not toparticipate and could withdraw from the study at any time withoutprejudice. We also explained that information would be collectedanonymously and that data would be presented as mean values(not as individual values). Study purposes and procedures wereexplained and participants were then allowed to decide whether toparticipate or not. Written informed consent was obtained frompatients who agreed to participate. When a patient was unable toprovide consent, written informed consent was also obtained fromfamily members.

Face and content validity testing

Face and content validities were tested by an expert panelcomprised of three nursing professors with experience of devel-oping cognitive rehabilitation programs for stroke patients, andfour nurses (3 with 5e10 years of neurological intensive care unitexperience and 1 with 3 years of experience in rehabilitationoutpatient clinics). All four nurses were especially trained toperform physical and cognitive assessments at the Cardio-Cerebro-Vascular Center of the study university hospital for face and contentvalidity testing, and three were graduate students in nursing.

Face and content validities were tested by evaluating whethercognitive domains and tasks included in the four cognitiveassessment tools (as summarized in Table 1) could properly assessmild type cognitive impairment. In addition, the expert panelestimated the level of difficulty posed by the words used in eachtask or question, particularly in memory, language, and reasoning/abstraction domains. Most items commented to be in need ofmodification by the members of the expert panel were revised.Notably, one item assessing problem solving/executive functioningwas devised using examples of real-world situations as suggestedby the expert panel (fire alarm ringing; taking cash from an ATM).Discussions regarding discrepancies were continued until anacceptable level of agreement was reached.

The expert panel evaluated whether completed 20 items of theMVCI assessment tool could properly assess mild cognitiveimpairment using a 3-point scale (1, not valid; 2, valid; 3, highlyvalid). The content validity index (CVI) was computed as thenumber of items that the experts gave a rating of either 2 or 3divided by the total numbers of item. In the present study, the CVI

of the MVCI assessment tool for stroke was 0.91, which supportedits face and content validities. That is, items of theMVCI assessmenttool for stroke appeared to be adequate for measuring band eachitem corresponded with items of the other four valid cognitiveassessment tools.

Measurements for criterion validity testing

To test the criterion validity of the MVCI assessment tool forstroke, the first step was selecting an adequate criterion tool. Basedon reviews of related literature, we considered the MoCA to be themost suitable measure for screening mild type cognitive impair-ment due to its wide applicability and good validity and reliability[29,30,32], and because it covers several cognitive domains(orientation, language, memory, attention, abstract thinking, andvisuospatial/executive function). Accordingly, the criterion validityof the MVCI assessment tool was examined by comparing resultsobtained with those of the MoCA. The purpose of this comparisonwas to examine whether the performance of the MVCI assessmenttool, which consists of written questions requiring verbal re-sponses, matches that of the MoCA, which consists of written andpictorial items, in terms of detecting MVCI. In addition, the MoCAwas also used to estimate a cut-off score for ROC curve analysis.

Construct validity of the original version of the MoCA wasexamined by the developers [27], who reported a sensitivity of100.0% and a specificity of 97.0% based on ROC analysis. However,they did not determine the reliability of theMoCA. Recently, Tu et al[28] confirmed good internal consistency and inter-rater reliabilityfor the MoCA. The MoCA was translated into Korean by Lee et al[29]. Since then, the validity and reliability of the Korean versionhas been consistently tested [sensitivity 89.0%, specificity 84.0%,Cronbach a ¼ .81e.84, and test-retest reliability r ¼ .75 (p < .001)].Cronbach a of the Korean version of the MoCA in the present studywas .85.

Data collection

Data collection was performed by the two research assistantsthat conducted pretesting. Both had worked as educational nursesin a regional Cardio-Cerebro-Vascular Center and were educatedabout the purposes of the present study and the structure of theMVCI assessment tool for stroke. In addition, both had practicedpresenting all items to stroke patients over a 2-week period. Toevaluate inter-rater consistency, the two assistants independentlyassessed cognitive functions of 20 stroke patients selected from the60 study participants. Data collection was conducted in patientrooms or outpatient clinics.

Data analysis

Statistical analysis was performed using SPSS/PC version 21.0(IBM SPSS Statistics-Korea, Seoul, Korea). Descriptive analysis wasused to describe general subject characteristics andmajor variables.Content and criterion (concurrent) validities of the MVCI assess-ment tool for stroke were evaluated using CVI values and by cor-relation analysis, respectively.

In general, factor analysis, known group method, multitraitmultimethod, or nomological analysis can be used to examineconstruct validity of a measurement tool [33,34]. Although themost frequently used method to examine construct validities isfactor analysis, it was not considered appropriate for the presentstudy because four domains (attention, reasoning/abstraction, vi-suospatial ability, and executive function/problem solving) of theMVCI assessment tool consist of only one or two items.


The specific methods to verify construct validity may depend onthe characteristics of a measurement tool. For a binary classifier,such as the presence or absence of a particular disease (MVCI in thepresent study), it is important to select optimal cut-off values. Inthis context, ROC curve analysis offers a powerful means of evalu-ating the performance of a binary classifier and for determiningcut-off values [29,32]. In the present study, ROC curve analysis wasconducted to determine optimal cut-off values and the sensitivity/specificity (construct validity) of the MVCI assessment tool forstroke.

Using ROC curve analysis, we calculated the AUCs, sensitivities,specificities, positive predictive values (PPVs), and negative pre-dictive values (NPVs) of different candidate cut-off scores. To testthe reliability of the MVCI assessment tool, its internal consistencyand inter-rater reliability were evaluated using Cronbach a andSpearman r, respectively.

Results

Descriptive statistics of participant characteristics

Mean age of the 60 study participants was 64.07 years (± 13.46),and 48 (80.0%) weremale and 12 (20.0%) female. Overall, 23 (38.4%)participants had lesions of the basal nuclei, 9 (15.0%) of the pons, 8(13.3%) of the right/left middle cerebral artery, 5 (8.3%) of the cer-ebellum, 3 (5.0%) of the diencephalon, 3 (5.0%) of the parietal lobe,and 9 (15.0%) of other areas (subarachnoidal hemorrhage, intra-ventricular hemorrhage, right/left posterior cerebral artery, andmultiple infratentorial infarction). Eleven (18.3%) of the study par-ticipants had hematoma and the mean amount of bleeding was27 mL (± 8.72). Regarding surgical modalities related to stroke, 3patients (5.3%) underwent surgery, such as craniotomy, clipping, orcoiling (Table 3).

Brain injury severity was assessed using the Glasgow ComaScale (GCS) or the National Institute of Health Stroke Scale(NIHSS), as determined by the department involved. The GCS andNIHSS were applied to 23.3% and 77.6% of the participants,respectively. The mean GCS and NIHSS scores at admission were[12.21 (± 3.07)/16] and [6.06 (± 4.23)/42], respectively, indicatingmoderate severity. Mean cognitive impairment scores deter-mined using the MVCI assessment tool for stroke and the MoCAwere [22.39 (±6.04)/30] and [22.69 (± 6.48)/30], respectively(Table 3).

Criterion validity testing

The criterion validity test is widely used to establish the validityof measurement tools. In general, criterion validity can bedemonstrated by comparing the selected measure with anothervalid measure; similarity between measures can be quantified us-ing correlation coefficients. The correlation coefficient between thecognitive impairment scores determined using the MVCI assess-ment tool and the MoCA was 0.88 (p � .001), indicating a highdegree of correlation (Table 4). Domain scores in orientation(r ¼ .79, p � .001), memory (r ¼ .62, p � .001), language (r ¼ .70,p � .001), attention (r ¼ .81, p � .001), and visuospatial ability(r ¼ .63, p � .001) were found to be significantly related betweenthe two scales (Table 4).

Construct validity and clinical adequacy testing

ROC curve analysis was conducted to evaluate the constructvalidity and the clinical adequacy of the MVCI assessment tool forstroke. A potential cut-off score for MVCI was estimated based oncut-offs reported for the MoCA, which ranged from 21 to 27[28,35e37], and averaged 24. Accordingly, participants with anMVCI score� 24 or� 23were allocated a normal cognitive functiongroup or a cognitive impairment group.

AUCs provide a numerical summary of ROC curves, and areconsidered suitable for quantifying the abilities of measurementtools to differentiate disease and nondisease states (i.e., MVCI andnormal cognitive function in the present study). AUC values can beinterpreted as probabilities, and range from 0.5 (incapable ofdiscriminating) to 1.0 (perfect ability to discriminate). In the pre-sent study, the AUC of theMVCI assessment tool for strokewas 0.90with a 95% confidence interval (CI) of 0.78e1.00 (p � .001, Table 4,Figure 1), indicating a probability of correctly identifying MVCI of90%, which is considered excellent [38].

To determine the optimal cut-off score for the MVCI assessmenttool for stroke, its sensitivities, specificities, PPVs, and NPVs atcandidate cut-off points of 23 and 24 (from the average cut-offscore of the MoCA) were determined. At a cut-off of 23, its sensi-tivity was 0.92, specificity was 0.89, PPV was 93.5% (95% CI [80.29,98.96]), and NPVwas 86.6% (95% CI [65.35, 97.19]). At a cut-off of 24,its sensitivity was 0.82, specificity was 0.89, PPV was 92.8% (95% CI[78.17, 98.86]), and NPV was 74.1% (95% CI [53.35, 89.16]) (Table 4).Because of its higher sensitivity, we subsequently used a score of 23as the optimal cut-off.

Table 3 Descriptive Statistics for Participant Characteristics.

Variables Frequencies (%)/M ± SD Variables Frequencies (%)/M ± SD

Age 64.07 ± 13.46 Gender Male 48 (80.0)Female 12 (20.0)

Site of lesion Basal ganglia 23 (38.4) Presence of hematoma Yes 11 (18.3)Pons 9 (15.0)MCA 8 (13.3)Cerebellum 5 (8.3) No 49 (81.7)Diencephalon 3 (5.0)Parietal lobe 3 (5.0)Elsea 9 (15.0)

Size of hematoma (mL) 27.00 ± 8.72 Surgery Yesb 3 (5.3)No 57 (94.7)

Scales used for assessing severity GCS 14 (23.3) Level of severity GCS 12.21 ± 3.07 (total 16)NIHSS 46 (76.6) NIHSS 6.06 ± 4.23 (total 42)

MVCI score 22.39 ± 6.04 (total 30) MoCA score 22.69 ± 6.48 (total 30)

Note. GCS ¼ glasgow coma scale; MCA ¼ right/left middle cerebral artery; MoCA ¼ montreal cognitive assessment; MVCI ¼ mild vascular cognitive impairment assessmenttool for Stroke; NIHSS ¼ national institutes of health stroke scale.

a Including subarachnoid hemorrhage; intraventricular hemorrhage; right/left posterior cerebral artery; multiple infratentorial infarction;b Including craniotomy, clipping, and coiling surgery.


Reliability testing

The internal consistency of the MVCI assessment tool for strokewas well supported by the present study. The inter-total correlationfor the 20 items ranged from .48 to .74, and most values were abovethe critical value of .40 of the summated rating scale [39]. Cronbacha (a coefficient of internal consistency) was high at .89, while itsinter-rater reliability was confirmed by a Spearman r of 0.93(p � .001, Table 4).

Discussion

According to Leys, Henon, Mackowiak-Cordoliani, and Pasquier[40], the prevalence of poststroke dementia ranges from 21.0% to32.0%, which is higher than the prevalence of nonstroke dementia(11.0%) [41]. Furthermore, the risk of dementia is markedlyincreased after recurrent stroke [26]. Dementia can be easilydetected because it affects the activities of daily life, but mildcognitive impairment is often difficult to diagnose [18]. Accordingto Jin et al [7], 64.0% of stroke survivors had some form of cognitiveimpairment, and usually this amounts to a mild type of cognitiveimpairment. Furthermore, individuals with mild impairmentsprogress to dementia faster than age-matched healthy controls[42].

In the present study, we developed the 20-item MVCI assess-ment tool, which can be administered face-to-face or by telephoneinterviews to enable long-term follow-up assessments, based onpreviously verified cognitive assessment tools. The items weredesigned to assess seven cognitive domains that have beenconsistently reported to be affected by stroke [25,26,43]. Theseseven domains include orientation, language, attention, memory,executive function and problem solving, reasoning/abstraction, andvisuospatial ability.

According to Oh and Seo [31], stroke patients may experiencediverse types of mild cognitive disabilities that involve impair-ments of language, memory, problem solving, and/or safety andsocial behavior. In this previous study, mild cognitive impairmentswere found in 23.3%e35.4% of patients during the first month afterstroke, and at 6months after stroke, the proportion of patients witha language or memory impairment decreased to 6.1%e10.2%, butthe proportion with a problem solving, safety behavior, or socialbehavior impairments remained high (20.2%e20.4%). Oh and Seo[31] concluded that such differences in recovery rates might berelated, in part, to levels of complexity, that is, more complexcognitive abilities, such as, problem solving, safety behavior, andsocial behavior, are probably more severely affected after stroke.

In the present study, the validity of theMVCI assessment tool forstroke was examined using content, face, and criterion validitytests. An expert panel conducted content and face validity testingand found that all 20 items were adequate for measuring MVCI.Each item corresponded with the conceptual definition of post-stroke cognitive impairment and with the items of other validcognitive assessment tools, which indicated that the content andface validities of the MVCI assessment tool for stroke were wellsupported.

The criterion validity of the MVCI assessment tool for stroke wastested by comparing its results with those of the MoCA, which wasused as a criterion tool due to its relatively good sensitivity fordetecting mild type cognitive impairment and its ability to assessdiverse cognitive domains associated with stroke [11,36]. In thepresent study, we found cognitive scores determined by the MVCIassessment tool and MoCA were highly correlated.

ROC curve analysis was conducted to assess the clinical ade-quacy of the MVCI assessment tool for stroke. Initially, we consid-ered using a cut-off score of 24, whichwas the average of previouslyreported MoCA cut-off values. However, at a cut-off of 23, thefractions of participants with MVCI that the MVCI assessment toolcorrectly identified as having such a condition was 92.0% (sensi-tivity of 0.92), and the fractions of participants with normalcognitive function that the MVCI assessment tool correctly identi-fied as having normal cognitive function was 89.0% (specificity of0.89). In addition, the overall probability of correctly diagnosingMVCI using the MVCI assessment tool for stroke was 90.0%. On theother hand, when a cut-off of 24 was used, sensitivity decreased to0.82, but specificity was maintained (0.89).

At a cut-off of 23, 93.5% (PPV) of the participants with cognitiveimpairment (identified using the MVCI assessment tool) werefound to actually have cognitive impairment (identified using theMoCA). In addition, 86.6% (NPV) of the participants with normalcognitive function (identified using the MVCI assessment tool)were found to actually have normal cognitive function (identifiedusing the MoCA). On the other hand, at a cut-off of 24, PPV and NPVwere markedly reduced to 92.7% and 74.1%, respectively. Based onthese findings, a score of 23 was taken to be an optimal cut-off withrespect to sensitivity and specificity.

Pendlebury et al [37] reported a sensitivity and specificity of77.0% and 83.0%, respectively, for the detection of mild cognitiveimpairment using the MoCA at a cut-off score of 25. Theses authorsalso reported a sensitivity and specificity of 83.0% and 73.0%,respectively, for Addenbrooke's Cognitive Examination Revised(ACE-R), at a cut-off score of 94. These findings demonstrate thatthe sensitivities and specificities of the MoCA and ACE-R for

Table 4 Validity, Reliability, and Clinical Adequacy of the MVCI Assessment Tool for Stroke.

Test Types Statistical Methods Statistics

Validity Criterion validity Correlational analysis Overall: r ¼ .88 (p � .001)Orientation: r ¼ .79 (p � .001)Memory: r ¼ .62 (p � .001)Language: r ¼ .70 (p � .001)Attention: r ¼ 0.81 (p � .001)Visuospatial: r ¼ .63 (p � .001)

Reliability Internal consistency Correlational analysis (item total correlation) r ¼ .48e.74Reliability analysis (Cronbach a) a ¼ .89

Inter-rater reliability Nonparametric correlational analysis(Spearman r)

r ¼ 0.93 (p � .001)

Clinical adequacy Discriminant ability ROC curve analysisAUC (p) Cut-off value Sensitivity Specificity PPV (95% CI) NPV (95% CI)0.90 (< .001) � 23 0.92 0.89 93.5 [80.3, 99.1] 86.6 [65.4, 97.2]

� 24 0.82 0.89 92.8 [78.2, 98.9] 74.1 [53.4, 89.2]

Note. AUC ¼ area under curve; CI ¼ confidence interval; MVCI ¼ mild vascular cognitive impairment; NPV ¼ negative predictive value; PPV ¼ positive predictive value;ROC ¼ receiver operating characteristic.


detecting mild cognitive impairment are lower than those obtainedusing the MVCI assessment tool for stroke devised in the presentstudy. Pendlebury et al [37] detected mild cognitive impairmentusing the Neuropsychological Assessment Battery, whereas weused the average cut-off score of the MoCA, and this differencebetween the criteria used to diagnose mild cognitive impairmentmay have contributed to the sensitivity and specificity differencesobserved between the two studies. The NeuropsychologicalAssessment Battery used by Pendlebury et al [37] assesses execu-tive function, attention, language, visuospatial, and memory do-mains, and includes six specific tests (the Trail Test, the SymbolDigit Modalities Test, the 30-item Boston Naming Test, the Rey-Osterrieth Complex Figure Copy Test, the Hopkins VerbalLearning TesteRevised, and letter and category fluency tests) andtakes approximately 50e60 minutes to apply. Accordingly, it isconsiderably more complex and probably more accurate atdetecting mild cognitive impairment than the MVCI assessmenttool for stroke.

In the present study, 63.0% of the 60 stroke patients were foundto have mild cognitive impairment according to the MVCI assess-ment tool, which concurs with the rate determined by Jin et al [7],who found that 64.0% of stroke survivors had cognitiveimpairment.

Nursing implication

The early detection of MVCI in stroke patients is important forfacilitating intervention, delaying or preventing the onset of de-mentia, and for developing effective cognitive rehabilitation plansthat improve outcomes and quality of life. A well-validated cogni-tive assessment tool containing accurate and simple items wouldbenefit nursing care and the development of rehabilitation plansfor stroke patients in acute and subacute stages. To meet suchneeds, we designed to the MVCI assessment tool for stroke, whichcan be completed in only 10 minutes but yet addresses the majorcognitive domains associated with stroke. Our findings showedthat theMVCI assessment tool for stroke has excellent criterion andconstruct validity and relevant clinical applicability with highsensitivity and specificity. We believe that the MVCI assessmenttool offers clinically useful means for detecting MVCI and forproperly assessing degrees of cognitive impairment in stroke pa-tients. In addition, the 20 items of the MVCI assessment tool arewritten questions requiring verbal responses. This allows the MVCIassessment tool to be administered over the telephone or by face-to-face interview, and thus, facilitate longitudinal follow-up ex-aminations of patterns of cognitive changes after stroke.

Conclusion

The present study was conducted to develop an MVCI assess-ment tool for stroke and to determine its validity, reliability, andclinical adequacy. The 20 items of the MVCI assessment tool forstroke were developed based on valid, pre-existing cognitiveassessment tools, and designed to assess seven cognitive domainsincluding orientation, memory, language, attention, reasoning/abstraction, visuospatial perception, and executive function/prob-lem solving. The present study supports the content, face, andconstruct validities of the MVCI assessment tool for stroke. Clinicaladequacy testing revealed that the overall probability of correctlydiscriminating vascular cognitive impairment using the MVCIassessment tool for stroke was 90.0%, which was statistically sig-nificant. A score of 23 was found to be an optimal cut-off point formaximizing in sensitivity and specificity. Our evaluation of thereliability supported its internal consistency and inter-rater reli-ability. Furthermore, our findings indicate that the MVCI

assessment tool for stroke offers a clinically useful means fordetecting MVCI and for properly assessing degrees of vascularcognitive impairment in stroke patients.

Conflicts of interest

No conflict of interest has been declared by the authors.

Acknowledgment

This work was supported by Fundamental Research Fund of theNational Research Foundation of Korea, 48118, and Inha UniversityResearch Grant.

References

1. Erkinjuntti T, Gauthier S. The concept of vascular cognitive impairment. FrontNeurol Neurosci. 2009;24:79e85. http://dx.doi.org/10.1159/000197886

2. Erkinjuntt T. Vascular cognitive deterioration and stroke. Cerebrovasc Dis.2007;24(suppl 1):189e94. http://dx.doi.org/10.1159/000107395

3. Gaffett KD, Browndyke JN, WhelihanW, Paul RH, DiCarlo M, Moser DJ, et al. Theneuropsychological profile of vascular cognitive impairmentdno dementia:comparisons to patients at risk for cerebrovascular disease and vascular de-mentia. Arch Clin Neuropsychol. 2004;19(6):745e57.http://dx.doi.org/10.1016/j.acn.2003.09.008

4. Jokinen H, Kalska H, M€antyl€a T, Pohjasvaara T, Ylikoski R, Hietanen M, et al.Cognitive profile of subcortical ischaemic vascular disease. J Neurol NeurosurgPsychiatry. 2006;77(1):28e33. http://dx.doi.org/10.1136/jnnp.2005.069120

5. Stephan BC, Matthews FE, Khaw KT, Dufouil C, Brayne C. Beyond mild cognitiveimpairment: vascular cognitive impairment, no dementia (VCIND). AlzheimersRes Ther. 2009;1(1):4. http://dx.doi.org/10.1186/alzrt4

6. Nys GM, van Zandvoort MJ, van der Worp HB, de Haan EH, de Kort PL,Jansen BP, et al. Early cognitive impairment predicts long-term depressivesymptoms and quality of life after stroke. J Neurol Sci. 2006;247(2):149e56.http://dx.doi.org/10.1016/j.jns.2006.04.005

7. Jin YP, Di Legge S, Ostbye T, Feightner JW, Hachinski V. The reciprocal risks ofstroke and cognitive impairment in an elderly population. Alzheimers Demen.2006;2(3):171e8. http://dx.doi.org/10.1016/j.jalz.2006.03.006

8. Kim BJ, Oh MY, Jang MS, Han MK, Lee JS, Lee JY, et al. Medial temporal atrophyand memory dysfunction in poststroke cognitive impairment-no dementia.J Clin Neurol. 2012;8:43e50. http://dx.doi.org/10.3988/jcn.2012.8.1.43

9. Wentzel C, Rockwood K, MacKnight C, Hachinski V, Hogan DB, Feldman H, et al.Progression of impairment in patients with vascular cognitive impairmentwithout dementia. Neurology. 2001;57(4):714e6.http://dx.doi.org/10.1212/WNL.57.4.714

10. Pohjasvaara T, Erkinjuntti T, Vataja R, Kaste M. Dementia three months afterstroke. Baseline frequency and effect of different definitions of dementia in theHelsinki Stroke Aging Memory Study (SAM) cohort. Stroke. 1997;28(4):785e92. http://dx.doi.org/10.1161/01.STR.28.4.785

11. Dong Y, Sharma VK, Chan BP, Venketasubramanian N, Teoh HL, Seet RC, et al.The Montreal cognitive assessment (MoCA) is superior to the mini-mentalstate examination (MMSE) for the detection of vascular cognitive impair-ment after acute stroke. J Neurol Sci. 2010;299(1e2):15e8.http://dx.doi.org/10.1016/j.jns.2010.08.051

12. Hachinski V, Munoz D. Vascular factors in cognitive impairment-Where are wenow? Ann New York Acad Sci. 2006;903(1):1e5.http://dx.doi.org/10.1111/j.1749-6632.2000.tb06343.x

13. De Ronchi D, Palmer K, Pioggiosi P, Atti AR, Berardi D, Ferrari B, et al. Thecombined effect of age, education, and stroke on dementia and cognitiveimpairment no dementia in the elderly. Dement Geriatr Cogn Disord.2007;24(4):266e73. http://dx.doi.org/10.1159/000107102

14. Narasimhalu K, Ang S, De Silva DA, Wong MC, Chang HM, Chia KS, et al.Severity of CIND and MCI predict incidence of dementia in an ischemic strokecohort. Neurology. 2009;73(22):1866e72.http://dx.doi.org/10.1161/STROKEAHA.110.594671

15. Nys GM, van Zandvoort MJ, de Kort PL, Jansen BP, de Haan EH, Kappelled LJ.Cognitive disorders in acute stroke: prevalence and clinical determinants.Cerebrovasc Dis. 2007;23(5e6):408e16. http://dx.doi.org/10.1159/000101464

16. Moorhouse P, Rockwood K. Vascular cognitive impairment: current conceptsand clinical developments. Lancet Neurol. 2008;7(3):246e55.http://dx.doi.org/10.1016/S1474-4422(08)70040-1

17. Sachdev PS, Brodaty H, Valenzuela MJ, Lorentx L, Koschera A. Progression ofcognitive impairment in stroke patients. Neurology. 2004;63(9):1618e23.http://dx.doi.org/10.1212/01.WNL.0000142964.83484.DE

18. Stephan BC, Matthews FE, McKeith IG, Bond J, Brayne C. Early cognitive changein the general population: how do different definitions work? J Am Geriatr Soc.2007;55(10):1534e40. http://dx.doi.org/10.1111/j.1532-5415.2007.01386.x

19. Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund L-O, et al. Mildcognitive impairmentdbeyond controversies, towards a consensus: report of


the international working group on mild cognitive impairment. J Intern Med.2004;256(3):240e6. http://dx.doi.org/10.1111/j.1365-2796.2004.01380.x

20. Cullen B, O'Neill B, Evans JJ, Coen RF, Lawlor BA. A review of screening tests forcognitive impairment. J Neurol Neurosurg Psychiatry. 2007;78(8):790e9.http://dx.doi.org/10.1136/jnnp.2006.095414

21. Blake H, McKinney M, Treece K, Lee E, Lincoln NB. An evaluation of screeningmeasures for cognitive impairment after stroke. Age Aging. 2002;31(6):451e6.http://dx.doi.org/10.1093/ageing/31.6.451

22. Bour A, Rasquin S, Boreas A, Limburg M, Verhey F. How predictive is the MMSEfor cognitive performance after stroke? J Neurol. 2010;257(4):630e7.http://dx.doi.org/10.1007/s00415-009-5387-9

23. Ihara M, Okamoto Y, Takahashi R. Suitability of the Montreal cognitiveassessment versus the mini-mental state examination in detecting vascularcognitive impairment. J Stroke Cerebrovasc Dis. 2013;22(6):737e41.http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2012.01.001

24. De Koning I, van Kooten F, Dippel DW, van Harskamp F, Grobbee DE, Kluft C,et al. The CAMCOG: a useful screening instrument for dementia in strokepatients. Stroke. 1998;9(10):2080e6.http://dx.doi.org/10.1161/01.STR.29.10.2080

25. Barber M, Stott DJ. Validity of the Telephone Interview for Cognitive Status(TICS) in post-stroke subjects. Int J Geriatr Psychiatry. 2004;19(1):75e9.http://dx.doi.org/10.1002/gps.1041

26. Pendlebury ST, Cuthbertson FC, Welch SJ, Mehta Z, Rothwell PM. Underesti-mation of cognitive impairment by Mini-Mental State Examination versus theMontreal Cognitive Assessment in patients with transient ischemic attack andstroke: a population-based study. Stroke. 2010;41(6):1290e3.http://dx.doi.org/10.1161/STROKEAHA.110.579888

27. Nasreddine ZS, Phillips NA, Bediriam V, Charbonneau S, Whitehead V, Collin I,et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mildcognitive impairment. J Am Geriatr Soc. 2005;53(4):695e9.

28. Tu Y, Jin H, Ding B, Yang X, Lei Z, Bai S, et al. Reliability, validity, and optimalcutoff score of the Montreal Cognitive Assessment (Changsha version) inischemic cerebrovascular disease patients of Hunan province. Demen GeriatrCogn Disord Extra. 2013;3(1):25e36. http://dx.doi.org/10.1159/000346845

29. Lee JY, Lee DW, Cho SJ, Na DL, Jeon HJ, Kim SK, et al. Brief screening for mildcognitive impairment in elderly outpatient clinic: validation of the Koreanversion of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol.2008;21(2):104e10. http://dx.doi.org/10.1177/0891988708316855

30. Pantoni L. Cerebral small vessel disease: from pathogenesis and clinical char-acteristics to therapeutic challenges. Lancet Neurol. 2010;9(7):689e701.http://dx.doi.org/10.1016/S1474-4422(10)70104-6

31. Oh HS, Seo WS. Changes in the acute functional and cognitive disability statesof severe hemorrhagic stroke patients. J Neurosci Nurs. 2010;42(5):245e54.http://dx.doi.org/10.1097/JNN.0b013e3181ecaf81

32. Pendlebury ST, Welch SJ, Cuthbertson FC, Mariz J, Mehta Z, Rothwell PM.Telephone assessment of cognition after transient ischemic attack and stroke:modified telephone interview of cognitive status and telephone MontrealCognitive Assessment versus face-to-face Montreal Cognitive Assessment andneuropsychological battery. Stroke. 2013;44:227e9.http://dx.doi.org/10.1161/STROKEAHA.112.673384

33. Kang H. A Guide on the use of factor analysis in the assessment of constructvalidity. J Korean Acad Nurs. 2013;43(5):587e94.http://dx.doi.org/10.4040/jkan.2013.43.5.587. Korean

34. Lee K, Shin S. Validity of instrument development research in Korean nursingresearch. J Korean Acad Nurs. 2013;43(6):697e703.http://dx.doi.org/10.4040/jkan.2013.43.6.697. Korean

35. Dong Y, Venketasubramanian N, Chan BP, Sharma VK, Slavin MJ, Collinson SL,et al. Brief screening tests during acute admission in patients with mild strokeare predictive of vascular cognitive impairment 3e6 months after stroke.J Neurol Neurosurg Psychiatry. 2012;83(6):580e5.http://dx.doi.org/10.1136/jnnp-2011-30207

36. Koski L. Validity and applications of the Montreal Cognitive Assessment for theassessment of vascular cognitive impairment. Cerebrovasc Dis. 2013;36(1):6e18. http://dx.doi.org/10.1159/000352051

37. Pendlebury ST, Mariz J, Bull L, Mehta Z, Rothwell PM. MoCA, ACE-R, and MMSEversus the national institute of neurological disorders and strokedCanadianstroke network vascular cognitive impairment harmonization standards neu-ropsychological battery after TIA and stroke. Stroke. 2012;43(2):464e9.http://dx.doi.org/10.1161/STROKEAHA.111.633586

38. McDowell I. Measuring health: a guide to rating scales and questionnaires. 3rded. New York: Oxford University Press; 2006.

39. Ware JE. SF-36 Health survey. Manual and interpretation guide. 2nd ed. Boston(MA): Nimrod Press; 1997.

40. Leys D, H�enon H, Mackowiak-Cordoliani MA, Pasquier F. Poststroke dementia.Lancet Neurol. 2005;4(11):752e9.http://dx.doi.org/10.1016/S1474-4422(05)70221-0

41. Ivan CS, Seshadri S, Beiser A, Au R, Kase CS, Kelly-Hayes M, et al. Dementia afterstroke: the Framingham study. Stroke. 2004;35(6):1264e8.http://dx.doi.org/10.1161/01.STR.0000127810.92616.78

42. Palmer K, Fratiglioni L, Winblad B. What is mild cognitive impairment? Vari-ations in definitions and evolution of nondemented persons with cognitiveimpairment. Acta Neurol Scand. 2003;107(suppl 179):14e20.http://dx.doi.org/10.1034/j.1600-0404.107.s179.2.x

43. Hachinski V, Iadecola C, Petersen RC, Breteler MM, Nyenhuis DL, Black SE, et al.National institute of neurological disorders and strokedCanadian strokenetwork vascular cognitive impairment harmonization standards. Stroke.2006;37:2220e41. http://dx.doi.org/10.1161/01.STR.0000237236.88823.47


Development and Clinical Validity of a Mild … Article Development and Clinical Validity of a Mild...

Documents

Transcript of Development and Clinical Validity of a Mild … Article Development and Clinical Validity of a Mild...