The Virtual Reality Functional Capacity Assessment Tool (VRFCAT): Data in Support of an Accepted Application for the FDA Clinical Outcome Assessment Qualification ProgramWilliam P Horan1, Alexandra S Atkins1, Luca Pani1,2, Philip D Harvey1,2, Tim Clayton1, Caren Gadigian1, Shane Baldwin1, Richard SE Keefe1,3 1 VeraSci (Durham, NC), 2 University of Miami (Miami, FL), 3 Duke University (Durham, NC)

BACKGROUNDThe FDA has guided drug developers focused on cognitive improvement to demonstrate not only basic improvement on cognitive endpoints, but also improvement on a co-primary measure of functional capacity.

The Virtual Reality Functional Capacity Assessment Tool (VRFCAT©) is a computer based virtual-reality measure of functional capacity. As shown below, the VRFCAT uses a realistic simulated environment to recreate routine activities of daily living.

Our application for the VRFCAT has officially been accepted into the FDA’s Clinical Outcome Assessment (COA) Qualification Program. This program is intended to qualify drug development tools that can be relied on to have a specific interpretation and application in any drug development program and regulatory review.

The FDA accepted the VRFCAT application into the COA Qualification Program after reviewing the development process and extensive quantitative evidence supporting its clinical trial utility, which is summarized in the accompanying wheel and spokes diagram

VRFCAT SCREENSHOTS AND 12 TASKS:Kitchen Bus Stop Grocery Store

Pick up the recipe on the counter Search for ingredients in cabinets and refrigerator Cross off the ingredients that you already have, pick up the bus schedule from the counter; Pick up the billfold on the counter; Exit the apartment and head to the bus stop

3

4

5

2

1

Apartment: Wait for the correct bus to the grocery store and board it when it arrivesAdd the exact bus fare and pay for the bus

7

6

Bus to store: Select a food aisle to begin shoppingContinue shopping for the necessary ingredients, and check out when finishedAdd up the exact amount for your purchase and pay for groceries

9

10

8

Store:

Spoke I

Spoke IISpoke IIISpoke

IVSp

oke V

Conceptof Interest

=Claim

VRFCAT modified and refined through a rigorous scale development processAn initial tutorial was added to facilitate subject comprehension and comfort with technologyTranslated and culturally adapted versions have been tested in 6 languagesValidation studies for new COU’s (MDD, MCI, PD, MS) are underway

V. Modify Instrument

Strong test-retest reliability and minimal practice effectsSensitive to treatment with social cognitive training in schizophrenia

IV. Longitudinal Evaluation of Measurement Properties

The VRFCAT was initially drafted based on extensive literature review and expert inputIt has been refined through an iterative, multi-stage process

II. Draft Instrument and Evaluate Content Validity

Strong discrimination of patients from healthy controls, psychometrics, and convergent validityFormal manual available

III. Cross-sectional Evaluation of Other Measurement Properties

COU: Clinically stable schizophrenia patients COI: Functional capacity, which refers to an individual’s capacity for performing key tasks of daily living

I. Identify Context of Use (COU) and Concept of Interest (COI)

www.VeraSci.com • 919.401.4642 • info@VeraSci.com • 3211 Shannon Rd, Suite 300 Durham, NC 27707 USA

NEXT STEPSWe received a grant from the FDA to collect qualitative evidence to confirm the content validity of the VRFCAT. We will conduct semi-structured interviews with patients, family members, and peer support specialists to determine whether they view the skills measured by the VRFCAT as important and meaningful for independent functioning in the community

VRFCAT data from several large industry drug treatment trials will provide data on the longitudinal properties (e.g., sensitivity to change) of the VRFCAT

These complementary new data collection efforts will be integrated with the existing evidence to support a formal qualification package to the FDA

1. Keefe, R.S.E., Davis, V.G., Atkins A.S., Vaughan, A., Patterson T., Marasimhan, M., Harvey, P.D. (2016). Validation of a computerized test of functional capacity. Schizophrenia Research, 90-96.

2. Ventura, J., Welikson, T., Ered, A., Subotnik, K.L., Keefe, R.S.E., Hellemann, G.S., Nuechterlein, K.H. (2019). Virtual reality assessment of functional capacity in the early coure of schizophrenia: Associations with cognitive performance and daily functioning. Early Intervention in Psychiatry, 1-9.

3. Nahum, N. (2019). Online plasticity-based social cognitive training for schizophrenia: A multi-site RCT. Paper presented at the Annual Cognitive Remediation in Psychiatry Meeting, New York, NY.

Baseline Mid-Assessment Post

900

800

1000

1100

1200

VRFC

AT

Control

SCV

VRFCAT in patients receiving Control (n = 71) vs. SocialVille (SCV; n = 76) interventions: SCV Cohen’s d = 0.51; Group X Time: b = -57.5, p = .025.

Treatment Study3IVRecent-Onset Schizophrenia Validation Study2

Age, Mean (Std Dev)

Male, N (%)

Non-Hispanic, N (%)

Years of education, Mean (Std Dev)

Parental education. Mean (Std Dev)

HC (N = 13)

21.92 (1.61)

4 (31%)

7 (53.8%)

14.43 (1.27)

13.08 (2.96)

SZ (N = 42)

23.05 (4.04)

31 (74%)

28 (70%)

13.04 (1.63)

14.38 (4.24)

III

VRFCAT Total Time T-Score

VRFCAT Total Errors T-Score

VRFCAT Progression T-Score

MCCB Composite T-Score

UPSA-2-VIM

GFS-Role functioning

GFS-Social functioning

.65**

.66**

-.70**

-.66**

-.52

-.43*

.83**

-.60**

-.53**

-.17

-.26

-.66**

-.62**

-.32

-.43

VRFCAT Total Time T-Score

VRFCAT Total Errors T-Score

VRFCAT Progression T-Score

Convergent validity within the recent-onset patients:Correlation analysis of the VRFCAT with MCCB, functional capacity (UPSA-2-VIM), and daily functioning (GFS) indicates a strong degree of correlation. *p < .05. **p < .01. Data from2

III

RECENT-ONSET SCHIZOPHRENIA SENSITIVITY TO CHANGE

Age, Mean (Std Dev)

Male, N (%)

Non-Hispanic, N (%)

English as Primary Language, N (%)

Unemployed, N (%)

Comfortable with PC, N (%)

Years of Education, Mean (Std Dev)

Mother’s Years of Education, Mean (Std Dev)

42.6 (13.93)

88 (53)

136 (82)

157 (95)

54 (33)

160 (97)

14.7 (2.41)

12.9 (2.98)

43.6 (11.84)

87 (55)

128 (81)

151 (96)

135 (85)

140 (89)

12.8 (1.99)

12.5 (3.33)

HC(N – 165)

SZ(N = 158)

Chronic Schizophrenia Validation Study1I V

Schizophrenia Patients Healthy Controls

Patients (N = 155) HC (N=163)

Assessments

VRFCAT Total Time T-score

VRFCAT Total Errors T-score

VRFCAT Progression T-score

MCCB Composite T-Score

UPSA-2-VIM

VRFCAT Total Time

T-score

VRFCAT Total Errors

T-score

VRFCAT Progression

T-scoreMCCB MCCB

---

0.70

0.71

0.57

0.56

0.77

---

0.65

0.39

0.41

0.59

0.73

---

0.45

0.43

0.69

0.54

0.38

---

0.70

0.65

0.56

0.41

0.75

---

Convergent validity: Correlation analysis of the VRFCAT and MCCB indicates a strong degree of correlation, comparable to the UPSA. All correlations p-values were < 0.001 (data from1).

III

Test re-test reliability for the VRFCAT and UPSA: Although reliability statistics were similar for the VRFCAT total time and UPSA, the UPSA demonstrated increased vulnerability to practice effects (data from1).

AssessmentsVisit 1

Mean (SD)Visit 2

Mean (SD)Cohen’s d

Intraclass Correlation Coefficient

(ICC)

VRFCAT Total Time T-score

VRFCAT Total Errors T-score

VRFCAT Progression T-score

UPSA-2-VIM*

50.1 (11.12)

49.7 (11.48)

49.8 (10.20)

83.4 (9.06)

32.3 (16.78)

37.1 (22.74)

40.4 (13.66)

70.7 (11.83)

50.9 (11.52)

49.8 (12.94)

50.3 (10.51)

86.7 (9.07)

31.8 (17.62)

36.7 (22.07)

40.8 (13.58)

74.5 (12.07)

0.07

0.01

0.05

0.36

-0.03

-0.02

0.03

0.32

0.65

0.54

0.29

0.75

0.81

0.65

0.61

0.78

HC SZ SZHC HC SZ SZHC

II IVdata from1

CHRONIC SCHIZOPHRENIA