USER ACCEPTANCE TEST RESULTS

Subcontracts Modification Request Form

Prepared By: Puja Parakh, Business AnalystOffice of Research Information ServicesUniversity of WashingtonMarch 2013

Introduction To Project

Background

OSP Subcontracts Team receives many emails daily from campus administrators and Principal Investigators. Much of the team's time is spent following up on these communications, and they are looking for a solution to help streamline and improve their customer service. The impact will also be to reduce the communication time spent by campus to discuss the needs for their subcontract.

Scope of Work

ORIS will investigate and provide a public facing web form to be housed on the OSP Website. This form's purpose is to guide the user through a series of structured questions – then allowing for subcontract changes to come in from a single point of entry and therefore substituting for unstructured emails with lack of information. This project is to re-use development functionality from the recently created Advance Budget Tool.  

Solution to build: Form to use logic to direct user to provide specific information necessary to request changes to Subcontracts.

Business Performance Objectives

• Decrease subcontract cycle time with use of the new form• Reduce the administrative burden for OSP Subcontract Team• Reduce the administrative burden for the PI/Admin

2 of 18

Objectives of User Acceptance Testing

Primary ORIS Objective for Testing

User Acceptance Testing (UAT) is conducted to ensure that the system satisfies the needs of the business as specified in the functional requirements and provides confidence in its use, and ensure that all issues are addressed in an appropriate manner prior to implementation.

OSP Subcontracts Team Objective

Identify where the system would create confusion for campus users, resulting in additional frustration (phone calls) and therefore increasing the burden to both campus and the Subcontracts Team.

Roles and ResponsibilitiesRoles and Responsibilities

UAT is a collaborative process that utilizes team members with various levels of expertise to ensure success of the product.

Test results will be analyzed from the following different perspectives:

• Business Owner: Brandon Alpert

• Business Analyst: Puja Parakh

• Lead Developer: Gavin Elster

• Decision Points of Contact: Jackie Salgado and Lynette Arias

3 of 18

Method Used For Testing

Methodology

• Testing was conducted by five campus users as provided by the OSP Subcontracts team who acted as Subject Matter Experts.

• Users were expected to execute all test scripts manually based on provided scenarios created by the ORIS and OSP Team.

• Users may have also perform additional tests not detailed in the plan but remain relevant and within the scope of the project.

High Level Test Scenario

As a PI/Admin, I should indicate whether they would like to request one or more of the following:

• Extension• Supplement• Carry Forward• De-obligation

• Change in Subcontract PI• Change in UW PI• Change in Scope• Other requests

4 of 18

What We Did

First Scenario: Extension & Supplement with Change of Scope of Work

Second Scenario: Carry Forward

Third Scenario: De-obligation

Fourth Scenario: Change a UW PI

Fifth Scenario: Change a Subcontract PI

Entrance Q&A :

• Most users typically spend anywhere from a half hour to a week gathering required before requesting a subcontract modification.• All participants felt comfortable requesting a subcontract modification, and what was needed in order to do so.• There was a mix of users who emailed only vs. called Tru (all specifically mentioned her) with questions and submissions• Users mentioned that their pain point with subcontracts was understanding the turn-around time• Most users view the SC website about once a quarter (or less) unless prompted.

Scenarios :Provided a script to follow to perform five task-based scenarios based on real information at their desk.

Exit Q&A :

• All users really liked the new Subcontract Modification Request Form.• Users felt that it would give them the prompts needed to provide the correct information necessary.• They felt that the form was quick enough to complete and did not mention any increased burden.• Advanced users felt that they could have novice users in their office make simple requests without too much hand-holding.• The email the campus user receives after completing the form will be helpful in printing out, forwarding to co-workers, and filing to remind them to follow up later. • There was some concern that the attachments on email might take up too much space in inbox, however, users did appreciate and notice the attachments and did not process too many subcontracts that would exceed the limit on their e-mail applications.

5 of 18

First Scenario: Welcome Screen

1

2

3

1 P1: Unsure what was required here. “How will I know?”

P4: “What is proof of sponsor? Is that an email or doc?”

P5: “Assume proof of sponsor approval is the award with the list of subs”

2 P3: User kept trying to click on text instead of the button.

P5: User clicked on the text “Next” instead of button, but figured out on her own to move to the blue button.

3 Only one of five participants were actively looking for this email address. Most responses were, “I’ll call Tru if I have questions”

Recommendations:

- Remove “Click Next to Begin” and bring action buttons further up the page

- Clarify what is meant by Sponsor Approval

6 of 18

First Scenario: Contact Information

1

2

3

1 P1: The current form auto-populates an email address based on who is logged in. The user has a separate email address she uses for Subcontracts and was unable to use that address.

2 P2: Was a bit confused by the Employee Self-Service Message. We had to interrupt him to let him know that he did not need to change his contact information in the UW Directory.

3 P2: Wasn’t quite sure how strict this field was.

P5: Wasn’t sure if it was home department, unit department, etc. “School, division, unit?”

Both participants were able to complete the task and move on.

Recommendations:

- Change how email is populated & ESS alert

- Discuss need for changing #3 label.

1

2

3

7 of 18

First Scenario: Request Options

1 P1, P2, and P4: Not aware of what all the terms meant, although further into testing there was some familiarity. All participants requested definitions, and several mentioned the need for scenarios. P4 said they would need to call OSP for clarification.

2 P2: Did not use footnotes as intended, instead relied on institutional knowledge (and he was wrong).

Recommendations:

- Define each term- Provide a scenario for each term- Move footnotes into the definition

1

2

P3, when prompted, expected to see “Termination of Contract” as an option.

8 of 18

First Scenario: Request an Extension

P2: “Date picker was intuitive and easy to use.”

Recommendations:

- None.

1

2

9 of 18

First Scenario: Request a Supplement

Recommendations:

- None.

1

2

This page was successful during testing.

Users liked the ability to attach documents.

10 of 18

First Scenario: Change Scope

1 P3: A scenario is helpful here, but the user did not need it to succeed with the task.

Recommendations:

- While this page was successful, it might be worth considering adding a definition and/or scenario on this page as well.

1

2

1

11 of 18

First Scenario: Final Step

1 P2 & P4: Wanted timeframe of response. Users mentioned various methods of tracking turnaround time (ex. flagging emails in inbox). One user mentioned they’d call OSP to find out more.

Recommendations:

- Add a timeframe of when user will hear back from Subcontracts Team

- Break instructions out into numbered list or other text format to clearly show next steps

1

P2: Wasn’t sure what was going to happen when hitting submit, but then saw the email in her inbox.

P3: Wanted to see a summary page before submission.

P3: Who to contact if edits need to be made?

12 of 18

Second Scenario: Carry Forward

1 There was a lot of confusion around how a Carry Forward would take place in this context. Some users referred to this as Carry Over, and all mentioned the need of a PO change.

Recommendations:

- Definition/Scenario on this page- Use “Budget” in title- Discuss how to handle Carry Forward options. Lots of confusion.

1

1

13 of 18

Third Scenario: De-obligation

1 P2, P4, & P5: Expected to change scope of work as well. Some users mentioned it out loud, while P4 specifically went backward and selected Change of Scope of Work as well. Some mention of Budget Justification as well.

There was some questions around exactly what a De-obligation was, but when given a scenario all users succeeded.

Recommendations:

- Provide a definition/scenario- Follow up with Change of Scope of Work question1

14 of 18

Fourth Scenario: Change UW PI

1 Every user felt comfortable on this screen.

Recommendations:

- None.

15 of 18

Fifth Scenario: Change SC PI

1 P4: Wasn’t sure what date to list here. Assumed it was when a new PI begins work.

Recommendations:

- Clarify ‘Effective Date’ label

1

16 of 18

Summary of Recommendations

1. ) Entry Screen:

A. Remove “Click Next to Begin”and bring action buttons further up the page

B. Clarify what is meant by Sponsor Approval

2. ) Requester Information:

A. Change how email is populated / ESS Notification

B. Discuss need for changing #3 label

3. ) Request Checkboxes:

A. Provide a definition/scenario for each term

B. Move footnotes into the definition

4. ) De-obligation:

A. Provide a definition/scenario

B. Follow up with Change of Scope of Work question

5. ) Carry Forward:

A. Definition/Scenario on this page

B. Use ‘Budget’ in title

C. Needs discussion with business

6. ) Final Step:

A. Add a timeframe of when user will hear back from SC Team

B. Break instructions out into numbered list or other text format to clearly show next steps

17 of 18

Next Steps*

Analysis Design Develop Test Dev/QA

1.) Assess Recommendations

Identify where there can be improvements made to the form in order to reduce burden on OSP as well as PI/Admin

2.) Prioritize Recommendations

Using this document as a guide, prioritize the request of work, identify ownership, and indicate deadlines

Release & Communicate

1.) Estimate Development

Work with ORIS Engagement and Development to understand the amount of work that will need to be done

2.) Identify Ownership

Work with Subcontracts Team to identify their role in Development

3.) ORIS to Develop and QA

Ensure that ORIS meets the business objectives and quality standards

1.) Release

ORIS to work with SC Team for final release and location to place form on Website

2.) Communications

Provide any resources necessary for Subcontracts Team to announce new method for requesting modifications

* To be discussed... of course :)

18 of 18