WARI Comparative Analysis Report forUportalThe project team will use the comparative analysis...

Wolverine Access Redesign & Implementation (WARI) Comparative Analysis Report

WARI Comparative Analysis Report_forUportal.doc 9/14/2009 of 14

I. Overview and Summary The WARI User Experience (UX) team conducted a Comparative Analysis to analyze several universities’ web portals to determine functionality that the WARI project could consider implementing, and approaches that the WARI project could consider following in Phase 2. In total, portals and websites from 33 universities were inspected for the comparative analysis. Below, we report findings on the following topics:

Portal Audiences Publicly Available Information Methods for Login Organization and Naming of Tabs Channels and Portal Functionality Transactional Information Personalization Features Methods for Collecting Feedback Methods for Collecting and Analyzing Site Usage Statistics Portal Development and Management Processes

In the final sections of this document, the UX team makes recommendations and considerations for future phases. Note that the recommendations listed here are recommendations from the comparative analysis alone. These should not be confused with the recommendations that are the outcome of the prioritization process. The project team will use the comparative analysis recommendations as one input (in addition to prioritization survey results, Gateway Service Catalog work, and other inputs) in producing the overall implementation recommendations. II. Methodology The UX team performed the following steps to conduct its comparative analysis:

1. Determine criteria for universities to investigate. These criteria included: a. Universities using uPortal v3.0 and higher. b. “Competitor universities” similar in size to UM, including CIC universities. (In these competitor

universities, the UX team looked for web portals comparable to uPortal.) c. Universities using older versions of uPortal that are interesting for other reasons.

2. Develop initial list of questions for UX team members to answer while inspecting each site in step 1. 3. Perform “visual inspection” of sites in step 1, to determine the answers to the questions developed in

Step 2. Due to lack of access to sites at this point, this visual inspection is necessarily superficial. The UX team performed visual inspections of portals from 33 universities: 8 CIC universities not currently using uPortal, and 25 universities using uPortal.

4. Gather contacts from uPortal and CIC universities from step 1. 5. Draft survey questions. 6. Send survey questions to uPortal and CIC contacts. 12 uPortal universities and 8 CIC universities not

currently using uPortal responded to the survey.



7. Contact a subset of universities from step 1 and conduct follow-up interviews with them. 8. Compile findings from visual inspection, survey, and interviews.

III. Findings This section lists findings from the visual inspection, survey, and interview, arranged topically. Note: a ‘channel’ on a web portal like uPortal is a self-contained piece of content. It usually allows a user to perform one task or a small number of related tasks. Channels are often displayed surrounded by a border of some kind, so that multiple channels can fit on a single page of the portal.

Image 1 - "Reporting" represents a channel on the Wolverine Access Gateway



A. Portal Audiences The UX team investigated what audiences each university uPortal was serving, and how university portals dealt with people who belong to multiple audiences (i.e. a student who is also a staff member)

Most uPortal universities offer content to students, faculty, and staff. However, o Several universities (Cal Poly, Duke, University of New England, Sault College) limit

scope to Students, or Students and Alumni.

o UC Irvine has 2 separate uPortal implementations-- one for Students, and one for Employees.

o In addition to its main portal, Johns Hopkins has 2 separate portals run by different schools: the Business School and School of Advanced International Studies.

To mitigate the problems of multiple overlapping audiences (students who are also staff, Faculty who have to run reports, etc) to some extent, uPortal universities are using the following methods:

o Using a topical organization of tabs. In a topical organization of tabs, different tabs do not denote different audience segments; instead, tabs organize the site by topic. At UW-Madison, students who are also staff just look at the Work Record tab for their paycheck, just like non-student staff. They don't have a Student tab versus a Staff tab.

Image 2 - Topical Organization of Tabs

o Provide a separate portal for each audience group, for example, UC-Irvine has a

separate Students portal vs. Employees.

o Allow users to have primary and secondary roles. For example, in the survey results, a representative from one uPortal university stated that "Each user has one primary role, but may have multiple roles. For example, I may be a staff and student; staff takes precedence and is my primary role, but I will also see content intended for students."



B. Publicly Available Information The UX team investigated the content available to the public-- that is, available without logging in-- on university uPortals. For most uPortal universities, the tabs and channels that are displayed prior to logging in (publicly

available) differ from the set of tabs and channels that the users see once they have logged in. Most of the content is contained behind login, with items like announcements, news, and help information on the publicly available site.

Image 3 - Tabs before Login

Image 4 - Tabs after Login

There were several channels that uPortal universities commonly made available to the public. These include: system status, campus news, "about the portal" information, links to commonly used university resources, and both directory search and campus website search. Many campuses also had weather and news feeds (New York Times, campus newspaper) channels available to the public.



Image 5 - Publicly available portal homepage with news and weather information

Many portals have publicly available help that users can access without logging in. This includes: explanations of the portal, portal behavior, portal functionality, and system information, in various formats (eLearning, screenshots, help documents, Q&As, etc.)



C. Methods for Login The UX team investigated the various methods that uPortal universities are using present portal login to users.

There was no overwhelming trend for this, with different universities handling the login in different ways, depending on the authentication systems (systems similar to cosign) and uPortal theme that each university was using. Some examples include:

o Including a login button on the publicly available site that directs users to a separate log in page.

Image 6 - Login button on portal homepage directs users to a separate log in page.

o Including login area with fields for user name and password on the publicly available portal homepage.

Image 7 - Login area on portal homepage



o Including login area with fields for user name and password in a publicly available channel on the portal homepage.

Image 8 – Login area in a channel on homepage

o Having the site’s URL (for example: http://my.wisc.edu) lead directly to a login page, without having an intermediate publicly available page. This could include having a link to launch the portal on the university’s main homepage.

D. Organization and Naming of Tabs The UX team investigated the names and organization of tabs on university uPortals.

Of the 7 universities from which the UX team was able to see all tab names, 6 universities' tabs were topical. That is, different tabs did not denote different audience segments; instead, tabs organized the site by topic.

o Common tab names include "Home", "Services", "Courses" or "Academics", "My Stuff" or "myInfo" or "Personal Info" or "Personal", "Library", and "Money" or "Finances".

Image 9 - Topical Organization of Tabs



A common approach is to have a mostly common set of tabs for all users, and one or two tabs that vary for different audiences. It is less common to have a completely separate set of tabs for different audiences. Channels on each tab can be delivered or not delivered to certain audiences.

E. Channels and Portal Functionality The UX team investigated the different types of channels and portal features being used by other university portals, to look for overall themes, and find any ideas that would enhance the Wolverine Access uPortal Gateway.

Most universities are using a combination of university-specific channels (to access university resources and business transactions) and more general channels (news, weather, etc.).

Most universities are using at least one channel to provide announcements to portal users. Many are using more than one channel, each with a different focus. These include channels like: Emergency alerts, what’s new in the portal, IT System Status, and general IT updates and news.

Image 10 - Multiple Announcement Channels



Image 11 - Example of an IT Systems Status Channel

Many portals contain channels that provide links out to information and sites related to the

services contained on a tab. For example, an “Academics” tab would provide a channel linking out to the academic calendar, library, etc.

Image 12 - Example of "Supplementary" Links in a Channel



Many universities are allowing users to set up Quick Links or bookmarks in the portal that take them directly to frequently used systems. One university even provided the ability for users to download a toolbar for their browser that contained these bookmarks.

Image 13 - Example of Bookmarks Toolbar

Many portals provide users with a search feature that allows them to perform searches of

websites including: the portal, the student/staff directory, the university’s main website, etc.

Many portals provide users with a way to submit and track IT help tickets.

Image 14 - Help Ticket Tracking in a Channel

Many universities allowed users to control how channels appear on the portal, including zooming in to make a channel a full page, or zooming out to see only the title bar of a channel.

In addition to the above, there were many channels and functionality included in the various portals that the UX team noted and contributed to the list of possible implementation ideas that will undergo the prioritization process.



F. Transactional Information The UX team investigated whether university uPortals used “transactional” channels. By transactional, the UX team refers to channels that allow users to add, edit, or delete data; contrasted with non-transactional channels, which allow users to search for and view data.

Surprisingly few of the representatives of uPortal universities said that their portals contained

"transactional portlets", which we are using to mean portlets in which users can write data instead of simply reading it.

o One university allowed transactions to be completed in the portal via web proxy to the source system.

o Duke allows the transfer of money from student bank accounts to a DukeCard via a portlet.

o California Polytechnic University - San Luis Obispo allows users to complete various requests through transactional portlets (requesting alternate testing arrangements for disabled students, voting on certain referendums, etc).

G. Personalization Features The UX team investigated user personalization of channels and layout on university uPortals. University uPortals that allow personalization allow users to add and remove certain channels from certain tabs.

Most uPortal universities allowed user personalization. Of the 15 uPortal universities where it was possible to tell whether they allowed user personalization or not, only 2 did not allow user personalization. However, the type of personalization varied.

o Some uPortal universities allow users to change the number of columns on tabs, or change the overall color scheme ("skin") of the site.

o Two universities (Sacramento State University and Sault College) have a "My Stuff" tab, which is the only tab that users can customize.

o One university (Bristol University) allows users to rename tabs.

UW-Madison reported (anecdotally) that only a small minority of users actually use the personalization features, although that minority is quite passionate about it. This suggests the need for careful design of default tab and channel layout, since a large number of users may not change the defaults.



H. Methods for Collecting Feedback The UX team looked at the various methods that uPortal universities were using to collect feedback from their users regarding the use and features of the portal. Most universities are collecting feedback using various methods, including: providing an email

address on the portal, creating a feedback form in a channel, using a survey to collect feedback, etc.

Many portals provide an opportunity for users to provide feedback from every page, using a persistent link in the navigation.

Several universities reported that they receive feedback through helpdesk tickets and stakeholder

committees.

There are a few universities using the uPortal standard feedback portlet.

I. Methods for Collecting and Analyzing Site Usage Statistics The UX team asked representatives from uPortal universities about the methods they were using to collect statistics on the use of their portals.

Overall, there are only a few universities who are collecting and analyzing site statistics for their uPortal implementation. These universities are using the following methods:

o Mining server log information for the portal server and authentication server. o Implementing Google Analytics or similar analytics software o Using the statistics functionality available in uPortal (versions 2.5, 2.6, 3.0, and 3.1)

J. Portal Development and Management Processes The UX team asked representatives from uPortal universities about the process that they used in designing the initial content available on their portal, and the process they use for approving or not approving requests for additional content.

Several universities mentioned receiving requests for enhancements through selected help desk

tickets, stakeholder committees, or through focus groups. This validates the UX team's user research approach, which proposes focus groups as a possible research method, and the portal governance committee approach.

Two universities mentioned having trouble keeping stakeholder committees active after the portal had been in place for several years. The portal governance committee should consider ways to work against this tendency.

UW-Madison performed a card sort, as one way to design portal information architecture. This also validates the UX team’s user research approach, in which a card sort was also proposed.



Many universities reported that once the portal had been in place for a few years, they did not actively solicit ideas for new channels or functionality from users, but rather took requests as they came.

V. Recommendations from Comparative Analysis Based on the findings from the comparative analysis, the UX team recommends that the following items be considered for the Wolverine Access Gateway Portal.

Implement a topical organization of tabs, with tabs like “Academics” and “My Job”, rather than

audience-based tab organization (students, faculty). (Actual tab names will be designed after user research has been done.)

Allow users to personalize their portal (e.g. adding, removing, and organizing channels), with careful attention paid to the design and user experience of the default tab organization and channel layout for those users who will not use personalization features.

Widen available functionality and links to be implemented on portal beyond existing functionality/links available through the gateway to increase overall usefulness of the portal. These could include links to additional systems such as OARS, channels for highly used ITS functionality such as the Chartfield Converter, and links to more help and feedback opportunities.

Keep the skin the same pre-login vs. post-login, but consider having a different set of tabs pre-login versus post-login. (Actual tab names will be designed after user research has been done.)

Revamp the available Gateway Help and FAQ to include more information on overall use of the portal and portal features.

Collect more information on portal use and web analytics. This could include: what is being used in portal, number of people using personalization features, number of people adding or deleting channels, usage levels for tabs and channels, and user information.

VI. Considerations for Future Phases Based on the findings from the comparative analysis, the UX team recommends that the following items be considered for future phases of the Wolverine Access Gateway Portal.

If the overall scope of the portal is revealed to be too large, consider breaking up the portal into

separate portals for separate audiences (i.e. students versus faculty).

Consider identifying primary audience/customer groups, and deciding whether content for certain secondary audience groups can be moved to the public (pre-login) area.

Increase the scope of the portal to provide access to all services provided by ITS.

In the future, allow schools & colleges to create their own customized tabs for their students and staff. (could include school room schedule, student group information, links to course sites, etc)



In the future, integrate the Portal with CTools/Sakai. (Could include course calendars, assignment

submission, announcements from course sites.)

WARI Comparative Analysis Report forUportalThe project team will use the comparative analysis...

Documents

Transcript of WARI Comparative Analysis Report forUportalThe project team will use the comparative analysis...