NILOA National Institute for Learning Outcomes Assessment

Introduction

Gathering evidence of academicaccomplishment – the knowl-

edge, skills and competencies stu-dents gain as a result of their collegeexperience, is a continuous process.It occurs at various levels and acrossmany dimensions, and the findingsare intended to enable institutions tomake improvements, assess the im-pact of changes in academic programsand provide evidence of learning out-comes to those to whom they are ac-countable. The TransparencyFramework addresses the question:How might these efforts be mademore visible? One avenue adopted bymany campuses is to share relevantinformation about student learningon the institutional website.

The National Institute for LearningOutcomes Assessment (NILOA) hasdeveloped a Transparency Frameworkto support institutions in sharing evi-dence of student learning on and offcampus. The Framework is based on areview of institutional websites1 andidentifies six key components of stu-dent learning assessment. Institutionsmay use the Framework to examinetheir institutional websites to gauge theextent to which evidence of student ac-complishment is readily accessible andpotentially useful and meaningful tothe intended audience. Each section ofthe Framework suggests a componentof student learning assessment thatmay be of interest to specific audiencesand outlines opportunities to advancepublic understanding.

The Framework also incorporatesbasic premises about website commu-nication. For instance, informationplaced on websites should be mean-ingful and understandable to multipleaudiences. To effectively communi-cate with various audiences, the web-site should enable users to providefeedback or offer comments on theposted material. Just as making stu-dent learning outcomes more trans-

parent is a work in progress, so is thisFramework. As more institutionsmake their assessment efforts accessi-ble and post meaningful evidence ofstudent learning, we will update theTransparency Framework using expe-rience from the field.

Transparency Framework

The Framework outlines components of a transparent institutional website in terms of student learning. Institutions should take into account the website’s purpose, structure, in-tended audiences, capacity of tech-nology staff and other available resources when reviewing the compo-nents. Information on student learn-ing can and should be presented in language that is understandable by specific and multiple audiences, widely available across the website, updated regularly, receptive to feed-back, and accessible by multiple web browsers and in various Internet speeds. A fully transparent website might well contain information about the following six components:

• Student learningoutcomesstatements

• Assessment plans• Assessment

resources• Current assessment activities• Evidence of student learning• Use of student learning evidence

How to use the TransparencyFramework2

The Framework is not a checklist tobe followed, but rather a guide tosuggest priorities and possibilitieswith an eye toward communicatingmeaningful information about stu-dent learning that will be useful tovarious audiences. Our review of in-stitutional website transparency sug-gests that the degree of transparencycan be viewed as a continuum fromnot transparent to transparent.

At this point in time, few institutionshave fully addressed the transparencychallenge. For example, an institu-tional website may be transparentwith specific components such as stu-dent learning outcomes statementsand current assessment activities, butnot transparent in others such as ref-erences to evidence of student learn-ing and/or the uses of evidence. Forthese reasons, each component of theFramework should be considered in-dividually when reviewing the entireinstitutional website to ensure thatinformation related to student learn-ing is prominently posted, clearlyworded, and updated regularly.

NILOANATIONALINSTITUTEFORLEARNINGOUTCOMESASSESSMENT

Not transparent

Transparent

Issues of transparency can be found in the following NILOA reports:

Jankowski, N. A., Ikenberry, S. 0., Kinzie, J., Kuh, G. D., Shenoy, G. F., & Baker, G. R. (2012). Transparency & accountability: An evaluation of the VSA college portrait pilot.

Jankowski, N., & Provezis, S. (20 l l ). Making Student learning Evidence Transparent: The State of the Art.

Jankowski, N., & Makela, J. P. (2010). Exploring the landscape: What institutional websites reveal about student learning outcomes activities.

ProvidingEvidenceofStudentLearning:ATransparencyFramework

February2011

NILOA NATIONAL INSTITUTE FOR LEARNING OUTCOMES ASSESSMENT

www.learningoutcomesassessment.org/TransparencyFramework.htm

Thisdocumentprovidesaframeworkforthinkingaboutcommunicatingstudentlearningassessmentinformation.AdditionaldetailscanbefoundontheNILOAwebpageincludingexamplesofinstitutionsthatarepostingassessmentinformationininnovativeandimportantways.

See:www.learningoutcomesassessment.org/TransparencyFramework.htm

The Transparency Framework serves as a basis for Excellence in Assessmentdesignation. It is also being endorsed by College Portrait of Undergraduate Education,Voluntary Framework of Accountability, Association of American Colleges andUniversities, American Association of State Colleges and Universities, and Association ofPublic & Land-Grant Universities. To view institutions that are using theTransparency Framework on their websites, seehttp://www.learningoutcomesassessment.org/TFfield.htm

See Development of NILDA s Transparency Framework, http:/ /learningou tcomesassessmen t. org/

documents/T ransparency%20 Framework%

20Development%202016.pdf

See the NILOA website for clear definitions of Transparency, Audiences, Not Transparent, and Transparent.

1

2 See the NILOA website for clear definitions of Transparency, Audiences, Not Transparent, andTransparent.

2

See Development of NILOA’s Transparency Framework, http://learningoutcomesassessment.org/documents/Transparency%20Framework% 20Development%202016.pdf

1

Transparency Framework: Components of Student Learning Assessment

Student LearningOutcomes Statements

Student learning outcomes statementsclearly state the expected knowledge,skills, attitudes, competencies, andhabits of mind that students areexpected to acquire at an institutionof higher education.3

Transparent student learning out-comes statements are:• Specific to institutional level and/

or program level• Clearly expressed and understand-

able by multiple audiences,including students

• Prominently posted or linked tomultiple places across the website

• Updated regularly to reflect currentoutcomes

• Receptive to feedback or commentson the quality and utility of theinformation provided

AssessmentPlans

Campus plans for gathering evidenceof student learning might includeinstitution-wide or program specificapproaches that convey how studentlearning will be assessed, the datacollection tools and approaches thatwill be used, and the timeline forimplementation.

Transparent assessment plans are:• Descriptive of institutional assess-

ment processes, procedures andactivities

• Inclusive of assessment measuresincluding what they are, how theyare applied, and frequency ofgathering evidence

•

•

• Updated regularly to reflect currentactivities

• Downloaded or accessed in a timelyfashion by multiple web browsers

and Internet download speeds• Receptive to feedback or comments

on the quality and utility of theinformation provided

AssessmentResources

Assessment resources encompassinformation or training provided tofaculty and staff to help themunderstand, develop, implement,communicate, and use evidence ofstudent learning.

Transparent assessment resources are:• Presented in language that is

understandable to the novice• Prominently posted or linked to in

multiple places across the website• Downloadable or accessible in a

timely fashion by multiple webbrowsers and Internet downloadspeeds

• Updated regularly to reflect currentactivities and new resources

• Receptive to feedback or commentson additional resources, training,or potential professional develop-ment needs

Current AssessmentActivities

Current assessment activities include information on a full range of projects and activities recently completed or currently underway to gauge student learning, make improvements, or respond to accountability interests.

Transparent current assessment activities are:• Clearly stated in language that is

understandable for variousaudiences

• Described and illustrated to clarifyhow a particular assessment activ-ity is used and how it relates to theinstitutional mission and vision

• Prominently posted or linked to inmultiple places across the website

• Updated regularly to reflect currentactivities

• Receptive to feedback on the pres-entation of the current assessmentactivities by having contact infor-mation or a form to send feedback

Evidence ofStudent Learning

Evidence of student learning includesresults of assessment activities. Thismay include evidence of indirect (e.g.surveys) and direct (e.g. portfolio)student learning as well as institution-al performance indicators (e.g.licensure pass rate).

Transparent evidence of studentlearning is:• Explained, analyzed, and inter-

preted in lay person’s language• Contextualized to clarify what the

results mean to the institution andto student learning

• Presented using both text andgraphics when appropriate

• Disseminated and summarized fordifferent groups, cohorts of stu-dents, and compared with peer in-stitutions or programs if appropriate

• Prominently posted or linked to inmultiple places across the website

• Updated regularly to reflectcurrent results

• Receptive to feedback on themeaning and interpretation of theevidence

Use of StudentLearning Evidence

This component represents the extentto which evidence of student learningis used to identify areas where changesin policies and practices may lead toimprovement, inform institutionaldecision-making, problem identifica-tion, planning, goal setting, facultydevelopment, course revision, pro-gram review, and accountability oraccreditation self-study.

Transparent uses of student learningevidence are:• Targeted to a particular audience

such as faculty, staff, administra-tors, students, families or govern-ing board members

• Inclusive of examples ofdocumented use of assessmentresults and information

• Focused on improvement ofstudent performance and

institutional processes through theuse of evidence

• Inclusive of next steps• Clearly stated in language that is

understandable for specific andmultiple audiences

• Prominently posted or linked to inmultiple places across the website

• Updated regularly to reflect currentactivities

• Receptive to feedback on selected

courses of action by having aperson’s contact information or aform to send feedback

3 Suskie, L. (2009). Assessing student learning(2nd ed). San Francisco: Jossey-Bass.

NILOA NATIONAL INSTITUTE FOR LEARNING OUTCOMES ASSESSMENT

www.learningoutcomesassessment.org/TransparencyFramework.htm

NILOA Transparency Framework

Clearly expressed and understand-able by multiple audiences Prominently posted or linked to multiple places across the website