ATransparency Framework Providing Evidence of Student Learning · back, and accessible by multiple...
Transcript of ATransparency Framework Providing Evidence of Student Learning · back, and accessible by multiple...
NILOA National Institute for Learning Outcomes Assessment
Introduction
Gathering evidence of academicaccomplishment – the knowl-
edge, skills and competencies stu-dents gain as a result of their collegeexperience, is a continuous process.It occurs at various levels and acrossmany dimensions, and the findingsare intended to enable institutions tomake improvements, assess the im-pact of changes in academic programsand provide evidence of learning out-comes to those to whom they are ac-countable. The TransparencyFramework addresses the question:How might these efforts be mademore visible? One avenue adopted bymany campuses is to share relevantinformation about student learningon the institutional website.
The National Institute for LearningOutcomes Assessment (NILOA) hasdeveloped a Transparency Frameworkto support institutions in sharing evi-dence of student learning on and offcampus. The Framework is based on areview of institutional websites1 andidentifies six key components of stu-dent learning assessment. Institutionsmay use the Framework to examinetheir institutional websites to gauge theextent to which evidence of student ac-complishment is readily accessible andpotentially useful and meaningful tothe intended audience. Each section ofthe Framework suggests a componentof student learning assessment thatmay be of interest to specific audiencesand outlines opportunities to advancepublic understanding.
The Framework also incorporatesbasic premises about website commu-nication. For instance, informationplaced on websites should be mean-ingful and understandable to multipleaudiences. To effectively communi-cate with various audiences, the web-site should enable users to providefeedback or offer comments on theposted material. Just as making stu-dent learning outcomes more trans-
parent is a work in progress, so is thisFramework. As more institutionsmake their assessment efforts accessi-ble and post meaningful evidence ofstudent learning, we will update theTransparency Framework using expe-rience from the field.
Transparency Framework
The Framework outlines components of a transparent institutional website in terms of student learning. Institutions should take into account the website’s purpose, structure, in-tended audiences, capacity of tech-nology staff and other available resources when reviewing the compo-nents. Information on student learn-ing can and should be presented in language that is understandable by specific and multiple audiences, widely available across the website, updated regularly, receptive to feed-back, and accessible by multiple web browsers and in various Internet speeds. A fully transparent website might well contain information about the following six components:
• Student learningoutcomesstatements
• Assessment plans• Assessment
resources• Current assessment activities• Evidence of student learning• Use of student learning evidence
How to use the TransparencyFramework2
The Framework is not a checklist tobe followed, but rather a guide tosuggest priorities and possibilitieswith an eye toward communicatingmeaningful information about stu-dent learning that will be useful tovarious audiences. Our review of in-stitutional website transparency sug-gests that the degree of transparencycan be viewed as a continuum fromnot transparent to transparent.
At this point in time, few institutionshave fully addressed the transparencychallenge. For example, an institu-tional website may be transparentwith specific components such as stu-dent learning outcomes statementsand current assessment activities, butnot transparent in others such as ref-erences to evidence of student learn-ing and/or the uses of evidence. Forthese reasons, each component of theFramework should be considered in-dividually when reviewing the entireinstitutional website to ensure thatinformation related to student learn-ing is prominently posted, clearlyworded, and updated regularly.
NILOANATIONALINSTITUTEFORLEARNINGOUTCOMESASSESSMENT
Not transparent
Transparent
Issues of transparency can be found in the following NILOA reports:
Jankowski, N. A., Ikenberry, S. 0., Kinzie, J., Kuh, G. D., Shenoy, G. F., & Baker, G. R. (2012). Transparency & accountability: An evaluation of the VSA college portrait pilot.
Jankowski, N., & Provezis, S. (20 l l ). Making Student learning Evidence Transparent: The State of the Art.
Jankowski, N., & Makela, J. P. (2010). Exploring the landscape: What institutional websites reveal about student learning outcomes activities.
ProvidingEvidenceofStudentLearning:ATransparencyFramework
February2011
NILOA NATIONAL INSTITUTE FOR LEARNING OUTCOMES ASSESSMENT
www.learningoutcomesassessment.org/TransparencyFramework.htm
Thisdocumentprovidesaframeworkforthinkingaboutcommunicatingstudentlearningassessmentinformation.AdditionaldetailscanbefoundontheNILOAwebpageincludingexamplesofinstitutionsthatarepostingassessmentinformationininnovativeandimportantways.
See:www.learningoutcomesassessment.org/TransparencyFramework.htm
The Transparency Framework serves as a basis for Excellence in Assessmentdesignation. It is also being endorsed by College Portrait of Undergraduate Education,Voluntary Framework of Accountability, Association of American Colleges andUniversities, American Association of State Colleges and Universities, and Association ofPublic & Land-Grant Universities. To view institutions that are using theTransparency Framework on their websites, seehttp://www.learningoutcomesassessment.org/TFfield.htm
See Development of NILDA s Transparency Framework, http:/ /learningou tcomesassessmen t. org/
documents/T ransparency%20 Framework%
20Development%202016.pdf
See the NILOA website for clear definitions of Transparency, Audiences, Not Transparent, and Transparent.
1
2 See the NILOA website for clear definitions of Transparency, Audiences, Not Transparent, andTransparent.
2
See Development of NILOA’s Transparency Framework, http://learningoutcomesassessment.org/documents/Transparency%20Framework% 20Development%202016.pdf
1
Transparency Framework: Components of Student Learning Assessment
Student LearningOutcomes Statements
Student learning outcomes statementsclearly state the expected knowledge,skills, attitudes, competencies, andhabits of mind that students areexpected to acquire at an institutionof higher education.3
Transparent student learning out-comes statements are:• Specific to institutional level and/
or program level• Clearly expressed and understand-
able by multiple audiences,including students
• Prominently posted or linked tomultiple places across the website
• Updated regularly to reflect currentoutcomes
• Receptive to feedback or commentson the quality and utility of theinformation provided
AssessmentPlans
Campus plans for gathering evidenceof student learning might includeinstitution-wide or program specificapproaches that convey how studentlearning will be assessed, the datacollection tools and approaches thatwill be used, and the timeline forimplementation.
Transparent assessment plans are:• Descriptive of institutional assess-
ment processes, procedures andactivities
• Inclusive of assessment measuresincluding what they are, how theyare applied, and frequency ofgathering evidence
•
•
• Updated regularly to reflect currentactivities
• Downloaded or accessed in a timelyfashion by multiple web browsers
and Internet download speeds• Receptive to feedback or comments
on the quality and utility of theinformation provided
AssessmentResources
Assessment resources encompassinformation or training provided tofaculty and staff to help themunderstand, develop, implement,communicate, and use evidence ofstudent learning.
Transparent assessment resources are:• Presented in language that is
understandable to the novice• Prominently posted or linked to in
multiple places across the website• Downloadable or accessible in a
timely fashion by multiple webbrowsers and Internet downloadspeeds
• Updated regularly to reflect currentactivities and new resources
• Receptive to feedback or commentson additional resources, training,or potential professional develop-ment needs
Current AssessmentActivities
Current assessment activities include information on a full range of projects and activities recently completed or currently underway to gauge student learning, make improvements, or respond to accountability interests.
Transparent current assessment activities are:• Clearly stated in language that is
understandable for variousaudiences
• Described and illustrated to clarifyhow a particular assessment activ-ity is used and how it relates to theinstitutional mission and vision
• Prominently posted or linked to inmultiple places across the website
• Updated regularly to reflect currentactivities
• Receptive to feedback on the pres-entation of the current assessmentactivities by having contact infor-mation or a form to send feedback
Evidence ofStudent Learning
Evidence of student learning includesresults of assessment activities. Thismay include evidence of indirect (e.g.surveys) and direct (e.g. portfolio)student learning as well as institution-al performance indicators (e.g.licensure pass rate).
Transparent evidence of studentlearning is:• Explained, analyzed, and inter-
preted in lay person’s language• Contextualized to clarify what the
results mean to the institution andto student learning
• Presented using both text andgraphics when appropriate
• Disseminated and summarized fordifferent groups, cohorts of stu-dents, and compared with peer in-stitutions or programs if appropriate
• Prominently posted or linked to inmultiple places across the website
• Updated regularly to reflectcurrent results
• Receptive to feedback on themeaning and interpretation of theevidence
Use of StudentLearning Evidence
This component represents the extentto which evidence of student learningis used to identify areas where changesin policies and practices may lead toimprovement, inform institutionaldecision-making, problem identifica-tion, planning, goal setting, facultydevelopment, course revision, pro-gram review, and accountability oraccreditation self-study.
Transparent uses of student learningevidence are:• Targeted to a particular audience
such as faculty, staff, administra-tors, students, families or govern-ing board members
• Inclusive of examples ofdocumented use of assessmentresults and information
• Focused on improvement ofstudent performance and
institutional processes through theuse of evidence
• Inclusive of next steps• Clearly stated in language that is
understandable for specific andmultiple audiences
• Prominently posted or linked to inmultiple places across the website
• Updated regularly to reflect currentactivities
• Receptive to feedback on selected
courses of action by having aperson’s contact information or aform to send feedback
3 Suskie, L. (2009). Assessing student learning(2nd ed). San Francisco: Jossey-Bass.
NILOA NATIONAL INSTITUTE FOR LEARNING OUTCOMES ASSESSMENT
www.learningoutcomesassessment.org/TransparencyFramework.htm
NILOA Transparency Framework
Clearly expressed and understand-able by multiple audiences Prominently posted or linked to multiple places across the website