1 Academic Learning Compacts Realigning FAU’s Assessment System to meet new SUS Guidelines.

1

Academic Learning Compacts

Realigning FAU’s Assessment System to meet new SUS

Guidelines

2

Slides 3 through 20 of this handout were covered in the workshop.

Slides 13 and 14 indicate action items for discussion and approval at a future BoT meeting.

Slides 21 and 22 provide a status report of the progress FAU’s academic departments have made on this initiative. By May 2005, all departments are expected to complete the first two columns, and by December 2005 all departments are expected to complete the last two columns.

Slides 23 through 38 provide supplemental materials and examples of the underlying terms found in slides 3 through 20 of the handout.

Several slides include comments from the April 13, 2005 BoT workshop. To view comments, right-click “Edit Slides.” Comments will appear as yellow boxes on several slides. Scroll and stop on a yellow box to view the comment.

3

What are ALCs? “Academic Learning Compacts…include concise

statements of what active and successful participants in the joint teaching-learning process will know and be able to do, expressed in terms of the core student learning outcomes embodied in the requirements for each baccalaureate degree. The Compacts also will list the types of assessments used in the program. Compacts provided to students must be written in a user-friendly, jargon-free format.”

Policy Guideline 04.08.26State University System/Division of

Colleges and Universities

4

Where did ALCs originate?

• In recent years, there has been increased emphasis on the identification and assessment of core student learning outcomes in higher education.

• Through its strategic planning process, the Florida Board of Governors has articulated the importance of student achievement.

• Board of Governors’ resolution of April 22, 2004 directed university personnel to develop ALCs and related processes.

5

What is the purpose of ALCs?

• Improve student achievement and program effectiveness

• One of seven BOG-approved accountability measures

• Part of annual Departmental Performance Review and Program Review.

6

How is FAU implementing ALCs?

ALCs fit within Florida Atlantic University’s established comprehensive assessment

system with minor modifications.

7

DEVELOPMENT OF ALCs

Each university shall construct clearly defined policies and procedures for developing, implementing, and reviewing Academic Learning Compacts and associated activities.

FAU is realigning the assessment processes it established in 2000 to meet ALC requirements.

In 2002, SACS affirmed and complimented FAU on the breadth and depth of its assessment process.

BoG Policy Guideline FAU’s Compliance

8

1. a. Identify, at a minimum, the expected core student learning outcomes for program graduates in the areas of

i. content/discipline knowledge and skills;

ii. communication skills*;

iii.critical thinking skills*;

* these skills may be defined at either the university level or the program level

FAU’s programs identify one or more of each of the following outcomes:

• Content knowledge– Terms, Theories– Research skills– Technical skills

• Communication skills*

Written – Oral – Other

• Critical Thinking skills*

Analytical – Creative – Practical

* FAU’s Writing Across the Curriculum requirement


Tom Pusateri

During the April 2005 workshop, some BoT members suggested that departments articulate two or more distinct outcomes for each category (e.g., at least two distinct outcomes for content, two for communication, two for critical thinking)

9

1. b. Identify corresponding assessments used to determine how well student learning matches those articulated expectations.

Where/When are students assessed?

How are students held to performance standards?

Departments assess all students through one or more of the following:

• Core required courses

• Capstone experience– Honors thesis– Senior

project/portfolio– Field placement

• Standardized test


Tom Pusateri

During the April 2005 workshop, some BoT members did not feel that a list of "core required courses" alone would be sufficient. Departments who choose this option should be prepared to provide justification for how the content and assignments in core required courses adequately represent the expected outcomes for Academic Learning Compacts.

Tom Pusateri

During the April 2005 workshop, the BoT indicated that all departments should aspire to develop senior capstone experiences for all undergraduate degree programs. If some departments would require resources to achieve this objective, this could be included in the department's submission to FAU's Strategic Plan.

10

Departments establish standards through one or more approaches:

• Standardizing courses– Common syllabi– Common expectations– Common assignments

• Grading rubrics– Faculty define levels

of student performance

– Periodic calibration• Setting cut-off scores on

required exams

1. b. Identify corresponding assessments used to determine how well student learning matches those articulated expectations.

Where/When are students assessed?

How are students held to performance standards?


Tom Pusateri

During the April 2005 workshop, the BoT suggested that all departments develop grading rubrics for assignments used in Academic Learning Compacts that require subjective scoring of student work.

11

2. University personnel will develop the evaluation systems (including external validations) necessary to corroborate that the assessments referenced above measure student achievement against the expected core learning outcomes…. (continued)

FAU’s departments rely on one or more systems:

• Multiple raters– Juried performances– 2+ faculty members– Non-faculty raters

• SamplingCollection and periodic review of student work

• Comparisons to norms on standardized tests


12

(continued)… and results will be used to improve student achievement and program effectiveness.

FAU’S online Assessment Reporting Database

Each year (since 2000), departments enter reports in which they provide data from assessment studies and identify resulting program improvements.

In Summer 2005, FAU will align this database for both ALC and SACS reporting.


13

[U]niversity personnel shall develop an action plan that will be approved by their university’s board of trustees and submitted to the Division of Colleges and Universities.

(There are two components of this action plan that are presented on the next two slides.)

History of developing FAU’s action plan

• Briefing at University Undergraduate Programs Committee (Sep 04)

• Consultations with academic departments(Fall 04 - Spring 05)

• Progress report to Faculty Senate Chair (Mar 05)

• Board of TrusteesInformation (Oct 04)Workshop (Apr 05)Approval (May 05)


14

The plan shall include the following:

• A proposed timeline for developing policies and implementing procedures to capture each element outlined above, as well as in making Academic Learning Compacts readily available to students.

Status reports to BoG: 5-05: Initial report 9-05: Follow-up report Policies/procedures to

be completed by 9-0512-05: Final report

ALCs will appear on departmental Web

sites during Fall 2005


15

The plan shall include the following:

• A description of how university personnel will certify that each baccalaureate graduate has completed a program with clearly articulated core student learning expectations in content/discipline knowledge and skills, communication skills, and critical thinking skills and that corresponding robust and effective assessment mechanisms have been used to ensure that graduates have met the criteria of the Compacts.

FAU will certify students through the awarding of a baccalaureate degree from programs that have articulated ALCs and that require students to achieve acceptable performance standards prior to graduation.


16

Academic Learning Compacts ensure that no student will graduate without meeting the requirement of:

• a senior capstone course, or

• an exit examination, or

• a series of required courses

in which students have demonstrated sufficient competence in content knowledge, communication skills, and critical thinking skills within the discipline.

Tom Pusateri

See comment on Slide 9.

17

In addition to submitting the action plan, university personnel shall submit a status report that includes an analysis of the progress being made in each baccalaureate program toward the development and implementation of the elements outlined above, as well as in making Academic Learning Compacts readily available to students.

See Slide 14


18

IMPLEMENTATION OF ALCS

University personnel will provide students and prospective students with Academic Learning Compacts, which include concise statements of what active and successful participants in the joint teaching-learning process will know and be able to do, expressed in terms of the core student learning outcomes embodied in the requirements for each baccalaureate degree. The Compacts also will list the types of assessments used in the program. Compacts provided to students must be written in a user-friendly, jargon-free format.

Examples from FAU are on a Web site:

http://iea.fau.edu/pusateri/assess/

ALC.htm


19

University policies and procedures shall delineate how and where Academic Learning Compacts will be made readily available to students. However, the Academic Learning Compacts must be made available to students beginning no later than the fall semester of 2005.

ALCs will be posted on departmental Web pages.

Future undergraduate catalogs will include a general description of ALCs and will visit departmental Web pages for the ALCs for specific degree programs.


20

REVIEW OF ALCsUniversity personnel are expected to demonstrate how information from the periodic review of student learning outcomes, as well as from the evaluation of corresponding assessment mechanisms, has been used to improve student achievement and program effectiveness.

This is an integral part

of each Dean’s annual

departmentalperformance reviews


21

FAU's Progress on Academic Learning Compacts

(as of April 13, 2005)

No Action

Plan Entered, Much Revision needed

Plan Entered, Some Revision needed

Task Complete

COLLEGE

Program Title (Major)

Learning Outcomes

Effective Assessments

EvaluationSystems

Available to

Students

CAUPA

Architecture

Criminal Justice

Public Management

Social Work

Urban and Regional Planning

ARTS & LETTERS

Anthropology

Art

Interdisciplinary Studies

Multimedia Studies

Communication

English

History

Jewish Studies

Linguistics, It, Ger, Fr, Sp

Music

Philosophy

Political Science

Sociology

Theatre

22

COLLEGE Program Title (Major)Learning Outcome

s

Effective Assessment

s

Evaluation

Systems

Available to Students

BUSINESS

Accounting

Economics

Finance

Real Estate

Health Services

Management Information Systems

Management

International Business & Trade

Marketing

EDUCATION

Varying Exceptionalities Program

Exercise Science & Health Promotion

Florida Teacher Certification

ENGINEERING

Civil Engineering

Computer Science/Engineering

Electrical Engineering

Mechanical Engineering

Ocean Engineering

HONORS Liberal Arts & Sciences

NURSING Nursing

SCIENCE

Biology

Chemistry

Geology, Geography

Mathematics

Physics

Psychology

23

FAU’S DEFINITIONS OF CONTENT KNOWLEDGE IN THE DISCIPLINE

Declarative knowledge: Students will demonstrate knowledge of the vocabulary, history, theories or concepts specific to the discipline. Declarative knowledge is usually assessed via in-class or standardized tests, typically in an objective (multiple-choice; short answer) format.

Procedural knowledge: Research skills: Students will demonstrate knowledge of the procedures involved in discipline-specific research (e.g., idea generation, literature review, data collection, reporting). These skills are usually measured via the evaluation of a research project, thesis, or dissertation.

Procedural knowledge: Technical skills: Student will demonstrate technical skills related to the discipline (e.g., developing a business report or lesson plan, mastering a musical instrument, learning how to use statistical software). This type of procedural knowledge may be assessed via an in-class project (e.g., case study, paper), observation (e.g., juried performance), or portfolio.

Tom Pusateri

During the April 2005 workshop, the BoT suggested that all departments identify outcomes for at least two of the categories below for each undergraduate degree program.

24

FAU’S DEFINITIONS OF COMMUNICATION SKILLS

Written communication: Students will produce writing that is grammatically correct, well-organized, and properly formatted for the purpose of the assignment and the discipline. This is usually assessed via individual paper assignments where instructors assess the quality of written expression.

Oral communication: Students will prepare and deliver informative and/or persuasive oral presentations that attend to the audience and are well-organized. This is usually assessed via individual oral presentations in a classroom setting, although sometimes instructors and/or supervisors may assess oral communication across time (e.g., during classroom discussions or at an internship site).

Other forms of communication: Students will prepare and present information and/or persuasive material using media that may or may not include written or oral communication (e.g., a musical or theatrical performance; a visual, audiovisual, or graphical product). This is usually assessed via classroom projects, juried performances, and/or portfolios.

Team/Collaborative communication: Students will demonstrate team-oriented, collaborative skills in which they contribute to group products. This is usually assessed via group papers or presentations and may involve peer-assessment (e.g., group members assess each other) and/or assessment by audience members (e.g., an instructor, supervisor, or peer).

Tom Pusateri


25

FAU’S DEFINITIONS OF CRITICAL THINKING SKILLS(Based on Robert Sternberg’s Theory of Successful Intelligence)

Analytical skills: Students will analyze, evaluate, compare/contrast or judge discipline-specific theories, issues, events, or other content. This is usually assessed through examining the quality of argument in a student essay, oral presentation, or formal report.

Creative skills: Students will create a product by synthesizing knowledge from a discipline (e.g., create a new piece of art, interpret a piece of music in a personal and appropriate way, develop a new theory or research proposal). This is usually assessed by an instructor or supervisor examining a student product for the quality of its synthesis of current knowledge into a new product.

Practical skills: Students will put into practice their knowledge and skills within a discipline (e.g., developing a business proposal or lesson plan, writing a grant). This is assessed ideally by examining an authentic product the student employs in an appropriate setting (e.g., a student teacher delivering a lesson, an intern proposing a plan to a supervisor); it may also be assessed via in-class projects in which students propose practical solutions based on their understanding of the discipline (e.g., case study analysis).

Tom Pusateri


26

EXAMPLES OF ALCs CONTENT KNOWLEDGE

PSYCHOLOGY (Terms, theories): Graduates in Psychology will understand basic concepts, theories, and experimental findings in four core areas of psychology (cognitive, developmental, social, and psychobiology).

HISTORY (Research & Technical skills): Students will demonstrate the ability to conduct archival research utilizing both primary and secondary sources. They will develop procedural and technical skills involved in historical research such as library research, oral history, book review writing, and proper citations.

FINANCE (Technical skills): Students will develop proficiency in the use of Microsoft Excel to solve financial problems.

Tom Pusateri

During the April 2005 workshop, the BoT suggested changes in the wording of several of the examples. In this example, they suggested changing "understand" to an action verb such as "demonstrate and accurately apply".In this example, they also suggested that the department provide an additional outcome for content knowledge. This outcome refers to declarative content knowledge; the department should also include an example of procedural content knowlege (research skills) relevant to the discipline of psychology.

Tom Pusateri

During the April 2005 workshop, the BoT suggested changing "Microsoft Excel" to more generic language, e.g., "spreadsheet software" just in case the department chooses to teach a different program in the future.

27

EXAMPLES OF ALCs COMMUNICATION SKILLS

GEOGRAPHY: Graduates in Geography will be able to produce writing that is grammatically correct and well-organized and to deliver clear and well-organized oral presentations. Both written and oral communications should illustrate clear communication skills utilizing graphic tools in geographic information systems, remote sensing and other techniques.

REAL ESTATE: Students will demonstrate their abilities to report the results from their feasibility analyses both orally and in written form in a clear and grammatically sound manner. Students will also demonstrate skills in discussing the analyses presented by their peers.

TEACHER EDUCATION: Students will demonstrate the ability to communicate effectively verbally and in writing (EAP 2, Indicator 2.2) by successfully developing and implementing a lesson plan.

28

EXAMPLES OF ALCs CRITICAL THINKING SKILLS

ENGLISH: Students will analyze/interpret a complex literary work, demonstrating the capacity to formulate and develop in a formal essay a significant topic in depth and in an independent and even original way. Essays are expected to exhibit mature creative and analytical thinking, and an ability to create a mature critical argument quickly with no external assistance.

COMPUTER ANIMATION IN ARTS: Students will demonstrate proficiency in understanding and apply the stages of problem-solving skills (e.g. research, thumbnail, roughs, comprehensive and mechanical preparation) as applied to 3D computer arts.

ELECTRICAL ENGINEERING: Design and conduct scientific and engineering experiments including analysis and interpretation of data. Deliver engineering results that meet performance standards for cost, safety, and quality. Make and defend ethical judgments in keeping with professional standards.

29

CORE REQUIRED COURSES

PSYCHOLOGY: For each of the courses listed below, faculty teaching that course will develop a master syllabus which will include student learning outcomes to be addressed in all sections of the course. Faculty will share strategies for developing course assignments (e.g., exams, exercises, term papers) to assess student attainment of the learning outcomes. The courses are:EXP 3505: Cognition,DEP 3054: Psychology of Human Development,SOP 3004: Social Psychology, and PSB 3002: Biological Bases of Behavior.

MUSIC: All students must complete four specific courses in the history and literature of music: MUH 3013 (Introduction to Music History), MUH 4211 (Music of Western Civilization I), MUH 4212 (Music of Western Civilization II), and MUH 4371 (20th Century Music History).

Tom Pusateri

See comments on the BoT's concerns about core required courses as expressed during the April 2005 workshop.During this workshop, the BoT did not feel it is sufficient to simply pass a series of courses without a clear articulation of how each course assures students have demonstrated the key outcomes in the department's Academic Learning Compacts.

30

CAPSTONE EXPERIENCE

SOCIAL WORK: All BSW students are required to take SOW 4510 (Field Education in Social Work), which includes a classroom field seminar plus 400 agency-based hours supervised by a professional social worker. This experience integrates classroom knowledge from the Human Behavior in the Social Environment (HBSE) sequence, practice sequence, research sequence, and policy sequence; students use this knowledge base to develop skills to intervene with various clients.

HONORS COLLEGE: During the senior year, students will enroll in thesis research and thesis writing and will produce a draft honors thesis to be evaluated. To prepare for writing a senior thesis, students will complete (1) courses in their concentration aimed at providing the content knowledge needed to write a thesis, (2) writing courses aimed at providing competence in effective written communication, and (3) courses in the core and concentration designed to build critical thinking skills.

Tom Pusateri

During the April 2005 workshop, the BoT suggested that all departments develop capstone experiences in the future. Those departments who may need additional resources to accomplish this should enter this information into their strategic planning documents.

31

STANDARDIZED TEST

NURSING: Approximately three months prior to graduation, all seniors will take the Health Education Systems, Inc. (HESI) examination. This test is a valid predictor of performance on the National Licensing Examination (NCLEX). Students must achieve a score of 850 or better on the HESI to graduate. Students who do not receive a passing score are provided with advising and a remediation plan and the opportunity to re-take the exam; they will not graduate until their score is 850 or better.

OCEAN ENGINEERING: One of the requirements for graduation is that a student must take the nationally-administered Fundamentals of Engineering (FE) examination. The examination tests one's knowledge of basic and fundamental engineering subjects: statics, dynamics, thermodynamics, fluid mechanics, computer programming, engineering economics, electrical networks and strength of materials.

32

STANDARDIZING COURSES

PSYCHOLOGY: Faculty teaching core courses will develop a master syllabus which will include student learning outcomes to be addressed in all sections of the course. Faculty will share strategies for developing course assignments (e.g., exams, exercises, term papers) to assess student attainment of the learning outcomes.

ACCOUNTING: Faculty members teaching the undergraduate accounting core courses: ACG 3131 (Financial Reporting), ACG 3341 (Managerial Decision Making and Accounting)., and TAX 4001 (Federal Taxation I) are required to teach a set number of chapters from a common text selected by departmental decision. Faculty members teaching a particular course will develop and periodically review a common list of educational objectives and the topics and critical thinking skills appropriate to these objectives to teach in each course.

33

GRADING RUBRICS

ENGLISH: Essays in all 3000- and 4000-level English courses will be assessed holistically according to a rubric developed and shared by faculty and communicated to students.

SUPERIOR Addresses the question or prompt fully

Demonstrates substantial comprehension of relevant material Shows substantial depth, complexity, and creativity of thought

Demonstrates clear and coherent organization Develops arguments fully with ample supporting detail

Demonstrates superior control of diction, syntactic variety, and transitions

STRONGAddresses the question or prompt substantively, yet not fully

Demonstrates comprehension of relevant material Shows depth, complexity, and creativity of thought

Demonstrates clear organization Develops arguments with supporting detail

Demonstrates control of diction, syntactic variety, and transition

Tom Pusateri

During the April 2005 workshop, the BoT suggested that all departments develop grading rubrics for components of Academic Learning Compacts that involve subjective scoring.

34

ENGLISH RUBRIC (continued)

COMPETENT Adequately addresses the question or prompt

Demonstrates adequate understanding of relevant material Shows clarity of thought but may treat the topic simplistically or

repetitively Demonstrates adequate organization

Develops arguments adequately, with some detail Demonstrates adequate facility with syntax, mechanics, and usage

but contains some errors

INADEQUATE May distort or neglect parts of the question or prompt, and/or

Fails to comprehend relevant material Lacks clarity of thought; demonstrates confused/simplistic thinking,

and/orLacks adequate organization, and/or

Fails to provide adequate or appropriate details to support generalizations, or may provide details without generalization, and/or

Demonstrates significant errors in language, syntax, or mechanics

35

HONORS COLLEGE RUBRIC:

Honors with distinction: The thesis demonstrates a profound knowledge of the fundamental concepts in the subject area. The thesis shows mastery of grammar, syntax, structure and style: it is lucid, well-organized, and stylistically elegant. The thesis engages in sophisticated critical thinking and rigorously considers alternate resolutions of a problem in an exemplary fashion or otherwise shows sophisticated critical thinking in an exemplary manner appropriate to the area of study.

Honors: The thesis shows a thorough knowledge of the fundamental concepts in the subject area. The thesis shows effective command of grammar, syntax, structure and style: it is clearly written and sensibly organized. The thesis engages in sophisticated critical thinking and explores alternate approaches or resolutions of a problem effectively or otherwise shows sophisticated critical thinking in a manner appropriate to the area of study.

Unacceptable: The thesis does not show competent knowledge of the fundamental concepts in the subject area. The thesis does not show mastery of grammar, syntax, structure, or style. The thesis does not competently engage in critical thinking or explore alternate approaches or resolutions of a problem.

36

CUT-OFF SCORE ON STANDARDIZED EXAM

NURSING: Approximately three months prior to graduation, all seniors will take the Health Education Systems, Inc. (HESI) examination. This test is a valid predictor of performance on the National Licensing Examination (NCLEX). Students must achieve a score of 850 or better on the HESI to graduate. Students who do not receive a passing score are provided with advising and a remediation plan and the opportunity to re-take the exam; they will not graduate until their score is 850 or better.

COMPARISON TO NORMS ON STANDARDIZED TESTS

37

MULTIPLE RATERS

MUSICCONTENT KNOWLEDGE (Technical skills) andCOMMUNICATION andCRITICAL THINKING (Creative skills): Graduates will have ability to perform a cross section of the music from the complete repertoire of the performance medium and exhibit the appropriate technical skills for artistic self-expression.

Students in the Bachelors Music program will be evaluated by the appropriate faculty at jury performances each semester. They are also required to successfully complete a senior recital, which must include representative examples from the appropriate repertoire for the performance medium. A pre-hearing evaluated by three faculty members must be approved before the recital is scheduled.

38

SAMPLING

LANGUAGES & LINGUISTICS: Will randomly sample the performance of students writing term papers for program-specific 3000- and 4000-level courses to compare demonstrated research skills for these with the research-skill performance of students taking and passing FOL 3880.

Will randomly sample the accuracy of linguistic structures and attention to linguistic nuance in the essays of students enrolled in selected 3000- and 4000-level courses to compare rankings of student performance in linguistics-specific courses with rankings of students’ linguistic awareness as evidenced in other courses.

JEWISH STUDIES: Periodically selected exams and papers will be read by members of the Jewish Studies Executive Committee to ensure that the criteria of the Academic Learning Compacts are being met by individual students.

1 Academic Learning Compacts Realigning FAU’s Assessment System to meet new SUS Guidelines.

Documents

Transcript of 1 Academic Learning Compacts Realigning FAU’s Assessment System to meet new SUS Guidelines.