SPNHA Learning Assessment 2008. Outcomes/Objectives MPA Good Communication Skills Good Critical...

SPNHA Learning Assessment 2008

Outcomes/Objectives

MPA

• Good Communication Skills

• Good Critical Thinking Skills

• Post Graduation Employment

• Successful & Valuable Internship Experience

PNA major

• Post graduation employment / graduate school

• Meaningful Internship • Communication Skills • Critical Thinking

Measures

MPA

• Writing Skills Analysis• Critical Thinking

Analysis• Internship Analysis • Job Attainment Rate • Internship Approval

Rate

PNA major

• Writing Skills Analysis• Critical Thinking

Analysis• Internship Analysis• Job Attainment Rate

Writing Skills Analysis Actions

MPA

• Writing Skills Analysis Rubric

• Applied to papers from PA619

PNA major

• Writing Skills Analysis Rubric


Criteria 3 = Good 2 = Satisfactory 1 = Needs Improvement 0 = Absent

Content

Thoughtful and insightful analysis of a clearly present thesis or purpose. Reader gains good insight

Basic analysis of a thesis or purpose. Reader gains sufficient insight.

Some analysis of a thesis or purpose. Reader gains few insights.

Paper does not identify thesis or purpose. Analysis vague or missing. Reader is confused or misinformed.

Organization

Ideas arranged logically. Flow smoothly and are topics links. Reader can follow reasoning.

Ideas are, for the most part, arranged logically and linked. Reader can follow most of the reasoning.

Writing is not logical and ideas sometime fail to make sense. Reader needs to work to figure out meaning.

Little semblance of logical organization. Reader cannot identify reasoning.

ToneTone is consistently professional and appropriate of audience.

Tone is generally professional and mostly appropriate for audience.

Tone is occasionally professional or occasional appropriate for audience.

Tone is not professional. It is inappropriate for audience and purpose.

MechanicsWriting is free or almost free of errors.

Occasional errors in writing, but they do not represent a major distraction.

Writing has numerous errors and distracts the reader.

Errors are so numerous that they obscure meaning.

ReferencesSources of presented evidence are clearly and fairly represented

Complete references are generally present.

Occasional and/or incomplete references are provided.

References are not or mostly not presented.

FormatA recognized format is correctly followed

A recognized format is generally followed; a few mistakes, however.

Format of document reflects incomplete knowledge of standard.

No standardized format followed.

Communication Skills Rubric

Writing Skills Analysis Goals

MPA• Papers in each

assessment category rated satisfactory or better: 75%

• All papers rated satisfactory or better: 75%

• Average total communication skills score: 12 points

PNA major• Papers in each



• Average total communication skills score of 9

Writing Skills Analysis Findings

MPA• All of communication skill

targets are achieved with exception of “content” and “mechanics.”

• “Content” is close, but “mechanics” is substantially lower than the established target.

• Average paper’s total score 12.7 points

PNA major

• All of the communication skills targets fall short, except "organization.“

• “Mechanics," "references," and "format" are worst.

• Average paper’s total score 11 points

Writing Skills Analysis Actions

MPA• A "Writing Mechanics" module

will be developed and introduced in all sections of PA520.

PNA major• A module should be developed

for PA420 that focuses the students attention on: – making an argument – using good information– citing that information.

• More interaction is needed among the faculty to develop greater consistency in evaluating critical thinking and writing skills. The next evaluation rounds will require faculty to compare their scores on the same paper.

Writing Skills Analysis Committee Comments

MPAnone

PNA major“You did a very good job developing a rubric to standardize the evaluation process. It may be useful in the future to expand the scale to allow for more subtle differences between students, for example a 1-5 or 1-10 scale. There were several typographical and grammatical errors in this entry that may have occurred during importing of the text.”

Critical Thinking Analysis Measure

MPA

• Critical Thinking Analysis Rubric


PNA major

• Critical Thinking Analysis Rubric


Criteria 3 = Good 2 = Satisfactory 1 = Needs Improvement 0 = Absent

Quality of evidence

Completely evaluates the evidence and sources of evidence. Can distinguish between fact, inference, opinion, and value judgment.

Adequately evaluates the evidence and sources of evidence. Can usually distinguish between fact, inference, opinion, and value judgment.

Superficially evaluates evidence and sources. Often substitutes opinion and value judgment for fact and inference.

Merely repeats information provided or denies evidence with no justification. Confuses facts with inferences, opinions, and value judgments.

Supports arguments

Able to support a position with a substantial amount of information, little or no bias, and valid arguments.

Able to support a position with adequate information and few, if any, logical fallacies.

Able to support a position with some evidence.

Cannot articulate a position or offers no supporting evidence.

Used models

Accurately and completely analyzes purpose using appropriate models; finds connections between the material and the models.

Satisfactorily analyzes purpose using appropriate models; misses minor connections.

Attempts to use appropriate models, but gives attention to only the most significant or obvious connections.

Models are misapplied or not used.

Recognizes alternatives

Recognizes all plausible alternative viewpoints or solutions; completely considers each one before choosing.

Recognizes alternative solutions or viewpoints; gives them some consideration.

Recognizes a few alternative solutions or viewpoints; but dismisses them without justification.

Unable to recognize alternative solutions or viewpoints.

Assessment Rubric for Critical Thinking Skills

Critical Thinking Analysis Goals

MPA• Papers in each



• Average total communication skills score: 7 points

PNA major• Papers in each



• Average total communication skills score of 6

Critical Thinking Analysis Findings

MPA• We met our target of 70 percent

of all papers rated as satisfactory or better in all critical thinking areas.

• All of the papers assessed received a satisfactory or better rating.

• Average paper’s total score: 7.9.

PNA major• With exception to "recognizing

alternatives," our target of a 65 percent satisfactory rating was achieved or nearly achieved.

• While two categories (“quality of evidence” and “supports arguments”) are two points lower than the established target, the “use of models” is substantially lower than the target of 65 percent.

• Average paper’s total score: 6.8.

Critical Thinking Analysis Actions

MPA PNA major• A module should be developed

for PA420 that focuses the students attention on: – making an argument – using good information– citing that information.

• Faculty are not satisfied with the critical thinking rubric used. They will adopt a modified rubric at the Fall retreat

• More interaction is needed among the faculty to develop greater consistency in evaluating critical thinking and writing skills.

Critical Thinking Analysis Committee Comments

MPA

nonePNA major

• “Again, it may be useful in the future to expand the scale to allow for more subtle differences between students, for example a 1-5 or 1-10 scale. How you plan to apply the specific rubric citeria (sic) was also not clear.”

• “Several measures appeared to be based on rubric which is to be commended. The criteria for applying these rubrics and the scale chosen were somewhat confusing. Perhaps an example of how the rubric was applied to arrive at the final score for each student would help clarify the process. Measures seems to rely heavily on one course PA307 and on writing samples. Are there other measures that could provide information regarding critical thinking skills?”

Internship Analysis Measure

Internships were evaluated on:• Internship provided exposure to management decision

making• Internship provided opportunity to participate in various

parts of the organization's operation, including budgeting, core work and outcome assessment activities.

• The host organization's work matched with the intern's primary career objectives.

• Student reported a good internship experience.• Host organization reported satisfaction with the intern's

knowledge, attitude and reliability.

MPA & PNA majors

Internship Analysis Goals

• 70% of internships should be satisfactory or better in all five categories.

• 80% of host organizations should report that the students were satisfactory interns.

MPA & PNA majors

Internship Analysis Findings

MPA• 65% of internships had

satisfactory scores on all five criteria. One issue was that some host organizations, although well intentioned, were not prepared to use their interns effectively.

• Although all host organizations reported being satisfied with their interns, both students and host feedback indicated that students needed better preparation in budgeting, finance and fundraising. .

PNA major• All host organizations were

satisfied with their interns.• However, only 50 percent of

internships had satisfactory scores on all five criteria. Well short of the 70% desired. One of the most obvious problems was that students were taking convenient internships rather than strategic internships. Thus they did not match well with their career objectives.

Internship Analysis Actions

MPA• Students taking internships for

credits should have at least on year of course work(9 credits) including finance(PA615 or PA632)

• We will establish a relationship with organizations that can provide quality internship experiences on a regular basis. we will place interns there on a rolling basis. We will initiate 4 such pipeline internships per year for the next few years.

PNA major• Beginning in Fall 2008 SPNHA

and Career Services will offer each semester a workshop exploring internship and volunteer options. Governmental and Nonprofit employers and agencies will be invited to campus in the fall and spring to conduct informational and recruitment sessions. These will be organized opportunities for local governmental and nonprofits to meet with undergraduates for internship, volunteerism and entry-level career placement. And for undergraduates to get tips on seeking out, applying for and getting the most out of internship experiences.

Internship Analysis Committee Comments

MPA

none

PNA major

“The criteria applied to evaluate this measure is not clear. Will you be polling.”

Job Attainment Rate Measure

• Ever year, GVSU Career Services surveys recent graduates. Taking the most recent two years of data available, we looked at the rate at which recent MPA and PNA graduates were employed in the public or nonprofit sector, or where attending graduate school. .

MPA & PNA majors

Job Attainment Rate Goals

MPAWe expect that our recent MPA graduates will be employed in the public or nonprofit management field (or attending graduate school) at a rate equal to or higher than

– graduates from the MBA program

– equal to or higher than all graduates from GVSU`s Master`s and Doctoral programs.

PNA majorWe expect that our recent PNA graduates will be employed in the public or nonprofit management field or attending graduate school at a rate

– equal to or higher than graduates from business management

– equal to or higher than graduates from political science

– equal to or higher than all GVSU undergraduates

Job Attainment Rate Findings

MPA• 84% of recent MPA

graduates are employed in their field or in graduate school. That is on par with the MBA program. However, it is short of the 95% rate for all GVSU Masters and Doctoral graduates

PNA major• 80%of recent PNA

graduates are employed in their field or in graduate school. That is on par with Political Science (more working, less in grad school) and with all GVSU BS/BA graduates. It is higher than graduates in business management.

Job Attainment Rate Actions

MPAnone

PNA majornone

Job Attainment Rate Committee Comments

MPAnone

PNA major“This measure seems like a good one for evaluating success after graduation. Changes over time will be very important as well as consitency (sic) in polling of alumni.” “Some of your objectives are worded broadly to the point that some sound more like program goals rather than student learning objectives. For example, objective 1 “to have graduates able to obtain a job in public or nonprofit management, or enter graduate school” may be more appropriate in a self study. For assessment report purposes the objective should be more student learning focused and provide student learning outcomes that are measurable. Can you identify specific, measurable, skills and knowledge that prepare your students to be successful in locating employment or pursuing graduate school.” (from memo)

The genesis of learning assessments

In the beginning was the Final Exam.And then came the Student Learning Assessments.And the Student Learning Assessments were without form.And darkness was upon the faces of the Faculty.And they spoke among themselves, saying, "It is a crock of shit, and it stinketh."And the Faculty went unto their Unit Heads and said, "It is a pail of dung, and none

may abide the odor thereof."And the Units Heads went unto their Deans, saying, "It is a container of

excrement, and it is very strong, such that none may abide by it."And the Deans went unto the Executive Committee of the Senate (ECS), saying,

"It is a vessel of fertilizer, and none may abide its strength."And the Senators spoke amongst themselves, saying one to another, "It contains

that which aids plant growth, and it is very strong."And the ECS Chair then went to the Provost, saying unto her, "It promotes growth

and is very powerful."And the Provost went unto the President, saying unto him, “Our great and wise

Faculty has sent us word that these new Student Learning Assessments will actively promote the growth and vigor of our grand university, with powerful effects."

And the President looked upon the Student Learning Assessments, and saw that they were good.

Or “How Shit Happens”

SPNHA Learning Assessment 2008. Outcomes/Objectives MPA Good Communication Skills Good Critical...

Documents

Transcript of SPNHA Learning Assessment 2008. Outcomes/Objectives MPA Good Communication Skills Good Critical...