Assessment

Robert Gibson, Ed.D.eLearning InstituteOctober 9, 2008Emporia State University

Effective OnlineAssessment Strategies

Is Cheating a Problem on Your

Campus?

Research suggests it may bea very big problem…

perhaps bigger than you realize

2 Effective Online Assessment Strategies

How to Cheat Online

3

Why do students cheat?Everybody does it“If I don’t cheat, my grade suffers”“Unrealistic” demands by facultyOpportunity presents itself

Effective Online Assessment Strategies

How to Cheat Online

4

In a recent survey of high school students:80% of high-achieving students admitted to cheating at

least once50% said they did not believe cheating was necessarily

wrong95% of the cheaters said they’ve never been caught

According to Duke’s Center for Academic Integrity, 75% of college students confess to cheating at least once

Technology makes it easy. Students use the Internet, pagers, and cell phones to cheathttp://chronicle.com/free/2008/09/4705n.htmhttp://tinyurl.com/4jler4Using “Slugs”


How to Cheat Online

5

In a study of 500 middle and high school students, Rutgers management Professor Don McCabe found that more than two thirds said working with classmates didn't constitute cheating, and half thought it was fine for parents to do their homework

Males report significantly higher incidences of cheating than do females

No difference between freshmen and sophomoresNo difference between full- and part-time studentsResidential college students perceive cheating to

be more socially acceptable than do off-campus students


How to Cheat Online

6

Business majors tend to score lower on measures of moral development and reasoning than some other majors (particularly when compared to liberal arts students)

Cheating behaviors are a function of situational factors that appear to be either independent of or are at least poorly correlated to the level of moral development

Data suggest that instructors who repeatedly use the same examination each semester encourage cheating. Thornton & Peterson (2000)


7 Blogging Made Easy with Wordpress

How to Cheat Online

…and the faculty?

How to Cheat Online

8

A 25-question survey was completed by 742 faculty:

Faculty do not perceive academic dishonesty to be a serious problem!

Faculty claimed to be familiar with current policy and procedure and were unconcerned with implementation

Most faculty did not regularly follow institutional policy; rather, handled incidents of cheating and plagiarism on an individual basis. They believed responsibility for reducing academic dishonesty lay primarily with students and individual faculty


How to Cheat Online

9

86% of faculty have suspected and 65% have been certain of academic dishonesty in their classrooms

800 faculty at 16 institutions surveyed:40% indicated they never reported cheating54% indicated they seldom reported cheating6% indicated they often reported cheatingConcerns regarding potential legal action or

lengthy student appeals dissuaded many faculty from reporting this problem


Continuum for Cheating

10

ExcusesBlaming the TechnologyDishonest CollaborationExploiting the

TechnologyHacking the Technology

Low

High



11

Excuses, excuses…Exploit unclear course expectationsExploit faculty disorganization

Prevent ExcusesInclude clear statements on syllabus

(consider syllabus quiz)Indicate or link to institution policiesEstablish or link to institution honor codeDemonstrate honesty



12

Blaming the Technology…“Virus” claim“Computer crashed” claim“Quiz did not submit” claim

Prevent ExcusesInclude clear statements on syllabusShift responsibility to students (consider E-

test helper guide)Provide links to Helpdesk, tutorials, helper

guides



13

Dishonest Collaboration…Print the testSplit the workRecycle papers

Prevent Dishonest CollaborationConsider no-print/no-copy script (Princeton)Use random question poolsUse assessment timersUse SafeAssign (plagiarism detection)Utilize proctors for high-stakes, online

assessmentshttp://www.ncta-testing.org/cctc/



14

Exploiting the Technology…False identityPrint and distribute assessmentsUtilize slugs and other proxiesPaper mills

Prevent ExploitationConsider using systems that defeat printing

and copyingConsider using secure testing software

(SecureExam)Use SafeAssign (plagiarism detection)Do not place high-stakes assessments online



15

Hacking the Technology…Packet-sniff passwordsExploit predictable passwordsAttempt to gain access to the server itself

Prevent Hackinghttp://www.birthindex.comUse strong passwordsAvoid unsecured networksDon’t rely on online grade book and backup

often


16

Designing Objective Questions: Primer


17

Objective Question MistakesMultiple-multiple choice (cognitive load is

redirected to understanding the question-choice permutations)

Example: When writing a learning objective, you should always include:An action verbA criterion measureThe tools usedA and BB and CB, C, and E.


18

Objective Question MistakesThe long correct option (cues the

correct choice)

Example: Most beginning book writers will seek out:Writing softwarePeer reviewAn agent who can represent them to a

publisherGood advice


19

Objective Question MistakesExcessively long item choices (assesses reading

comprehension as much as subject matter knowledge, and it’s difficult to remember the choices)

Example: Used hypodermic needles should be disposed by:Bagging them in a leak-proof plastic bag and calling Environmental

Health and Safety for Disposal.Dropping all the disposables into plastic containers free of punctures

or warps and then calling Environmental Health and Safety.Collecting the disposables into a large metal bin that is leak-proof

and free of punctures, and then calling Environmental Health and Safety

Carrying the sharp objects to a Satellite Accumulation Area and storing them into a leak-proof plastic tub, and then calling Environmental Health and Safety.


20

Objective Question MistakesRidiculous or obvious foils (Humorous and

an anxiety reducer. However, using them basically makes your multiple-choice test item a true-false item)

Example: Who is best known for his/her contributions to mathematics?Paris HiltonAlfred North WhiteheadMaya AngelouSponge Bob


21

Objective Question MistakesFoils without parallel construction. (Items using the

same grammatical structure. Cognitive resources directed to understanding the question and not applying knowledge)

Example: A lazy coach is a manager who:Provides positive feedback but avoids negative

feedback. Provides negative feedback but no positive feedback. When an annual review for the employee comes up,

he gives feedback, but never at any other time. A manager who is the type of person who never

provides any feedback.


22

Objective Question Mistakes“All of the above” or “none of the above”

choices (Avoid these foils unless you are attempting to remind the student that all/none of these apply. If the student knows one of the options to be true, it reduces her choices to that one or the all of the above choice.)

Example: Coaching is best done:During the annual review. During the semiannual review. When a coachable moment presents itself. All of the above.None of the above.


23

Objective Question MistakesNegative questions (Readers overlook the not

word in the question. Avoid these items when possible, especially if using a negative in the stem.) If negatives must be used, capitalize, underscore, bold, or otherwise highlight

Example: Which of the following is not recommended for the victim of an accident:Not leaving the victim unattended.Not moving the victim.Notifying the authoritiesNot talking to the victim.


24

Place as much of the question in the stem as possible, rather than duplicating material in each of the options

Avoid using ALWAYS and NEVER in the stem. Test-wise students are likely to rule out such universal statements from consideration

Research supports using from three to five foils

Avoid clues to the correct answer in the grammatical construction of the question


Objective Question Mistakes

25

Multiple-Choice-True-False Questions.Create two true and two false items, and

add reasons why the item is true or false.Example: The Colorado Supreme Court

banned the use of cell phones in elevators:True, it was deemed a safety hazard.True, it was judged a public nuisanceFalse, because it violated freedom of speech.False, there was a hung jury


Test Item Alternatives

26

Scenario-based. Smaller than a case study but longer than a standard multiple-choice stem, scenarios put the student into situations that they must solve.

Example: You have just given Ariel her performance review. You ask her to sign her evaluation (Meets Expectations), but she believes she is underrated. What should you do?Give Ariel the option of signing or being suspended for 30 days.Note and date her disagreement in writing, and have her sign it.Set several performance based 30-day goals that will upgrade

your evaluation if she achieves them.Verbally acknowledge her right to disagree, and indicate that you

are obligated to submit the evaluation anyway.



27

Testlets (Context-Dependent Item Sets) A test problem or scenario is presented and a set of related test items are asked.

Example: Mark is a blood lab technician who has worked at the University Health Sciences Center for the last five years. Mark has just found out he has HIV. Is Mark covered by ADA?a. Yes, HIV is covered by ADAb. Yes, as long as he is taking medicine for HIV.c. No, only AIDS is covered by ADA, not HIV.d. No, neither HIV nor AIDS are covered by ADA.



28

Continued…Must the university allow Mark to continue to work

with blood since he has HIV? a. No, the university is required to reassign Mark to a

position where he is not working in a medical facility. b. No, the university is required to reassign Mark to a

position where he is not working with blood. c. Yes, unless Mark's HIV posses a demonstrable safety

risk to him or others, the university must continue to allow Mark to work with blood.

d. Yes, the university cannot make HIV employees switch positions.



29

Alternate-Choice. A two-option MC question that works better with high achieving examinees (grad students and majors) who might readily eliminate several obvious foils in a four-item MC test. Best used to generate more items than you would with a standard four item MC test.

Example: Which is a proper closing line for an autumn haiku?

a. An island song Like a floating river Rain Rain Fall Fallb. An island song Like a floating river With falling rain a ‘quiver.



Alternative Forms of Assessment

30

Self AssessmentReflective Journal WritingDiscussions/Blogs/WikisPeer AssessmentElectronic Portfolios

(Assess.)Case StudiesRole

Playing/Simulation/GamesTeach-Back/Oral ProjectsGroup Projects

Native Blackboard ToolSelf-and-Peer

AssessmentBaylor Journal/Learning

ObjectsDiscussion Board/LOSelf-and-Peer

AssessmentNo native Blackboard

toolStudent Group ToolVirtual Classroom or

chatVirtual Classroom or

chatStudent Group Tool


Contact and Conclusion

32

Dr. Robert GibsonTechnology and Computing ServicesEmporia State [email protected]://rob-gibson.net


Assessment

Education

Transcript of Assessment