The Effective Management of Student Teams Using the CATME/Team-Maker System: Practice Informed by Research

Capstone Conference 2010 Workshop

1

Everyone please make your wireless connections.

The development team is multi-institutional and multidisciplinary.

Richard Layton, Rose-Hulman

Misty Loughry, Georgia Southern

Matt Ohland, Purdue University

Hal Pomeranz, Deer Run Associates

Lisa Bullard, NC State

Cindy Finelli, U Michigan

Rich Felder, NC State

Doug Schmucker, Trine University

By the end of the workshop, participants should be able to…

List key findings from team-formation and peer-evaluation literature including recommendations for distributing skills and attributes formative assessment and interventions summative assessment

Use Team-Maker to create, administer, and evaluate a team-formation survey

Use CATME to create, administer, and evaluate a peer evaluation survey

We have a 3-hour agenda with breaks about every 40 minutes

Min. Activity

10 : Introductions, workshop goals, wireless connections

15 : Activity: Problems encountered in assigning teams

15: Activity: Connecting your practices to scholarship

5 : Hands-on: Setting up Team-Maker

10 : —Break—

10 : Hands-on: Practice using Team-Maker

15: Demonstration: Evaluating results using Team-Maker

20 : — Break —

10 : Activity: Problems encountered in peer evaluations

15: Activity: Connecting your practices to scholarship

10 : Hands-on: Practice using CATME

10 : — Break —

10 : Hands-on: Practice using CATME

15 : Demonstration: Evaluating results using CATME

10 : Discussion, feedback, assessment

What problems have you encountered in assigning teams?

Individually Think about team-forming views / practices Write an outline of your practices and what

problems you’ve encountered

In pairs Discuss, reach consensus on two or three most

important items

As a group Relate to Team-Maker features / gaps

Instructor selection is better than self-selection and random assignment

What does your experience tell you?How do you address student concerns

with instructor selection?

There are negative consequences of ignoring race ,gender, and ability in forming teams

What does your experience tell you?How do you address student concerns

when using these criteria?Lack of diversity can make it impossible

to consider race and genderSchedule compatibility may surpass all

of these

Set up a practice account to use Team-Maker

Connect to test server at “catme.org” A copy of the real-world system Test data won’t appear in the real system Real account set up separately

Go to “request faculty account” we will assign the password “catme” for all

Download sample CSV file from catme.org/sample.csv to your machine (desktop or "My Documents")

Break

Check the timePlease be back in 10 minutes

Team-Maker Practice

Login to your faculty account Create and issue a survey (fake student CSV) The survey as the student sees it

Login as one of the students in the survey you created

For your survey, responses of fictitious students are auto-generated

View survey results Form teams

Team-Maker helps you make decisions about how your teams are formed

Review team make-upLook for extreme casesReality checkReconsider your priorities

Adjust weightsRe-form teamsRepeat as necessary

Break

Check the timePlease be back in 20 minutes

What problems have you encountered by using or not using peer evaluations?

IndividuallyThink about peer evaluation views/practicesWrite an outline of your practices and what

problems you’ve encounteredIn pairs

Discuss, reach consensus on two or three most important items

As a groupRelate to CATME features / gaps

Students need guidance and structure

on task and process

What does your experience tell you?How do you address student concerns

when using these criteria?Training on providing feedbackMultiple ratingsFirst rating low stakes

Students must be accountable but safe in rating their teammates

What does your experience tell you?If ratings have no consequences,

raters will try to please the audienceConfidential vs. anonymous Faculty judgment is critical

CATME Practice

Use same faculty account Create and issue a survey (to previous teams) Teams can be loaded w/o using Team-Maker The survey as the student sees it

Login as one of the students in the survey you created

For your survey, responses of dummy students are auto-generated

Break

Check the timePlease be back in 10 minutes

CATME helps you discover what might be happening in your teams

View feedback to students and faculty Raw scores / comparisons Exceptional conditions

Manipulator (red) Overconfident (yellow) Underconfident (yellow) High performer (green) Low performer (yellow) Personality conflict (red) Clique (red)

Discussion

IndividuallyThink about advantages, disadvantages,

potential research usesIn pairs

Compare, prepare to share with groupAs a group

Discuss

Workshop Assessment

What features do you like about the system?

What missing features would make the system more useful to you or others?

Do you think you will use the system? Why or why not?

Any suggestions for improving the workshop?

Instrument / NSF Support

Web-based instrument / tools found at http://www.catme.org/

This material is based upon work supported by NSF Award 0243254 “Design of a Peer Evaluation Instrument that is Simple, Reliable, and Valid” and 0817403, “SMARTER Teamwork: System for management, assessment, research, training, education, and remediation for teamwork”