Data Collection: An Introduction

PowerPoint Slides to be used in conjunction with the Facilitator’s Guide

Recommended citation:Wakeman, S., & Henderson, K. (2012). Data collection: An introduction – A PowerPoint presentation for professional development. Modules Addressing Special Education and Teacher Education (MAST). Greenville, NC: East Carolina University.

This resource includes contributions from the module developer and MAST Module Project colleagues (in alphabetical order) Kelly Henderson (Facilitator Guide Editor), Tanner Jones (Web Designer), Diane Kester (Editor), Sue Byrd Steinweg (Project Director), Bradley Baggett (Graduate Assistant), and Sandra Hopfengardner Warren (Principal Investigator).

Copyright © 2012, East Carolina University.

Session Agenda

• Introduction• Session Goals and Objectives• Background Overview • Legislation • Data Collection Design • Response Modes

Session Agenda, continued

• Writing Measurable Skills/Tasks • Data Collection • Practice Matching Skill to Sheet • Reliable Data Collection • Summary • Evaluation

IntroductionBrian is a student in Mrs. Stephens’ 7th grade class. He works very hard and tries on every task he is given. One task in particular is troubling for Mrs. Stephens. Brian needs to work on solving for a variable in an algebraic equation.

Introduction, continued He works on this task in his inclusion class

as well as on his homework, therefore Mrs. Stephens rarely sees him solve a problem. He can add and subtract with regrouping, so Mrs. Stephens feels comfortable that he can complete that part of the task. He continues to get each question wrong, however, and she is not sure what to do about it.

Introduction, continued• What ideas do you have to help her

gather information about Brian’s performance?

• One way Mrs. Stephens could find out what is happening within Brian’s performance is to collect data during his performance using a systematic method or plan.

Introduction, continued• Mrs. Stephens could:

– identify the steps to solve for the variable and create a task analytic data collection sheet and use it to gather data as she observes Brian attempting to solve for the variable.

– Or Mrs. Stephens could analyze work samples to identify error patterns by examining papers from Brian on which he has worked out on paper each step to solve the problem.

• Either approach would allow Mrs. Stephens to collect performance data from Brian.

Introduction, continued

• Data could be collected by the teacher, paraprofessional, general education teacher, and/or parent using a specified method. Whichever method, the data collected should inform their instructional efforts for supporting Brian.

Session Goal and ObjectivesObjectives: Participants will be able to:1. Identify types of data collection systems

with a focus on the data collection sheets (e.g., task analysis, repeated trial, repeated opportunity).

2. Identify measurable skills with a condition, behavior, and criterion for mastery.

3. Identify the correct data collection sheet for a given skill.

Overview• Data collection is the objective and accurate

measurement of a student’s present level of performance of and progress, or lack of progress, on a task, activity, or behavior.

• Starting point- the baseline data of what the student is currently doing or not doing.

• Data collection ends at the point where the student reaches the criteria for mastery.

Overview, continued• Accurate data collection allows the

teacher or service provider to determine:(a) a student’s present levels of performance

(PLOP); (b) any changes needed to provide

adaptations to the materials or the interventions;

(c) student progress or lack of progress; (d) program modifications or supports which

are necessary; and (e) any patterns of behavior.

Overview, continued

• Data can be used on a:– daily basis to inform instructional and

behavioral efforts for individual students. – monthly, quarterly, or yearly basis to inform

programming efforts (e.g., the need for additional or tertiary level interventions, the need for support services, and the plan for sequential instructional efforts).

Legislation • The Individuals with Disabilities Education

Act (1997) required schools to:– monitor and provide parents with

documentation of a child’s progress towards mastery for annual Individual Education Program (IEP) goals and objectives; and

– provide documentation of special education student participation in state-wide alternate assessments.

Legislation, continued• Data collection plays an essential part in this

documentation, particularly in states that require teachers to provide evidence of student work as a part of assessment.

• Data collection has been recognized as a cornerstone for research-based practices. For example, Response to Intervention (RTI) is based upon the collection of valid student performance data.

Data Collection Design • Data collection of student progress is

designed to be:– Objective: data should be measured

objectively. The skill must be observable and measurable. It must be written in a manner that enables multiple observers to see the same outcome. The use of multiple observers is an important step to demonstrate that the data is a valid representation of the student’s performance.

Data Collection Design, continued– Systematic: the process of collecting data must

follow a systematic pattern. Random collection of different expectations or outcomes creates the chance of a variety of interpretations of the data. Defining the cues, prompts (including error corrections), materials, and data collectors is necessary. A plan should be identify when data collection should occur; data would typically be taken at least 2-3 times a week.

Data Collection Design, continued

– Defensible: a data collection system should be carefully thought out. Why should the teacher collect this data and collect it this way? Data provides teachers with the necessary information to make defensible decisions about instruction.

Response Modes • Data collected must be an accurate

representation of what the student knows and can do.

• Tasks must be designed and supports must be utilized in such a way as the student can independently respond to any items presented.

Response Modes, continued– For example, if a task was defined for a student

to identify sight words but the student used picture symbols and photographs for receptive and expressive language, the task would be too difficult for the student.

– However, if the student was asked to increase his picture vocabulary paired with the written word, the student could respond within the task as designed. How the student responds best must be considered when designing a task in which data will be collected.

Response Modes, Activity ?

– Review sample data sheets. Examine the objective for each data sheet to see (a) what responses students would have to have to be able to show their understanding of the task as well as (b) what supports are provided to them to make an accurate response possible.

Response Modes, continued• Designing the system to collect data typically

falls to the teacher or behavior specialist. These practitioners are responsible for:(a) identifying and defining skill/behavior; (b) establishing the baseline performance of the

student (what the student can do) and the criteria, including mastery (what they want the student to be able to do);

(c) creating a user-friendly data collection method;

Response Modes, continued(d) training team members to collect data across

environments; (e) reviewing and analyzing data weekly; and(f) modifying programs based on data.

• The skill or target behavior is typically an IEP objective, a skill related to prioritized content standards, a behavior that impedes the student’s learning, or a skill to be assessed for a formal assessment, such as the alternate assessment.

Writing Measurable Skills/Tasks

• There are three components to each skill or task: condition, behavior, and mastery level.

• The following narrated slides are available at http://mast.ecu.edu/modules/dc_intro/lib/media/slides01/SlideShow.html.

Writing Measurable Skills/Tasks

Data Collection: An Introduction

Condition

• What will be presented to the student?

• What materials will be used (be generic)?

• Given manipulatives, a worksheet with 10 problems, and scaffolded support, if necessary, and a pencil, Josh will independently solve 9 out of the 10 problems correctly for 3 consecutive trials.

Behavior• What will the

student be doing?

• Given manipulatives, a worksheet with 10 problems and scaffolded support, if necessary, and a pencil, Josh will independently solve 9 out of the 10 problems correctly for 3 consecutive trials.

Criteria for Mastery• What level of

increase or accuracy are you expecting?

• How many days will the student have to reach that criteria to be considered as having mastered the task?

• Given manipulatives, a worksheet with 10 problems and scaffolded support, if necessary, and a pencil, Josh will independently solve 9 out of the 10 problems correctly for 3 consecutive trials.

What is missing?

• Jane will independently read 4 out of 5 sight words correctly for 3 consecutive trials.

• Given a writing instrument or stamper, Michael will independently mark on paper as an emerging signature.

• Given her phone number and a distractor, Gerry will independently point to her own phone number 4 of 5 trials correctly for 3 out of 5 days.

Non Examples

• What is wrong with the way the following tasks are written?– Sherrie will participate in an inquiry

based science lesson with physical assistance for 4/5 trials.

– Given a book of her choice, Lynn will read a book independently.

– Given a crosswalk sign, Joel will cross the street safely on 8 of 10 occasions.

Activity suggestion 

- What unusual experiences have you had with writing measurable skills and objectives and the results?

Data Collection

• Creating a user-friendly data collection method starts with the skill or task and deciding how to document progress. Options include data sheets, permanent products (e.g., work samples), anecdotal notes, or a combination.

• Data collection sheets should be determined prior to instruction or assessment. Selection is guided by the purpose the data and the way the task is written.

Data Collection, continued

• Some examples of different types of data collection sheets follow.

a) Task Analysis- Outlines the steps necessary to complete a task. When the student is presented the task to complete, the number of steps correct is scored. The teacher decides on the number of steps presented in each trial (total task versus forward or backward chaining).

Data Collection, continued

• For example, a task analysis data sheet would likely be used to record the steps for a student to use money in a vending machine or the steps in a science experiment.

• An example of a task analytic data sheet follows and is available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Task_Analysis_Data_Sheet.pdf .

Data Collection, continued

b) Repeated Trial- One of the most common data sheets. Teacher delivers teaching trials in a set (e.g., present a sight word, then another word, then another word, etc.) Responses given by student are charted on the data sheet (either correct/incorrect or independent/prompted depending upon if the task is instructional or assessment

Data Collection, continued

• An example of a repeated trial data sheet is provided here. A copy of the repeated Trial Data Sheet follows and is available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Repeated_trial_data_sheet.pdf.

Data Collection, continued

• Note on the data sheet that items presented to the student are listed on the left side of the data sheet and each unique item has its own row for data collection.

• It’s easy to see if a pattern emerges for specific items (e.g., the student can identify snowy consistently but not cloudy).

Data Collection, continued• This is important as specific items may

be too difficult for the student and prevent mastery of the overall task

• For example- the task is 8 out of 10 sight words and the student consistently gets 7 correct, the teacher can decide what instructionally to do next to support student acquisition of the 3 words that are preventing mastery of the task.

Data Collection, continued

c) Repeated Opportunity- appropriate for skills that are taught throughout day, i.e., the trials are spread out. The student’s responses are charted as they are made. Examples of skills that could be recorded on a repeated opportunity data sheet are using an object schedule or telling clock time at start of each lesson.

Data Collection, continued

• An example of a Repeated Opportunity Data Sheet with data in it follows and is available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Repeated_Opportunity_data_sheet.pdf.

Data Collection, continued• On the previous data sheet, the who,

where, and potentially with what the student makes the response for the task or item is recorded for each opportunity.

• In this case, the student is learning to independently transition when given a prompt during different parts of his schedule.

Data Collection, continued

• Some students may respond differently to different people or in different locations depending upon what distractors may be present.

• This type of repeated opportunity data allows the teacher to see what patterns, if any, exist that prevent or promote the students response.

Data Collection, continuedd) Frequency- typically used to measure

the degree to which there is an increase or decrease in number of times the student uses a new response or refrains from an unwanted response. This skill or task may be measured throughout day (e.g., hand raising instead of calling out) or in one lesson (e.g., activating a communication device to respond).

Data Collection, continued

• An example of a frequency data sheet follows and is available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Frequency_data_sheet.pdf.

Data Collection, continued• The previous sheets is an example of how

frequency data can be taken. The student is sorting a series of pictures into two categories. The frequency of how many pictures were sorted correctly is recorded.

• Another example of how frequency data could be collected is to simply record a number for each date the items or prompt is given.

Data Collection, continued• For example, students could record how

many sit ups they do each day, the number of correct math facts in a minute, or the number of out of seat occurrences in a class period. A simple table can include date and number of responses.

Date Response

3/16/10 14

3/17/10 16

3/18/10 18

Data Collection, continued

e) Duration- used to record a skill that is measured in time, specifically, the total amount of time the student engages in task. The purpose of the instruction may be to increase the amount of time (e.g., attending to task) or decrease the amount of time (e.g., length of tantruming behavior). Time could be recorded in seconds or minutes.

Data Collection, continued• A duration data sheet may be appropriate

when the student is expected to work for 30 consecutive minutes on a vocational task or to indicate the length of time for a student to transition between tasks.

• An example of a duration data sheet with data follows and is available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Duration_data_sheet.pdf.

Data Collection, continued• This duration data sheet has four columns,

but a sheet could have an additional column in which the prompting level (i.e., physical, model, or independent) is listed on its own.

• In the example, mastery was described in the objective as an independent performance of the task under 30 seconds of the given prompt for 3 consecutive days.

Data Collection, continued

• It is possible that the time may also be increased (e.g., actively engaged in completing a given task by 10 seconds each day until student completes a given task without prompting up to 5 minutes).

Data Collection, continued

f) Cumulative- appropriate when one discrete response is to be used throughout the day or over a period of days. Responses are totaled over days to reach a criterion. An example of a skill for which a teacher may use a cumulative data sheet is when a student is expected to independently ask for help when appropriate.

Data Collection, continued

• An example of a cumulative data sheet with data follows and is available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Cumulative_data_sheet.pdf.

Data Collection, continued• Notice the objective on the previous data

sheet. The expectation for mastery is such that the student will acquire a set of information over a longer period of time rather than within a given response set (like the repeated trial data sheet).

• The student is to independently recognize 10 new science terms within context over the school year.

Data Collection, continued• Student responses are considered in the

context of natural cues or when the chance is given to elicit the response that may not occur on a daily or even weekly basis.

Data Collection, continued• One important consideration when

designing a data sheet is to first determine the purpose for collecting the data. – Is the data collected being used for

assessment purposes or is it being used within instructional trials?

Data Collection, continued• The video included later is an

assessment. When presenting assessment trials, the teacher will typically not provide correct responses unless the student replies, “I don’t know” as instructed prior to the beginning of the assessment.

Data Collection, continued• If it was an instructional video rather than

an assessment video, the teacher would correct errors and support student acquisition of the words within each trial, likely using a prompting system. Note the difference when you watch the video.

• The table that follows helps illustrate the type of scoring that would occur depending upon the purpose or use of the data.

Data Collection, continued• For an assessment probe (a set of items

presented to test what the student knows and can do independently) the first row shows what would be recorded on the data sheet- accuracy of responses.

• The repeated trial data sheet in this module is an example of an assessment probe.

Data Collection, continued

• For an instructional probe, typically the data sheet will list a prompting system in the key. The duration data sheet in this module is an example of an instructional probe.

Data Collection, continued

Response is correct/incorrect (ASSESSMENT OR NON INSTRUCTIONAL)

+ = Correct– = Not correctx = Does not attempt

Type of prompt or highest level of prompt used

I = IndependentG = Gestural NV = Non-Specific Verbal (e.g.,

“What’s next?”) V = Verbal M = ModelP = Partial Physical F = Full Physical

Practice Matching Skill to Sheet

• Watch the slide show at http://mast.ecu.edu/modules/dc_intro/lib/media/slides02/SlideShow.html to practice matching skills to the correct data sheet. Copies of the slides follow.

Matching skills to data sheets

• Data collection: An Introduction

Which is this?

Student:

Task: Given a computer and website, Student will complete all steps to locate chosen website independently at 100% accuracy for 3 websites.

Steps: Dates: 9/02 9/07 9/09 9/12 9/16 9/18 9/20

1. Find and double-click Internet Explorer icon I I I I I I I 2. Wait for Home page to appear V V V V I I V 3. Find and click “Search” icon in toolbar M M V V M M V 4. Type in search topic of choice V V V V V V V 5. Find and click “Search” button M I I M M M I 6. Click desired web page I V M M V V I Total Independent Correct: 2 2 2 1 2 2 3 Where: CL H MC CC CL CC H With Whom: T PA T CTS A CTS PA

And this?

Student:

Task: When given a book on tape and switch, student will activate the switch to start the book in less than 30 seconds for 3 consecutive trials.

Date Student Response/Time

on Task Where With Whom

10/01 M/65 secs. CL T 10/02 M/62 secs. LG A 10/06 M/60 secs. CL T 10/09 P/59secs. LG P 10/10 P/60 secs. CL A

And this? Student:

Task: Given sight words written on index cards, Student will identify 5 out of 7 weather related words for 3/5 trials.

Date: 3/03 3/04 3/07 3/10 3/12 3/13 3/18 3/20 3/21 3/25 3/26 1 Snowy - + + + + + + - - + + 2 Cloudy + + - + - - + - + - + 3 Sunny + + + + + + + + + + + 4 Rainy + - + + + + + + + + + 5 Windy + - + - - - - + - + - 6 Hot + + + + + + + + + + + 7 Cold + + + + + + + + + + + Where: H MC CL CL H CL MC H CL MC H With: PA A T P PA T A PA P A PA Material Used: HWK N W W HWK W N HWK W N HWK Total Independent Correct:

6 5 6 6 5 5 6 5 5 6 6

How about this one? Student:

Task: Given her AAC device and a story, Student will activate a voice output device to indicate a repeated story line 4/5 times requested for 3 stories.

Date: 3/03 3/04 3/07 3/10 3/12 3/13 3/18 3/20 3/21 3/25 3/26 1 - + + + + + + - - + + 2 + + - + - - + - + - + 3 + + + + + + + + + + + 4 + - + + + + + + + + + 5 + - + - - - - + - + + Where: H MC CL CL H CL MC H CL MC H With: PA A T P PA T A PA P A PA Material Used: HWK N W W HWK W N HWK W N HWK

Total Independent Correct:

4 3 4 4 3 3 4 3 3 4 5

What about this one?DATE Given a reminder card on his

desk, Marcus will get out of his seat less than three times a day without asking for 5 consecutive days.

4/11 llll=4

4/12 lllllllll=9

4/15 lll=3

4/16 lll=3

4/17 lll=3

How about which sheet would you use for these skills?

• Given a jig and materials, Matt will set the table correctly (follow 5 steps) each day for 3/5 days.

• Given 15 minutes of leisure time, Jerry will stay in the leisure area leaving no more than once without asking for 5 trials.

• Given familiar people, Kitty will initiate a greeting as she approaches for 3 or more people each day.

Reliable Data Collection• One important aspect of data collection is

ensuring that the data collected is objective:– Some measures in place to avoid biased or

incorrect data being included in the data set.– Biased data is not objective. – Data collectors who have preconceived

ideas about how the data should be may not be able to see what is actually happening, particularly in behavioral issues. here.

Reliable Data Collection, continued– Incorrect data can happen when the person

collecting data does not recognize the prompt or does not understand what a correct response looks like for the student.

– Data should be apparent and available to stakeholders.

– The data should be defensible in its accuracy and should be a valid representation of what the student is actually doing as it plays a key role in making decisions about the student’s education.

•  

Reliable Data Collection, continued

• Watch the video at http://mast.ecu.edu/modules/dc_intro/lib/media/MOV051.html. The video illustrates a child reading sight words: Galaxy, Saturn, Jupiter, Earth, Venus, Mars, Mercury, Uranus, Pluto, Neptune.

• Use a repeated trial data sheet and take data with the assessor.

Reliable Data Collection, continued

• After completing your own data collection, view the answers in the data sheet available at http://mast.ecu.edu/modules/dc_intro/lib/documents/Data_Sheet_answers_solar_system_sight_words_data_collection.pdf once you have watched the video. A copy of the data sheet follows.

Reliable Data Collection, continued• Check your answers with the answer sheet

provided. A few of the incorrect responses are worth discussion: – Emma mispronounces Uranus in trial two so it

is first checked by the teacher and then recorded as an incorrect response.

– Emma also says in trial 2 that she does not know a word (Mercury), so that word is also marked as incorrect.

Reliable Data Collection, continued

– When reviewing the mastery level set in the objective for the task, Emma does not master this task as she was required to get 9 out of 10 correct for three consecutive trials and Emma only has one trial at the mastery level.

Summary• Data collection is one of the most important

aspects of instruction and assessment. • If a plan to collect observable, measurable,

valid data is in place, practitioners can identify student response patterns and support student progress towards mastery.

• Skills or tasks must be written with a condition, behavior, and criteria for mastery.

Summary, continued

• Data collection systems can include student work samples, anecdotal records, data sheets, or a combination of all three.

• Data can be collected using a data sheet that best matches the purpose or use of the data and the type of skill to be measured.

Summary, continued

• Examples of data sheets include task analytic, repeated trial, repeated opportunity, frequency, duration, and cumulative.

• Finally, reliable data collection is a must before analysis of the data can occur.

Focus and Reflection Questions

• What do you think is the most difficult part of creating and using data collection strategies?

• Why? • What can you suggest to make the

task easier?

Application and Extension Activities

1. Data can be collected using a data sheet that best matches the purpose or use of the data and the type of skill to be measured. Examples of data sheets include task analytic, repeated trial, repeated opportunity, frequency, duration, and cumulative.

Application and Extension Activities, continued

Design their own data sheets for specific purposes and types of skills for other tasks and/or objectives, keeping in mind your classroom and students. •Task analytic•Repeated trial•Repeated opportunity

• Frequency• Duration• Cumulative

Self-Assessment

• A self-assessment with response feedback is available at http://mast.ecu.edu/modules/dc_intro/quiz/ . Participants may take this assessment online to evaluate their learning about content presented in this module.

Session Evaluation

• A form for participants to evaluate the session is available in the Facilitator’s Guide.