Assessment and Assessment and Evaluation of L2 Evaluation of L2

LearningLearning

Matthew McIntyre, Debbie Matthew McIntyre, Debbie Degraffenreid, Leon Nishball, Blake VildersDegraffenreid, Leon Nishball, Blake VildersTESOL 410TESOL 410

Assessment and Evaluation Assessment and Evaluation DefinedDefined

Assessment: Assessment: ““Procedures used to obtain Procedures used to obtain information about student performanceinformation about student performance”” (Woolfolk, (Woolfolk, 2005, p. 504).2005, p. 504).Evaluation: Evaluation: ““Decision making about student Decision making about student performance and about appropriate teaching performance and about appropriate teaching strategiesstrategies”” (Woolfolk, 2005, p. 504).(Woolfolk, 2005, p. 504).““Assessment is the process of identifying, gathering Assessment is the process of identifying, gathering and interpreting information about students' and interpreting information about students' learning. The central purpose of assessment is to learning. The central purpose of assessment is to provide information on student achievement and provide information on student achievement and progress and set the direction for ongoing teaching progress and set the direction for ongoing teaching and learning (NSWDET, 2008).and learning (NSWDET, 2008).

Assessment and Evaluation Assessment and Evaluation Defined cont.Defined cont.

““Sometimes the term assessment is used to describe the Sometimes the term assessment is used to describe the process of trying to determine what students already process of trying to determine what students already know about a topic before instruction, whereas the term know about a topic before instruction, whereas the term evaluation refers to the process of monitoring progress evaluation refers to the process of monitoring progress during and after instructionduring and after instruction”” (Nelson & Price, 2007, p. 65)(Nelson & Price, 2007, p. 65)““Assessment Assessment is defined as is defined as ‘‘the gathering and the gathering and synthesizing of information concerning studentssynthesizing of information concerning students’’ learning,learning,’’while while evaluation evaluation is defines as is defines as ‘‘making judgments about making judgments about studentsstudents’’ learning. The processes of assessment and learning. The processes of assessment and evaluation can be viewed as progressive: first evaluation can be viewed as progressive: first assessment; then evaluationassessment; then evaluation’”’” (Echevarria, et al, 2004, p. (Echevarria, et al, 2004, p. 148)148)

How Do Assessment and How Do Assessment and Evaluation Relate to Student Evaluation Relate to Student

Learning?Learning?Learning CycleLearning Cycle

Cont.Cont.

Where are my students Where are my students now? now? Assessment provides Assessment provides information about what information about what students already know, students already know, understand, or what they understand, or what they can do. can do. What do I want my What do I want my students to learn?students to learn?StandardsStandardsObjectivesObjectivesPerformance Measures Performance Measures

How will my students How will my students get there? (Teaching get there? (Teaching and learning strategies)and learning strategies)Consider:Consider:Teaching StrategiesTeaching StrategiesClassroom OrganizationClassroom OrganizationLearning EnvironmentLearning EnvironmentHow can students How can students demonstrate learning?demonstrate learning?How do I know when How do I know when my students get there? my students get there? Assessment:Assessment:Conventional Conventional Assessment (Testing) Assessment (Testing) and Alternative Methodsand Alternative Methods

Formative vs. Summative Formative vs. Summative AssessmentAssessment

Formative Assessment: Formative Assessment: Happens during the lesson.Happens during the lesson.

Allows teachers to see how successfully students Allows teachers to see how successfully students are progressing in relation to objectivesare progressing in relation to objectivesAllows students to benefit from the teacherAllows students to benefit from the teacher’’s s feedback while the lesson is still happeningfeedback while the lesson is still happeningCan tell teachers where to go next (review, spend Can tell teachers where to go next (review, spend more time, move on)more time, move on)Examples: Examples: ““Checks for UnderstandingChecks for Understanding””

““Checks for understanding (CFU) are monitoring Checks for understanding (CFU) are monitoring opportunities that, when done correctly provide teachers opportunities that, when done correctly provide teachers with excellent ways to evaluate whether students are with excellent ways to evaluate whether students are learninglearning”” (Nelson & Price, 2007).(Nelson & Price, 2007).

ObservationObservationConversations with studentsConversations with students

Summative Assessment:Summative Assessment:Can happen at the end of a lesson, unit, school year, etc.Can happen at the end of a lesson, unit, school year, etc.

Allows teachers to compare student performances to Allows teachers to compare student performances to objectives and standardsobjectives and standardsTells the teacher whether a student has mastered the Tells the teacher whether a student has mastered the objectiveobjectiveHelps shape future instructional plansHelps shape future instructional plansExamples:Examples:

Criterion Reference TestsCriterion Reference TestsWork Samples/PortfoliosWork Samples/Portfolios

Assessment can be many things for many situations.Assessment can be many things for many situations.Assessment can be informal, formal, authentic, Assessment can be informal, formal, authentic, multidimensional, alternative, dynamic, written, oral, etc.!multidimensional, alternative, dynamic, written, oral, etc.!

No matter what the type of assessment, it should No matter what the type of assessment, it should emphasize learning rather than better grades or emphasize learning rather than better grades or scores.scores.

““Assessment information is not valuable if it does not lead Assessment information is not valuable if it does not lead to a course of action for the studentto a course of action for the student’’s (and teachers (and teacher’’s) s) benefitbenefit”” (Hargett, 1998, p196).(Hargett, 1998, p196).

Reliability and ValidityReliability and Validity“…“…regardless of the means used to evaluate cognitive abilities, acregardless of the means used to evaluate cognitive abilities, academic ademic achievement, linguistic competence, or language proficiency, conachievement, linguistic competence, or language proficiency, confidence fidence

in the reality of the assessment rests with the selection of relin the reality of the assessment rests with the selection of reliable and iable and valid measures, with careful consideration of the context of thevalid measures, with careful consideration of the context of the

evaluationevaluation”” (Collier, 2007, p 169).(Collier, 2007, p 169).Reliability:Reliability:The consistency of a test includes:The consistency of a test includes:““testtest--retestretest”” reliabilityreliabilityAlternate form reliabilityAlternate form reliabilitySplit half reliability (internal consistency)Split half reliability (internal consistency)

Validity:Validity:CriterionCriterion--related validityrelated validityContent validityContent validityConstruct validityConstruct validity

““A test must be reliable in order to be validA test must be reliable in order to be valid……However, reliability However, reliability will not guarantee validitywill not guarantee validity (Woolfolk, 2005, p514).(Woolfolk, 2005, p514).

Perception of IntelligencePerception of Intelligence

What does it mean to be intelligent?What does it mean to be intelligent?What does an intelligent student act like?What does an intelligent student act like?What should an intelligent student be able to do?What should an intelligent student be able to do?Will all EnglishWill all English--language learners express their language learners express their intelligence in the same way?intelligence in the same way?Robert J SternbergRobert J Sternberg’’s work with intelligence and s work with intelligence and cultureculture““Intelligence may be conceived in different ways in Intelligence may be conceived in different ways in different culturesdifferent cultures”” (Sternberg, 2007, p 148).(Sternberg, 2007, p 148).““Whether teachers take into account the differences Whether teachers take into account the differences in conceptions of who is intelligent and who acts in conceptions of who is intelligent and who acts intelligently can also affect how well students learn. intelligently can also affect how well students learn. When students are taught in ways that take into When students are taught in ways that take into account their cultural contexts and that are culturally account their cultural contexts and that are culturally appropriate for them, they can achieve higher appropriate for them, they can achieve higher levelslevels”” (Sternberg, 2007, p 148).(Sternberg, 2007, p 148).

Assessment BiasAssessment Bias““Biases are aspects of the test such as content, language, or Biases are aspects of the test such as content, language, or

examples that might distort the performance of a groupexamples that might distort the performance of a group--either for either for better or for worse (Woolfolk, 2005, p514).better or for worse (Woolfolk, 2005, p514).””

Types of Assessment Bias:Types of Assessment Bias:Unfair Penalization: Tests may penalize Unfair Penalization: Tests may penalize groups of students for their lack of knowledge, groups of students for their lack of knowledge, or experience with situations, examples, or or experience with situations, examples, or language.language.

Offensiveness: Content, such as situations, Offensiveness: Content, such as situations, examples, or language may be offensive to examples, or language may be offensive to certain groups of peoplecertain groups of people

So What is the Point?So What is the Point?

Make sure you are testing what you want to Make sure you are testing what you want to test!test!Assessment and evaluation of proficiency in L2 Assessment and evaluation of proficiency in L2 should measure proficiency in L2, not other should measure proficiency in L2, not other factors.factors.Make sure assessment measures and Make sure assessment measures and expectations are clear.expectations are clear.““When standards and expectations are clearly When standards and expectations are clearly communicated to students, it also allows communicated to students, it also allows students to become partners in assessing their students to become partners in assessing their own progressown progress”” (Hargett, 1998, p 199).(Hargett, 1998, p 199).Make sure that both formal and informal Make sure that both formal and informal measures of assessment are free from bias.measures of assessment are free from bias.Have colleagues help.Have colleagues help.

KK--12 Classroom and Teacher12 Classroom and Teacher

Understand the Classroom Understand the Classroom Population.Population.

Cultural backgroundCultural backgroundPrevious EducationPrevious Education

Understand the TeacherUnderstand the TeacherTeaching methodTeaching methodClassroom effectiveness Classroom effectiveness

Assessment Models Assessment Models

Language SamplesLanguage SamplesDynamic AssessmentDynamic AssessmentNarrative AnalysisNarrative Analysis

Problems with Assessment Problems with Assessment ModelsModels

Standardized TestsStandardized TestsExaminer CompetencyExaminer CompetencyReliability Reliability

WASLWASL

Washington Washington Assessment of Assessment of Student Student LearningLearning((WASLWASL) )

Problems of Poor AssessmentProblems of Poor Assessment

Student Self EsteemStudent Self EsteemStudent PlacementStudent Placement

Assessment ImprovementAssessment Improvement

Teacher Teacher ResponsibilityResponsibilityStandardized Standardized Testing Testing

Assessment ConclusionAssessment Conclusion

AccommodationAccommodationEnvironment Environment

ASSESSMENT WITH ADULT ASSESSMENT WITH ADULT ENGLISH LANGUAGE LEARNERSENGLISH LANGUAGE LEARNERS

Reasons for TestingReasons for TestingThe Adult Education and Family Literacy Act (Title II The Adult Education and Family Literacy Act (Title II of the Workforce Investment Act of 1988) requires of the Workforce Investment Act of 1988) requires each state to report educational gains of learners in each state to report educational gains of learners in terms of level descriptors defined by the National terms of level descriptors defined by the National Reporting System (NRS) document.Reporting System (NRS) document.

WHY TEST IN ADULT BASIC WHY TEST IN ADULT BASIC EDUCATION (ABE) OR ESL?EDUCATION (ABE) OR ESL?

To place learners in appropriate To place learners in appropriate instructional levelsinstructional levelsTo measure their ongoing progressTo measure their ongoing progressTo qualify them to enroll in academic or To qualify them to enroll in academic or job training programsjob training programsTo verify program effectivenessTo verify program effectivenessTo demonstrate learner gains in order to To demonstrate learner gains in order to justify continued funding for a programjustify continued funding for a program

TYPES OF ASSESSMENTS TYPES OF ASSESSMENTS USEDUSED

Michigan Test of English Language Michigan Test of English Language The Adult Basic Learning Examination The Adult Basic Learning Examination (ABLE) (ABLE) BESTBESTCASAS CASAS

Alternative AssessmentAlternative Assessment

SurveysSurveysPerformance SamplesPerformance SamplesInterviewsInterviewsChecklistsChecklists

Learner Portfolios as another Learner Portfolios as another optionoption……

Learner portfolios:Learner portfolios:Collections of individual work, are common Collections of individual work, are common examples of alternative assessment. examples of alternative assessment.

Portfolios can include such items as: Portfolios can include such items as: reports on books reports on books notes from learner/teacher interviews notes from learner/teacher interviews learnerslearners’’ reflections on their progress writing samples reflections on their progress writing samples data from performancedata from performance--base assessments base assessments scores on commercially available testsscores on commercially available tests

Best PracticesBest Practices

Clearly identify the purpose of Clearly identify the purpose of the assessment (why the the assessment (why the learners are being assessed) learners are being assessed) and what learning is to be and what learning is to be assessed (e.g., increased assessed (e.g., increased speaking proficiency). speaking proficiency).

Select assessment instruments Select assessment instruments and procedures that match the and procedures that match the program's learning goals (e.g., program's learning goals (e.g., an oral interview to show an oral interview to show progress in speaking skills, progress in speaking skills, writing samples to show writing samples to show progress in writing) and that progress in writing) and that engage learners so they are engage learners so they are interested and will strive to do interested and will strive to do their best. their best.

Whenever possible, use multiple Whenever possible, use multiple measures to present a more measures to present a more complete picture of what has complete picture of what has been learned. been learned.

Ensure that adequate resources Ensure that adequate resources are available to carry out the are available to carry out the assessments (e.g., enough assessments (e.g., enough materials, comfortable materials, comfortable environment, adequately trained environment, adequately trained administrators and scorers). administrators and scorers).

Be aware of the limitations of the Be aware of the limitations of the assessments selected. assessments selected.

Remember that assessment is Remember that assessment is not an end in itself, but a means not an end in itself, but a means to an end. Share assessment to an end. Share assessment results with learners and results with learners and instructors, as well as with instructors, as well as with administrative staff. administrative staff.

Student ProfilesStudent Profiles

BibliographyBibliographyAlamprese, J.A., & Kay, A. (1993). "Literacy on the cafeteria liAlamprese, J.A., & Kay, A. (1993). "Literacy on the cafeteria line: Evaluation of Skills Enhancement ne: Evaluation of Skills Enhancement Training Program." Washington, DC: COSMOS Corporation and RuttenTraining Program." Washington, DC: COSMOS Corporation and Ruttenberg, Kilgallon & Associates. (ED berg, Kilgallon & Associates. (ED 368 933)368 933)Balliro, L. (1993). What kind of alternative? Examining alternatBalliro, L. (1993). What kind of alternative? Examining alternative assessment. "TESOL Quarterly", ive assessment. "TESOL Quarterly", "27"(3), 558"27"(3), 558--560.560.Beaumont, C, Scherba, J. and Trumbull, E. (2002). Alternative ABeaumont, C, Scherba, J. and Trumbull, E. (2002). Alternative Assessment for Transitional Readers. In ssessment for Transitional Readers. In C. Collier (ed). C. Collier (ed). Handbook for Second Language Acquisition.Handbook for Second Language Acquisition. (p205(p205--220). Ferndale, WA. Crosscultural 220). Ferndale, WA. Crosscultural Developmental Education Services.Developmental Education Services.Collier, C. (2007). The Assessment of Language Acquisition. InCollier, C. (2007). The Assessment of Language Acquisition. In C. Collier (ed). C. Collier (ed). Handbook for Second Handbook for Second Language Acquisition.Language Acquisition. (p166(p166--179). Ferndale, WA. Crosscultural Developmental Education Serv179). Ferndale, WA. Crosscultural Developmental Education Services.ices.Collier, C. (2007). The Assessment of Language Acquisition. Collier, C. (2007). The Assessment of Language Acquisition. Handbook for Second LanguageHandbook for Second Language

AcquisitionAcquisition, 166, 166--228.228.Davis, A.E. & Yap, K.O. (1992). "Results of field research ABE/EDavis, A.E. & Yap, K.O. (1992). "Results of field research ABE/ESL assessment." Portland, OR: SL assessment." Portland, OR: Northwest Regional Educational Laboratory. (ED 376 379)Northwest Regional Educational Laboratory. (ED 376 379)Echevarria, J., Short, D. J., and Vogt, M. (2004). Echevarria, J., Short, D. J., and Vogt, M. (2004). Making Content Comprehensible for English Making Content Comprehensible for English Language Learners: The SIOP Model.Language Learners: The SIOP Model. Boston: Allyn & Bacon.Boston: Allyn & Bacon.Genesee, F. (1994). President's message: Assessment alternativesGenesee, F. (1994). President's message: Assessment alternatives. "TESOL Matters", "4"(5), 2.. "TESOL Matters", "4"(5), 2.Hargett, G. R. (1998). Assessment in ESL & Bilingual Education.Hargett, G. R. (1998). Assessment in ESL & Bilingual Education. In C. Collier (ed). In C. Collier (ed). Handbook for Handbook for Second Language Acquisition.Second Language Acquisition. (p196(p196--204). Ferndale, WA. Crosscultural Developmental Education 204). Ferndale, WA. Crosscultural Developmental Education Services.Services.Holt, D. & Van Duzer, C. (2000). Holt, D. & Van Duzer, C. (2000). Assessing success in family literacy and adult ESL.Assessing success in family literacy and adult ESL. McHenry, IL & McHenry, IL & Washington, DC: Delta Systems & Center for Applied Linguistics. Washington, DC: Delta Systems & Center for Applied Linguistics. http://www.cal.org/caela/books/assess.htmhttp://www.cal.org/caela/books/assess.htmIsserlis, J. (1992). What you see: Ongoing assessment in the ESLIsserlis, J. (1992). What you see: Ongoing assessment in the ESL/literacy classroom. "Adventures in /literacy classroom. "Adventures in Assessment", "2", 41Assessment", "2", 41--48.48.

Bibliography cont.Bibliography cont.

Lytle, S.L, & Wolfe. M. (1989). "Adult literacy education: ProgrLytle, S.L, & Wolfe. M. (1989). "Adult literacy education: Program evaluation and learner assessment." am evaluation and learner assessment." Columbus, OH: ERIC Clearinghouse on Adult, Career, and VocationaColumbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education. (ED 315 665)l Education. (ED 315 665)National Center for ESL Literacy Education. (2001). National Center for ESL Literacy Education. (2001). CAELA resource collections: Assessment and CAELA resource collections: Assessment and evaluation in adult ESLevaluation in adult ESL. . Washington, DC: Author.Washington, DC: Author.Nelson, K. L., & Price, K. M. (2007). Nelson, K. L., & Price, K. M. (2007). Planning Effective InstructionPlanning Effective Instruction. Belmont, CA: Thomson Wadsworth. Belmont, CA: Thomson WadsworthNew South Wales Department of Education and Training. (2008). New South Wales Department of Education and Training. (2008). Consistent Teacher JudgmentConsistent Teacher Judgment..<http://www.curriculumsupport.education.nsw.gov.au/consistent_te<http://www.curriculumsupport.education.nsw.gov.au/consistent_teacher/index.htm> (2008, May 10).acher/index.htm> (2008, May 10).SavilleSaville--Troike, M. (1991). Teaching and Testing for Academic AchievemenTroike, M. (1991). Teaching and Testing for Academic Achievement: The Role of Language t: The Role of Language Development. In C. Collier (ed). Development. In C. Collier (ed). Handbook for Second Language Acquisition.Handbook for Second Language Acquisition. (p180(p180--188). Ferndale, 188). Ferndale, WA. Crosscultural Developmental Education Services.WA. Crosscultural Developmental Education Services.Sternberg, R. (2007) Who Are the Bright Children? The Cultural Sternberg, R. (2007) Who Are the Bright Children? The Cultural Context of Being and Acting Intelligent. Context of Being and Acting Intelligent. Educational ResearcherEducational Researcher. Vol. 36, Iss. 3, 148. Vol. 36, Iss. 3, 148--155.155.Woolfolk, A. (2005). Woolfolk, A. (2005). Educational Psychology: Ninth Edition.Educational Psychology: Ninth Edition. Boston: Allyn & Bacon.Boston: Allyn & Bacon.Wrigley, H.S. (1992). "Learner assessment in adult ESL literacy.Wrigley, H.S. (1992). "Learner assessment in adult ESL literacy." ERIC Digest. Washington, DC: " ERIC Digest. Washington, DC: National Clearinghouse for ESL Literacy Education. (ED 353 863)National Clearinghouse for ESL Literacy Education. (ED 353 863)

Picture Sources:Picture Sources:www.eastmillsboro.com/images/2005esl1st.jpgwww.eastmillsboro.com/images/2005esl1st.jpgwww.iep.utsa.edu/images/UTSA/esl_awardwinners.jpgwww.iep.utsa.edu/images/UTSA/esl_awardwinners.jpgwwwimage.cbsnews.com/.../04/22/image1534727g.jpgwwwimage.cbsnews.com/.../04/22/image1534727g.jpgwww.southalabama.edu/.../misc/classroom1.jpgwww.southalabama.edu/.../misc/classroom1.jpghttp://www.ericdigests.org/1996http://www.ericdigests.org/1996--2/tools.html2/tools.htmlwww.lakeorion.k12.mi.us/.../K12_esl.htm (picture of flags globewww.lakeorion.k12.mi.us/.../K12_esl.htm (picture of flags globe))www.edfac.unimelb.edu.au/lale/research/vels.html (alphabet pictuwww.edfac.unimelb.edu.au/lale/research/vels.html (alphabet picture)re)http://www.cal.org/caela/printer.phphttp://www.cal.org/caela/printer.phpwww.caslt.org/images/esl.jpeg (www.caslt.org/images/esl.jpeg (ESL logo)ESL logo)http://www.cal.org/caela/esl_resources/collections/factsheets.hthttp://www.cal.org/caela/esl_resources/collections/factsheets.html ml