assessment in curriculum development

Assessment in Curriculum Design

Rahma Kaffa, Younes Taia, Zakaria Jamaati

Curriculum Development CourseProf. Mohamed El Fatihi

Outline

I. Assessment )Rational of incorporating in CD, procedures(

II. Types of Assessment.a. Placement assessmentb. Observation of learningc. Achievement assessment (short-term and long-term)d. Diagnostic Assessmente. Proficiency assessment

III. Approaches to Assessment:f. Validityg. Reliabilityh. Practicality

Assessment

• is the process of collecting information by reviewing the products of student work, interviewing, observing, or testing.

Rationale for Assessment

Guide and encourage effective approaches to learning; validly and reliably measure expected learning outcomes, in particular the higher-order learning that characterizes higher education; and define and protect academic standards.

General principles underpin Assessment Procedures:

• Assessment should be an aid to successful learning and should encourage students to apply their knowledge and skills in an analytical and critical manner;

• Modes of assessment should be specifically linked to the learning outcomes of a unit and course including outcomes for the development of graduate attributes;

• Assessment requirements should be based on pre-determined and clearly articulated criteria that describe standards of knowledge, skills, competencies and/or capabilities.

General principles underpin Assessment Procedures:

• Students should receive feedback on their work in a timely manner that assists them to monitor their progress towards the achievement of specified learning outcomes and to improve the quality of their work;

• Assessment should be inclusive and equitable for all students;

• Assessment should be valid and reliable;• Students and staff should act in accordance with

the Academic Regulations, the Academic Honesty Policy and the Assessment Policy and Assessment Procedures.

How to create assessment

Guiding Questions for Creating Assessments When creating any assessment, you should use the same backwards planning mindset that frames all of your instructional planning. To help you backwards plan your assessments, consider the following questions:

How to create assessment

1. What are your learning goals or standards?

2. What evidence would you need from students to demonstrate mastery of the standards/goals?

3. What method will you use to assess? 4. What are the criteria for success?

What are the characteristics of a high quality response?

Types of Assessment:

Placement assessment

The learners are assessed at the beginning of a course to see what level of the course they should be in. The aim of this testing is to ensure that the course is not going to be too easy or too difficult for the learner.

• It often has to be done just before a course begins.

• The results have to be available quickly.

Possible placement tests

A grammar-based test could ask the learners to complete sentences, choose appropriate items from multiple-choices, locate the items referred to by reference words in a text, or complete a cloze test or dictation.

Tests which focus on vocabulary. For example, a placement test which consists of a computerized yes/no test where the learners see individual words without context from various frequency levels and have to indicate whether they know the word or not. About one-third of the words are nonsense words which are used to correct for over-estimation by the learner.

Tests which focus on language in terms of pronunciation use include interviews, role plays, listening tests with message-focused questions, reading passages with message-focused questions and composition writing.

Note: Teachers should therefore not feel that they have

to include mainly language use tests in a placement test, and should feel comfortable with making use of

tests focusing on knowledge of language items.

Possible placement tests

Observation of learning

While the course is running, the activities that the learners do are carefully monitored to see if each particular activity is likely to achieve its learning goal. This involves technique analysis and classroom observation.

• Monitoring learners’ progress in a course can occur at the level of the learning activity.

• This monitoring does not assess the learners but is directed towards the tasks that they do.

• The purpose of the monitoring is to see if it is necessary to make changes to the learning activities in order to encourage learning.

Observation of learning

Aspect that should be Considered in Observation of learning

• There are four questions that should be asked when observing learning activities (Nation, 2001: 60–74).

1. What is the learning goal of the activity?

2. What are the learning conditions that would lead to the achievement of this goal?

Aspect that should be Considered in Observation of learning

3. What are the observable signs that these learning conditions are occurring?

4. What are the design features of the activity that set up the learning conditions or that need to be changed to set up the learning conditions?

Example:

In a spoken fluency development activity. The learning goal of the activity is to

Develop Spoken fluency• The learning conditions are (assumed)

- The learners are focused on the meaning of the task.- The task involves very limited language demands, i.e. all the language needed to do the task is familiar to the learners.

Example:

The observable signs:- involvement in communicating with a partner.- reasonably high speed of speaking with a small number of hesitations.- some signs of comprehension by the listener.

The design features of the activity. include opportunity for preparation, a chance to repeat the task several times to different listeners, a familiar topic and an involved listener, and time pressure.

Example:

Achievement AssessmentShort-term Achievement Assessment.• Short-term achievement assessment monitors the

product of activities, or small sets of activities applied in the classroom.

• The goal is to check whether learners are progressing throughout the course time.

• Provides feedback for both Ts and Ss.

• They can have motivational purposes:– Oblige learners to do work they are required to do in due time.– Develop feeling of achievement for Ss as they succeed on

tests

Achievement AssessmentShort-term Achievement Assessment.• In courses where there fewer classes, this type of

assessment is not feasible.

• Short-term objectives are effectively assessed if clear ‘performance objectives’ are set ahead for some learning goals.

• A Performance Objective: (brown, 1995) It is a specific, accountable and measurable description of the desired accomplishment we want the S to attain by the end of the assessment period.– 1. Subject. 2. The performance.– 3. The conditions 4. The measure.– 5. The criterion.

Achievement AssessmentShort-term Achievement Assessment.• The subject: who should achieve the objective

of the lesson.

• The performance: what the learner should be able to perform.

• The conditions: under what conditions the performance of the learner should occur.

• The measure: how the performance will be tested.

• The criterion: what level of the performance must be reached.

Achievement AssessmentShort-term Achievement Assessment.• E.g.:

The learner should be able to quickly read and

comprehend a 550 word text containing only well-

known vocabulary and constructions at a

speed greater than 250 words per minute and with

at least 70 per cent comprehension as measured by

ten four back to the text and while they answer the

questions.

Achievement AssessmentShort-term Achievement Assessment.

• Using performance objectives is recommended to clarify goals for both T and learners, and to monitor progress.

• Recently, most curriculum designers do write POs for some of the goals of the course.

• Generally, STAA should provide clear record of the progress that can be easily interpreted:– Should be administered in a user friendly format. (motivation)– Should not be time consuming.– Should be done at regular point of the class.– Activities should be familiar to the Ss

Achievement AssessmentLong-term Achievement Assessment.• It measures both the achievement of learners during a

period of a course and the effectiveness of the course.

• The length of the course determines the number of tests learners will sit for:– A final achievement test.– Tests part of the way through a course.

• The latter may be done to pick up learners that are struggling with the course.

• They may be also used to assess the first part of material

Achievement AssessmentLong-term Achievement Assessment.

• They have three characteristics:– 1. They are based on material taught in the course.– 2. Learners do know ahead the test’s type of questions

and material will be covered.– 3. They are criterion referenced (standard) # Norm

referenced.

• Achievement tests may also be considered Mastery tests where the learners have to reach a criterion of mastery to pass to the next level. If they fail they may be repeated opportunities to study for the test and sit for it.


• Rationale behind mastery tests:– If something is to be

learned, it should be learned.

– Every learners is capable of achieving mastery, regardless of time needed.

– T responsibility is to Ss and to learning.

– The mastery of area is rewarding and motivating for both T and Ss.

• Opposing view:– T’s responsibility is to

the subject matter, not to the student.

– One role of courses is to inform employers and institution’s admission responsible that capabilities are different.

– Learners are responsible for their own learning and managing it.


• It is advisable that questions in achievement tests should be slightly different to the activities done in class.

• Language learning is supposed to be able to transfer to a variety of uses.

• General: Achievement tests are:– A very important component of curriculum design..– Provide feedback for Ts, Ss, curriculum designers.– Have a “washback” effect on teaching and learning

(motivates and guides learning)

Diagnostic Assessment

• It is implemented to find out the weaknesses and gaps in learners’ knowledge so an action can be done about them and not to waste time on teaching what already students know well.

• DA is a very good instrument of needs analysis both before a course starts and during the course.

• The results of DA determines what should a course cover. Thus a good DA is accurate and easy and easy to interpret in terms of actions to be taken.


• Some other types of assessment can provide us with diagnostic information, such as achievement tests, proficiency test, achievement tests, or placement tests.

• There some designed specifically for diagnosis:– The vocabulary level test (Schmitt et al., 2002)

The test is designed to determine what should be the focus of the course: whether high frequency vocabulary or low frequency vocabulary as teaching each category requires different teaching strategies.


• The findings of diagnostic assessment can thus have a important effect on the design of a course.

• DA can be also in different forms:– Analysis of language use ( written compositions,

reading tasks, spoken performances).– Observation of learner performance (the process of

writing, note taking, etc.)– Learner self-assessment where learners work with check

lists to indicate their perceived areas of strength and weakness.

It is very difficult to determine learners subjective concerns from objective judgments

Proficiency assessment

• A unique feature about language proficiency test is it draws its items from the language itself not from any course.

• The purpose of proficiency assessment is that to reflect to what extent a learner knows of the language or a specific part of it.

• The well know proficiency tests are:– Test of English as Foreign Language (TOEFL)– International English Language Testing System (IELTS)


• These tests sometimes awaits a learner at the end of a course.

• They may be working as a criterion-referenced test to determine the admission of a learners into an English medium university:– TOEFL iBT (internet-based) is out of 120.– IELTS test uses a 9-point scale.

• In some cases these tests may act as norm-referenced to show learners their level of proficiency in relation to others


• TOEFL as a criterion referenced test can have an important washback effect on a language course that precedes it.

• It can have the effect of making the content of the course resembles the content and test items formats in the test.

• Courses that predict what will be in the test and do copy the format of the test in their learning activities do consider proficiency test as achievement tests for the course, except that the course is drawing items from old tests.


• For these courses the presence of a proficiency test at the end of the course affect different teachers in different ways. Thus, the quality of channels communication between test designers and test designers and the Ts and learners is paramount.

• There are constraints of proficiency test at the end of the course:– 1. courses become commercial business for publishers which

their books may mislead sometimes the test takers.– 2. making a course look like a test makes dependent to the

test.– 3. Remove the test, for some international universities who

offer intensive English programs, or their own proficiency tests


• Proficiency tests are a good source for evaluation data for a program as they represent an independent measure of the relevance, and adequacy of a language course.

Approaches to assessment

• All assessment needs to be checked to see that it is doing its job properly and that is not causing unnecessary work.

• Most investigate procedures including the tools for need analysis, course evaluation procedures, and tests and other measures for assessment can be examined by considering THREE criteria: Reliability, Validity and Practicality.

• We will discuss these three criteria in relation to tests.

Reliability

Reliability is synonymous with the consistency of a test, a survey, observation or other measuring device. Let us consider this example:Imagine stepping on your bathroom scale and weighing 140 pounds only to find that your weight on the same scale change to 180 pounds an hour later and 100 pounds an hour after that. So, base on the inconsistency of this scale, any research relying on it would certainly be unreliable, and its results would not be accurate.

• So, Reliability is the degree to which an assessment tool produces stable and consistent results.

• The reliability of a test refers to its degree of consistency and repeatability.

• Statistically, reliability is measured by having the learners sit the test twice and seeing if the learners get near enough to the same result, or more commonly, by splitting the scores on the individual test items into two equal groups and seeing if the learners get the same score on both groups.

Characteristics of a reliable test

• A reliable test is always given under the same conditions, which include the amount of time allowed for the test, and the instructions are always presented in the same way.

• It is consistently marked, because if the marking is unreliable, then the results are unreliable. Reliable marking is consistent, and consistency is helped by having some kind of answer key or well-thought-out scoring procedure. Markers need to be trained.

• It has many questions or points of assessment, then if something is wrong with one or two of the points, this will not greatly influence the result.

• Its questions and instructions are clear and unambiguous.

Validity

• A valid test measures what is supposed to measure. A valid achievement test measures what has been learned on the course. A valid listening test measures skill at listening.

• The most practical ways for a teacher or curriculum designer to check the validity of a test are to look at its face validity and content validity.

• Face validity: Face validity simply means that if the test is called a reading test, does it look like a reading test? If it is called a vocabulary test, does it look like a vocabulary test?Face validity is important because it reflects how the learners and perhaps their parents, and other teachers will react to the test. For example, a vocabulary test which has a low face validity, it presents words in isolation without a context, it does not ask for the learner to give a meaning, and it does not require him or her to use the vocabulary, this could affect the learners’ reaction to the test and their acceptance of its results.It is an advantage if tests look like the kind of test they are supposed to be.

• Content validity: It is little like face validity, except that the decision-making about validity is not made by looking at the test’s “face”, but by analyzing the test and comparing it to what it is supposed to test. For example, To find the content validity of an achievement test, we would have to look at the part of the course that was being tested and list the items and skills taught. Then we would have to look at the test and list the items and skills tested, if these two lists matched each other quite closely, or if the test involved a representative sample of the course list, we could say the test has content validity.

• In other words, validity exists when a test is used for the purpose for which it was designed.

Practicality

• Not only must a test be reliable and valid, it must also be practical. Practicality is examined by looking at the cost involved in administering and scoring the test, the time taken to administer and sit the test, the time taken to mark the test, the number of people needed to administer and mark the test, and the ease in interpreting and applying the results of the test.

• Tests can be made more practical by being carefully formatted for easy marking, by being not too long, and by using objectively scored items such as true/false or multiple choice.

• The requirements of practicality, reliability and validity do not always agree with each other, for example short tests are practical but not very reliable or valid. Sometimes it is necessary to sacrifice some practicality to get a valid test (For example an individual oral interview for 200 people), or to sacrifice validity to make the test more practical (multi-choice vocabulary tests). Where possible, reliability and validity should be preferred over practicality, but usually compromise is necessary.

• Assessment is a major source of information for the evaluation of a course and thus its gradual improvement. Assessment also contributes significantly to the teacher’s and learners’ sense of achievement in a course and thus is important for motivation.

• Curriculum design should include the planning of a well-thought-out program of assessment of various kinds.

References

• Macalister, J. (2010). Language Curriculum Design. NYC: Routledge.

• Retrieved from website on 20 May, 2013:

http://www.dartmouth.edu/~hrs/profldev/performance_management/performance_objective.html



assessment in curriculum development

Education

Transcript of assessment in curriculum development