The first year experience of assessment – realigning the learning

Alison BoneZoe Swan

What do we expect from our first year students?

• When they arrive - what communication skills do we expect them to have?

• By the time they come to do their first piece of assessed work, what would we expect them to be reasonably confident about?

• In practice, what are we most likely to be giving them feedback on as we comment on their early pieces of assessed work?

What do first year students expect from their assessment experience?

• Do students understand how different assessments develop them throughout their first year?

• Do students see their learning in boxes enclosed by the module/subject?

• How do we encourage students to view assessment as a developmental opportunity?

Context of research

• AB was awarded a L and T Fellowship to examine the first year assessment of undergraduates in BBS

• The first year experience has been the subject of wide-spread research in the UK (Yorke and Longden 2008), the US has its own resource centre and there has been an Australian L and T Council Project resulting in an international conference (leading to Kift 2010)

• Assessment can affect retention, motivation and performance

Undergraduate programmes in BBS

• Two main streams:- Finance, Accounting and Law- Business StudiesBoth streams have named degree

pathways which share many common modules in the first year e.g. all students study an Academic Skills module, all study some aspect of basic Accounting

Methodology

• Pilot study was based on the law degree (LLB). All first year module leaders (tutors) were interviewed and student focus groups established to ascertain views on assessment to date

• This was of limited use as at the time of the meetings students had only received feedback on two modules (out of six) but generated some interesting qualitative data

Methodology

• The full study covered all BBS degree programmes (Business Studies, Business Management, International Business, Law with Business, Finance and Investment, Economics and Finance, Accounting and Finance and other ‘with’ degrees such as Business Studies with Marketing).

• Questionnaires were distributed in revision lectures at the beginning of the summer term to students and by e-mail to staff

Response rate

• Student response varied across degree programmes as many did not attend revision sessions. Follow up electronic questionnaires were virtually ignored (only 3 returned). Overall response rate = 17% (n103)

• Staff response rate was better at 53% (n11)

Assessment audit• Types of assessment varied widely but were

usually individual (although every course had at least one piece of assessed group work)

• Virtually no choice involved (Economics allowed students a limited choice of company to use as basis of report)

• Skills courses were 100% coursework, quants and accounting were usually 100% examination (but often done in smaller bites eg 2 x 50%)

• Majority were 30% coursework plus 70% examination or occasionally 50/50 (there are around 20 separate modules being taught to first year UGs)

Staff views on FY assessment

• Staff were asked why they chose the coursework assessment tasks:

- (the task) assessed the learning outcomes in a way that an examination cannot (3)

- it has always been assessed this way (4) - large numbers make this particular form

of assessment manageable (0) - it has proved popular with the students in

the past (1)

Staff views on FY assessment

• Other reasons:“It seems an applied way to make students

think about Economics”“Some tasks are there because all students

need to [acquire these skills] e.g. excel spreadsheet, teamskills day”

“The exercises encourage regular attendance and engagement with material and cumulative rather than ‘topic by topic’ learning. They also familarise students with style of exam question”

Rationale for choice

• History ‘we’ve always done it this way’ was the most frequently given reason

• It was also appreciated by some that coursework assessed outcomes that examinations could not (although one ‘coursework’ took the form of surprise tests in class!)

• Number of students did not influence the choice and only one mentioned that the assessment task had proved ‘popular’ in the past

Student views

• Students were asked which of their assessment tasks they found the most and least challenging/difficult. This was meant to exclude exams but some mentioned these anyway.

• Also asked WHY they thought this

Results

• Most challenging was deemed to be Accounting/Management Accounting (30%) followed by Economics (22%) and Academic Skills (18%)

• Reasons all very similar regardless of subject: “I didn’t understand it” “Not explained well” “Difficult to research”

Results

• The LEAST challenging assessment was Academic Skills (28%) with various reasons given including “Easy” “No knowledge needed” “Done it before” (A variety of tasks were required including a literature review and a job search portfolio)

• Others also regarded as less challenging were Quants (15%) - “well-explained”, “done before” Marketing (15%) “done before” Economics (11%) “good teaching” “done before” and Law (10%) “explained well” “given everything we needed”

Results

• Students were also asked which assignment most helped to develop their understanding of the subject and explain why

• Economics was the clear leader (35%) with reasons given including the fact that it was a group task, “good support” and “great deal of research needed”. Others mentioned Academic Skills (13%) “I learned how to reference properly” and Financial Accounting (12%) “Practical application”

Results

• Finally students were asked if one of their subjects was ONLY to be assessed by exam which would they choose - ie which assessment task contributed least to their learning.

• 21% said “None” ie they thought exams only tested memory and coursework was needed to demonstrate their understanding. Others mentioned Economics (12%), Marketing (9%) and Law (9%)

Conclusions from audit

• Although response rate from students was relatively poor the answers were diverse and showed that coursework one might find challenging another might find easy (eg literature review in Academic Skills)

• Many quote tutor support (or lack of it) as the reason for their finding work easy or challenging

• Very few (under five) admitted their own lack of effort/reading may have contributed to the difficulties they faced

Conclusions from audit

• Staff varied in the reasons given for assessing students the way they do but ‘inheriting’ a method of assessment seemed common

• Interviews with staff on the pilot (n=6) indicated general lack of enthusiasm for any particular method of assessment

• Student focus groups on the pilot (total = 15) generally found staff helpful and supportive but did not directly associate assessment with learning (more with ‘testing’)

Recommendations

• The nature of assessment FOR learning (as opposed to ‘of’ learning) needs to be made clearer to students

• Giving assessed work back through personal tutors who can keep track of feedback across subjects would enable an holistic view of student performance to be taken

• Getting students to reflect on feedback and explain what action they will take as a result of it should help close the feedback loop and promote learning

Context of the next research phase

• Idea stemmed from the research and audit carried out into the FYE of assessment in 2009/10 (Bone 2010)

• Rationale for this phase drawn from the findings and aims to; • both build on and take forward the

recommendations for year 1 LLB students

Key findings to address from previous study; Assessments

Give students the opportunity to;• Develop, practise and execute key/core skills

needed for studying law at degree level • Appropriately challenging and set with the

aim of ‘scaffolding’ the students development through year 1 (Kift 2010)

• Appropriate for diverse range of students • Feedback is given in a way & at a time that

enables maximum, developmental benefit to be gained

Key recommendations to address

• Students understand the purpose and value of assessment in year 1

• Students appreciate the benefit of developing skills that are core to becoming successful students

• Opportunities are provided that enable students to reflect on their performance in order to develop (Bone 2010)

Overview of research strategy

• To realign the assessment methods for three first year subjects on the LLB Law with Business degree: Legal Academic Skills (LAS), Legal Institutions and Method (LIM) and Law of Tort Support performance in Public Law, (the module

assessed last in Year 1 summative)• Students given series of tasks to enable the development of;

• Research, presentation/referencing skills, law related problem-solving skills in class & for assessment purposes

• Using the topic of negligence as a base where possible• Strategically aligned - assessment & feedback

Overview of research strategy

Formative feedback from 1st task set in Week 6 (WK 6) for LAS will;

Feed into the 1st summative assessment in WK 9 for LAS and LIM

See hand out Realigning the assessment Year 1* Feedback from all assessments monitored Using scheduled workshop hand back sessions One-to-one feedback sessions when identified

as appropriate All in preparation for 3rd/4th assessment tasks

Methodology Phase 1

Formative feedback• Task set and agreed between LAS and

Tort module tutor prior to start of academic year

• Area of assessment, negligence (nervous shock/duty of care) problem question

• Same topic area & method of assessment as for Tort summative hand- in – WK 15

Methodology Phase 1

Rationale for formative feedback threefold;

1. Opportunity to prepare and practise a problem question at an early stage in Y1

2. Focus on an area of law and method of assessment to be used at a later point in the assessment regime

3. To support and build on the material already covered in a law of tort lecture.

Methodology Phase 1

Formative Feedback Exercise• Students submit a researched/referenced

answer to a tort problem scenario • Advising parties, word range 500 – 800  

• Prior to completion of the task, students taken through the requirements in a short tutor-led session WK 5

• Guidance also provided on both the assessment and marking criteria used for all assessed work.

• Hand-in WK 6 optional though message conveyed mandatory for development purposes

Methodology Phase 1

Marking & hand back formative feedback WK 7• Pedantic written feedback sheet, tick box sheet for

development areas, critiqued work sentence by sentence, use of CLEO – application of law (Strong 2006) & sources of law

• Comprehensive hand back workshop three aims:• To allow a tutor-led feedback session to consolidate

learning • One-to-one individual feedback to each student that

attended the session (particular attention given to those students that scored below 50%.)

• Feedback received prior to next hand in

Methodology Phase 2

Summative assessment• Hand in WK 9 LAS and LIM • Assessment 2,000 word essay (statutory

interpretation) & 1,000 word case note on a leading Tort case (Alcock)

• The essay used as summative assessment for both LAS & LIM

• Students required to submit two separate copies, for each module tutor

Methodology Phase 2

LAS & LIM assessment marking ;• Students given marking criteria in advance• Received 2 grades for essay (LAS, LIM)• Different assessment criteria used for each

module;• LAS - produce appropriately written, presented,

researched and referenced work, sources• LIM - content of the essay, application and use

of law and commentary to provide an effective answer to the title set.

Methodology Phase 2

Hand back of summative assessmentAll work handed back in WK 12 in order to feed forward in to next summative

assessments: • WK 14 - Group Legal Presentation in LAS

set on the case study scenario used for formative feedback in WK 6(Alcock)

• WK 15 - Tort assessment – tort negligence, nervous shock problem

• WK 16 - Public Law essay hand in

Methodology hand back

Hand-back workshop built into LAS timetable

• Detailed written feedback (front) and• Line-by-line corrective feedback on

referencing and use of sources of law. • Crib sheet attached to marked work

highlighting specific areas where the student needed to improve/develop/take note

• Students asked to identify and reflect on key areas for improvement and Q & A session

• One to one discussions

Methodology hand back workshop

• Students complete questionnaire in pairs • aimed at consolidating referencing and sources of law

skills. • The questionnaire picked up on the key areas where

common errors had been made.

• During this time tutor spoke to all students with marks below 40% and;

• Students (9 in total) whose summative assessment mark was lower than their formative assessment mark.

• LIM tutor spoke to all students marked below 40%

Findings

• 58 students undertook the formative feedback exercise in WK 6 for ML140

• 84.5% improved their marks post formative feedback

• 15.5% saw a reduction in marks post formative feedback of whom 45% did not collect their feedback prior to summative assessment • 4 of these students had not picked up their formative

feedback prior to summative assessment

• Formative feedback is highly resource intensive

Comparative mark averages

•

  LAS LAS LAS LIM

  22/11/2010 07/01/2011 07/12/2009 07/01/2011

  Formative Summative Summative Summative

No of Attempts 58 78 74 88Average Mark 52.2 54.9 53.3 53.1Max Mark 67.0 74.0 74.0 72.0Min Mark 20.0 28.0 35.0 33.0St Dev 9.0 10.8 8.4 9.0         < 35 2 1 0 135 - 39 1 8 4 840 - 49 20 20 18 2450 - 59 20 23 28 3160 - 69 15 21 21 22>= 70 0 5 2 2

Findings formative & summative feedback LAS 2010/11

Mark changes for formative feedback students LAS

Findings of formative & summative for LAS & LIM 2010/11 compared

•

Findings Comparison 2009 LAS and 2010/11 LAS formative & summative Assessment

Combined average mark for LAS & LIM 2010 compared to 2009 LAS only mark

•

Summary of key findings

• The average mark increased for students who completed formative feedback in 2010 in LAS by 85%

• The process does not appear to work for everyone as 15% failed to improve

• Does formative feedback really make a difference? It seems to have worked for this cohort

• Improvement appears slight based on comparison between 2009 and 2010 statistics improvement - why?

Work in progress observations

• Aligning the teaching strategies & approaches for year 1 students is of benefit to students

• (Kift 2010). Does it make a real difference?• Tracking progress of students at the end of Y1 will

be crucial. Consider quantitative & qualitative? • Realignment has encouraged a teaching team to

work more closely together; • to develop strategies of teaching and assessment to

provide a more holistic approach to ensuring the needs of the first year student are met and

• key skills are developed, practised and consolidated over a cross section of modules.

Points for future consideration

• Change the method used for formative assessment? • whilst still ensuring it is of value to the

student but less resource intensive for tutors• Smaller tasks could be set that focus on the

key skill sets required for the FYE.

• Consideration given to using outcome based peer assessment in class, focused on key skill development

• Is attendance/capability an issue?

References

• Yorke M. and Longden B. (2008) ‘The first-year experience of higher education in the UK’ Higher Education Academy. Available online at• www.heacademy.ac.uk/assets/York/documents/.../

FYEFinalReport.pdf• US National Resource Centre

• http://www.sc.edu/fye/research/index.htm• FYE Curriculum Design Symposium

• http://www.fyecd2009.qut.edu.au/resources/• Kift S. (2009) ‘Articulating a transition pedagogy to

scaffold and to enhance the first year student learning experience in Australian higher education. Available online at:• http://www.altc.edu.au/resource-first-year-learning-

experience-kift-2009