Assessment of Teacher Candidate Reflection in E-Portfolios

A paper presented at the 2013 AACE EdMedia Conference Presentation

Daihong ChenAndrew Lumpe

Dan Bishop

Teacher Reflection Reflection: an active, persistent, and careful consideration of any belief

or supposed form of knowledge in light of the grounds supporting it and future conclusions, to which it tends. (Dewey, 1933, p.43)

Teacher reflection: spontaneous critical scrutiny of teachers’ beliefs and knowledge pertaining to teaching and learning, as well as practice and effects elicited by those beliefs and knowledge. (Sung, Chang, Yu, Chang, 2009)

Teacher reflection is significantly correlated with effective instructional practice, classroom organization, professional development, and teaching efficacy. (Giovanelli, 2003; Sparks-Langer et al. 1990; York-Barr et al. 2001)

Schön’s Two Time Frames (1983)

Reflection-in-Action: reflective action occurs simultaneously during the teaching and learning. Modification is made immediately.

Reflection-on-Action: reflective action takes place before or after the action according to a retrospection of the teacher.The Focus of this Study

E-portfoliosA promising and progressive approach for teacher preparation (Granberg, 2010; Green, 2008; Kim, Ng, & Lim, 2010; Zawacki-Richter, Hanft, & Baecker; 2011)

Four pillars of e-portfolios (Barbera, 2009; Black & Willliam, 1998): 1. Metacognition

2. Authentic tasks

3. Contextual feedback

4. Student responsibility

Three main purposes of using e-portfolios in teacher education:5. Contribute to constructivist learning

(Mair, 2012; Maher & Gerbic, 2009; Meeus, Questier, & Derks, 2006; Ritzhaupt, Ndoye, & Parker, 2010);

6. Provide summative and formative assessment (Barrett, 2007; Granberg, 2010; Luchoomun, McLuckie & van Wesel, 2010; Ritzhaupt, Ndoye, & Parker, 2010);

7. Showcase teacher candidates’ achievements and competencies (Granberg, 2010; Johnson-Leslie, 2009; Lumsden, 2007).

E-portfolios & Teacher Reflection Reflective writing in e-portfolios is proposed as a contributor

for preservice teachers’ constructivist learning and professional preparation (Tzeng & Chen, 2012).

Web 2.0 portfolios may serve as effective vehicles for fostering critical reflection (Lumpe & Wicks, 2011; Tan, 2006; Godwin-Jones, 2008; Bartlett-Bragg, 2003; Oner, & Adadan, 2011; Pechone, Pigg, Chung, and Souviney, 2005).

Levels of quality in reflection vary and most reflective writing remains at a low level (Ayan, & Seferoglu, 2011; Bauer & Dunn ,2003; Lai & Calandra, 2010; Parkes & Kajder, 2010; Seng, 2004).

The depth of reflectivity in e-portfolios remains to be examined and developed (Carney, 2006; Delandshere &Arens 2003).

Instruments for classifying reflectivity

Sparks-Langer et al. (1990, p.27) : (1) no descriptive language

(2) a simple, layperson description

(3) events labeled with appropriate terms

(4) explanations with traditional or personal preferences given as the rationale

(5) explanation with principle or theory given as the rationale

(6) explanation with principle/theory and consideration of contextual factors

(7) explanation with consideration of ethical, moral or political issues

Hatton and Smith (1995) Kember et al. (2000, 2008) Ward and McCotter (2004)

descriptive writing descriptivereflectiondialogic reflectioncritical reflection

habitual action understandingreflection critical reflection

routinetechnicaldialogictransformative

6

Current Study- Context Graduate Teacher Preparation Program One year period All teaching candidates maintained a bPortfolio on

www.wordPress.com Random sample of 50% of bPortfolios (n=41)

7

Using bPortfolios Initial set up of bPortfolio and training Reflective posts made during courses/internships Posts linked to standards via categories Artifacts include text files, A/V, or web links Meta-reflections served as summative posts Peer and instructor feedback via comments link Summative evaluation by faculty at end of program

Tags forIndividual Post

Categories (typically programstandards)

ReflectivePost for Weekly Module

StudentGeneratedTitle

Optional Graphics,Links, etc. Comments for

Peer or TeacherFeedback

9

10

“Doing bPortfolios”

10 Questions about bPortfolios Assessing bPortfolios Executive Summary of bPortfolios Sloan C Effective Practice Award 2011 Sample student bPortfolios

http://hamiltonlauren.wordpress.com/ http://rollis1.wordpress.com

Research Questions What is the level of reflection in preservice teacher

candidates’ e-portfolios? What text analysis variables predict the level of

reflection?

Methods Blog coding for level of reflection Text analytics via www.semantria.com Descriptive statistics Multiple regression

12

Coding Blog Reflection Level

Two coders coded all posts. Reached 85% interrater agreement.

Adapted from Kember, 2000, 2009; and Lai & Calandra, 2010

• An activity is performed automatically1 – Habitual Action

• Learning occurs without personal/practical application2 - Understanding

• Self-centered concerns3 - Routine Reflection

• Response to a situation without change4 - Technical Reflection

• Focus on student learning5 - Dialogic Reflection

• Focus on change in professional practice6 - Transformative Reflection

• A major shift in personal perspective/beliefs over time7 - Critical Reflection

13

Text Analytics of Blog Text Form of learner analytics Used www.semantria.com Excel Plug-in Based on semantic linguistic algorithms from

http://www.lexalytics.com/ Text submitted to Semantria servers via API All blog text analyzed for

Themes- noun phrases with contextual relevance scores. “What are they writing about?”

Document Sentiment – positive or negative tone of text Facets – “Meta Themes” which rely on subject-verb-object parsing.

14

1 - habitual action 2- understanding 3 - routine reflection 4 - technical reflec-tion

5 - dialogic reflec-tion

6 - transformative reflection

7 - critical reflection

Percent 10.3644107903455 26.5026029342168 24.2782773308093 14.9077141504969 9.27591102697586 11.7368670137246 2.8868906767629

2.5

7.5

12.5

17.5

22.5

27.5

Reflection Level of Coded Blog Posts

• 36.9% of posts were non-reflective (orange bars)• 63.1% were reflective (red bars)

Findings and Discussion Higher level reflections occurred in the later writings. Blogs may promote reflection.

The teacher candidates’ reflection tends to focus on self-centered concerns (e.g. time management issues, workload, personal emotion, and recognition for personal success).

Teacher candidates’ were not giving much attention to the learning of students.

The candidates spent little time writing about changes or major shifts in professional practice.

Most teacher candidates posted two entries per week during their internship. The average number of blog entries was 78 with a range of 35 -122.

The average number of unique facets (meta-themes) was 17.8 per candidate with a range of 15-33.

The overall score of bPortfolio document sentiment or emotional tone was +.26 on a scale of -1.0 to +1.0 (mostly positive).

Findings and Discussion of Text AnalysisF = 2.8, p = .03. The adjusted R square was .29, with 29% of the variance in blog reflection explained by the model variables.

Variable B Std. Error

Beta t Sig.

Blog Themes-.002 .001 -.494 -2.426 .023

Blog Facets.000 .000 .544 2.645 .014

WEST-E certification content test -.026 .008 -.566 -3.225 .004

Blog Sentiment-3.468 2.2 -.282 -1.56 .131

Table 1: Variables in the Regression Model Predicting Reflection Level in the bPortfolios

17

Conclusions

Blogging portfolios may be an effective tool for fostering professional reflective practice.

Teacher candidates’ blogs displayed relatively low levels of reflection.

Preservice teachers may not develop well-grounded beliefs and concepts of student-centered learning.

Assessment and analysis of teacher reflection in e-portfolios could be an effective means to reveal preservice teachers’ beliefs.

There is a need for training on reflective writing and ongoing assessment and support.

18

Recommendations Professional preparation programs should consider using web-

based blogging portfolios to enhance reflective practice.

Professionals utilizing such portfolios should be given structured training on reflective writing via blogs.

Efforts on exploring attributors influencing reflective writing and strategies for promoting reflective writing are recommended in future study.

Provide effective strategies to transfer teacher candidates’ reflection from self-centered concerns to professional competencies directly related to student learning.

Comments or Questions?

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/us/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.