What works with worked examples: Extending self ...heckler/Published papers...What works with worked...

What works with worked examples: Extending self-explanation and analogicalcomparison to synthesis problems

Ryan Badeau,1 Daniel R. White,1 Bashirah Ibrahim,2 Lin Ding,2,* and Andrew F. Heckler1,†1Department of Physics, The Ohio State University, Columbus, Ohio 43210, USA

2Department of Teaching and Learning, The Ohio State University, Columbus, Ohio 43210, USA(Received 12 May 2017; published 31 August 2017)

The ability to solve physics problems that require multiple concepts from across the physicscurriculum—“synthesis” problems—is often a goal of physics instruction. Three experiments were designedto evaluate the effectiveness of two instructionalmethods employingworked examples on student performancewith synthesis problems; these instructional techniques, analogical comparison and self-explanation, havepreviously been studied primarily in the context of single-concept problems. Across three experiments withstudents from introductory calculus-based physics courses, both self-explanation and certain kinds ofanalogical comparison of worked examples significantly improved student performance on a target synthesisproblem, with distinct improvements in recognition of the relevant concepts. More specifically, analogicalcomparison significantly improved student performancewhen the comparisonswere invoked betweenworkedsynthesis examples. In contrast, similar comparisons between corresponding pairs of worked single-conceptexamples did not significantly improve performance. On a more complicated synthesis problem, self-explanation was significantly more effective than analogical comparison, potentially due to differences inhow successfully students encoded the full structure of the worked examples. Finally, we find that the twotechniques can be combined for additional benefit, with the trade-off of slightly more time on task.

DOI: 10.1103/PhysRevPhysEducRes.13.020112

I. INTRODUCTION

Problem solving is a complex and multifaceted process.Accordingly, there has been a significant investment inproblem solving research in physics exploring problemsolving frameworks, novice vs expert problem solvingstrategies, and related procedural skills (for reviews, seeRefs. [1–3]). However, the vastmajority of these studies havetypically focused on problems requiring the application ofone single, isolated physics concept (e.g., Refs. [4–10]).The following series of experiments seeks to investigate

a specific subclass of physics problem, which we will referto as a synthesis problem: namely, a question requiring theapplication of more than one major physics concept, oftenfrom disparate parts of the teaching timeline [11]. Synthesisproblems are of importance for both theoretical andpractical reasons. Practically, synthesis problems are oftencloser to real world situations in their complexity. As aresult, improving student success on synthesis problems isconsistent with the goal of better preparing future engineers

and scientists. Within physics education research, synthesisproblems are similar to context-rich problems in thispursuit—a topic of ongoing research in both generalproblem solving [12–14] and computer-aided tutoring[15,16]. Synthesis problems are also of theoretical interestas they provide unique difficulties for students in therecognition and joint application of multiple concepts[17–20]. Our previous studies showed that these distinctchallenges extend beyond just the sum of difficultiesrepresented by the individual component concepts. Inparticular, the recognition of multiple concepts becomesa significant bottleneck in the context of these morecomplicated problems [20]. This difficulty with synthesisproblems is likely exacerbated by end-of-chapter textbookexercises and homework activities focusing on practicingonly the most recently learned material in the context ofsingle-concept problems. Students often approach theseend-of-chapter exercises with documented “plug-and-chug” algorithms that do not necessarily scale successfullyto situations with multiple interconnected physics concepts[21–23]. As such, the experiments here represent a novelfocus on extending instructional methods based on workedexamples specifically to synthesis problems and the uniquechallenges therein, namely, multiple concept recognition.

A. Worked examples

Worked examples consist of a problem statement and acorresponding set of solution steps, often with the implicitgoal ofmodeling an expertlike approach to the solution of theproblem. Previous research has shown that worked examplescan be extremely effective in aiding novice learners as they

*Corresponding [email protected]

†Corresponding [email protected]

Published by the American Physical Society under the terms ofthe Creative Commons Attribution 4.0 International license.Further distribution of this work must maintain attribution tothe author(s) and the published article’s title, journal citation,and DOI.

PHYSICAL REVIEW PHYSICS EDUCATION RESEARCH 13, 020112 (2017)

2469-9896=17=13(2)=020112(27) 020112-1 Published by the American Physical Society

https://doi.org/10.1103/PhysRevPhysEducRes.13.020112




https://creativecommons.org/licenses/by/4.0/

https://creativecommons.org/licenses/by/4.0/

attempt to master domain-specific knowledge and problemsolving skills, especially in highly structured domains suchas physics [24–26].Moreover, seminalwork bySweller et al.demonstrated that with careful, principle-based instructionaldesign, studying worked examples can be significantly moreeffective than individual practice solving problems [24].This “worked example effect” has traditionally been framedin terms of cognitive load theory—worked examples areeffective because they reduce extraneous load associatedwith inefficient problem solving strategies [27,28]. Ratherthan devote limited cognitive capacity to plug-and-chugalgorithms and equation matching heuristics, a fully workedexample allows the novice to instead focus on extracting therelevant solution structure and construct a conceptual schemafor subsequent use on other novel problems.

B. Self-explanation

Effective interventions based on worked examplesare often coupled with prompted or spontaneous self-explanations, whereby novices seek to explain the rationaleand structure of the worked examples to either themselves oran interested third party. The importance of self-explanationwas identified in a study by Chi et al., which asked collegestudents to voluntarily self-explain to an experimenter asthey studied examples of introductory mechanics problems[29]. They found that students that generated more high-quality self-explanations performed significantly better onfollow-up problem solving tasks than their peers. That resultwas then confirmed and expanded upon in multiple studiesin physics and other domain areas, such as biology, algebra,and computer programming [30–33].However, as with many of the aforementioned studies on

worked examples, the problems and applications used inprevious work have predominately focused on masteringisolated concepts and their application to single conceptproblems, such as Newton’s second law in the context ofan equilibrium problem (in the case of the original Chiexperiments). As such, our goal is twofold: first, to extendthe application of self-explanation specifically to thedomain of synthesis problems in physics; and second, tocompare the effectiveness of self-explanation within indi-vidual worked examples to analogical comparison across apair of worked examples.

C. Analogical comparison

Analogical reasoning is a mechanism of applying whathas been previously learned from a base situation to a new,analogous target situation. Successful analogical reasoningrequires that a person recognize base-target similarity,perform structural mapping, and subsequently apply thebase solution to the target [34–39]. In physics, researchershave used analogical reasoning to facilitate student con-ceptual learning [40–45]. Although the methods and imple-mentation have differed, the primary goal has often been tohelp novices acquire understanding of a novel concept viaanalogies to a situation that the student already comprehends

(such as invoking the idea ofwater flow to understand currentin a circuit, or scaffolding a series of analogies to aidconceptual understanding of normal force).Here, we focus on a specific type of analogical reasoning

known as analogical comparison. Analogical comparisoninvokes student comparison between two worked exampleswith the intent that students extract the necessary structure totackle a related target problem.The techniquewas explored ina study by Gentner, Loewenstein, and Thompson that testedthe use of analogical comparison with undergraduates andbusiness-negotiation techniques [38]. In their studies, partic-ipants were asked to explicitly compare and contrast twoisomorphic base examples before solving a related targetproblem. It was found that this analogical comparisonbetween base examples facilitated learners to recognize,map, and apply key principles significantly better than didthe traditional technique of using only one single baseexample. Given previously documented student difficultiesrecognizing component concepts when solving synthesisproblems [20,43], we posit that this technique of analogicalcomparison may be particularly suited to helping studentssolve physics synthesis problems. Since analogical compari-son emphasizes the identification of conceptual structure, itmay assist students to overcome the characteristic multipleconcept recognition and joint application bottlenecks thatwere identified in our prior studies on synthesis problems[11,17–20].This proposal is further supported by previous studies in

physics that have tested the effectiveness of isomorphicworked examples and analogical reasoning as a method toimprove student problem solving. In particular, Lin andSingh have previously shown that invoking student discus-sion and comparison of a single isomorphic worked exampleto a target multiconcept problem can improve student use ofthe relevant physics concepts [46]. Interestingly, they foundthat students who were first asked to try to solve the targetproblembefore comparing it to the providedworkedexampleperformed significantly better on their subsequent solutionof the target problem compared to participants who wereprovided the worked example, scaffolding prompts, andexplicitly told that the target problem and provided workedexample shared the same physical concepts (energy con-servation and centripetal acceleration).In comparison to the work of Lin and Singh, where a

single isomorphic worked example was provided to thestudents for study and use on the target problems, the seriesof studies conducted here specifically employ analogicalcomparison across pairs of worked examples. By providingpairs of worked examples with similar solution structure,we test the hypothesis that analogical comparison can assiststudents to extract the overall solution structure of a targetsynthesis problem while minimizing the impact of surfacefeatures from the provided worked examples. In short,analogical comparison—through an appeal to similaritiesand differences across the worked examples—may serve asan effective way to help students create a generalizable, andthus readily transferable, solution schema.

RYAN BADEAU et al. PHYS. REV. PHYS. EDUC. RES. 13, 020112 (2017)

020112-2

D. Research goal and experiment overview

In light of these previous results, we sought to explorehow students utilize worked examples specifically in thecontext of synthesis problems. In particular, we sought totest whether or not analogical comparison across examplesfacilitated student recognition of relevant concepts andimproved their performance when solving a novel synthesisproblem. Along with this overall research goal, we con-sidered the following related questions. First, given theincreased complexity of synthesis problems, is it moreeffective to invoke comparisons between worked examplesthat break down the target synthesis problem into its single-concept parts, or worked examples that include the con-cepts in combination? Second, how does the focus of theprompts influence analogical comparison (i.e., promptsinvolving holistic structure and overall concept recognitionvs prompts for fine-grain applications of the individualconcepts)? Third, how does analogical comparison across apair of worked examples compare with self-explanation ofeach worked example independently? These questions havebeen addressed by a series of three experiments, illustratedschematically in Fig. 1.

II. EXPERIMENT 1: ANALOGICAL COMPARISONAND SYNTHESIS PROBLEMS

A. Method

1. Design

The goal of the first experiment was to compare severalmethods of analogical comparison to baseline performancefrom course instruction alone (control) and to recentpractice solving single-concept problems (priming). Inorder to test the effectiveness of analogical comparisonin training students to solve synthesis problems, wedesigned a target synthesis task that would require appli-cation of two physics concepts: energy conservation andcircular motion. The target synthesis problem used for thisstudy is shown in Fig. 2(c). In addition to being relevantto the students’ course—it represents a canonical situationpresented in various problems within introductory physicscourses—the problem was chosen based on previous workwhich documented significant student difficulties with asimilar problem [20].Three different interventions using variations of

methods for analogical comparison were designed. These

Experiment 1

Priming Condition (Solve single-concept problems)

Single-concept worked examples & mastery prompts

Synthesis worked examples & mastery prompts

Synthesis worked examples & recognition prompts

Unrelated physics tasks (10 -15 minutes)

Ana

logi

cal C

ompa

riso

n

Con

diti

ons

Experiment 2 Experiment 3

Synthesis worked examples & combined prompts


+ Annotations


+ Summarization

Control Condition (No training)

Single-concept worked examples & mastery prompts

Synthesis worked examples & mastery prompts

Synthesis worked examples & recognition prompts


Summarization Condition (Self-explain synthesis

worked examples)

Control Condition

Control Condition (No training)

Ana

logi

cal C

ompa

riso

n C

ondi

tions

Summarization Condition (Self-explain synthesis

worked examples)

Ana

logi

cal C

ompa

riso

n C

ondi

tion

s

Randomly assigned condition

Unrelated physics tasks (10-15 minutes)

Target synthesis problem







(No training)

FIG. 1. The experimental design used in each of the three experiments. Experiments 2 and 3 used the same target synthesisproblem.

WHAT WORKS WITH WORKED … PHYS. REV. PHYS. EDUC. RES. 13, 020112 (2017)

020112-3

interventions consisted of pairs of worked examples andsets of written prompts that asked students to compare theprovided examples. Students were asked to write theirresponses to these prompts without additional feedback.Although the worked examples were intended to be similarto one another and to the target question (in terms of bothphysical context and the underlying conceptual structure ofthe solution), we purposefully varied the work flow of theprovided solutions (i.e., the solutions applied the conceptsin different orders) to avoid emphasizing a particularsolution order for students to trace on the target problem.Examples of problems used as worked examples areincluded in Figs. 2(a) and 2(b). Full versions of all workedexamples are included in Appendix A.The three analogical comparison interventions varied

both the type of worked example to be compared and thequestions used for comparison, resulting in the followingconditions: (i) single concept examples with masteryprompts (subsequently referred to as single-concept–mastery),(ii) synthesis examples with mastery prompts, and (iii) syn-thesis examples with recognition prompts.

The two sets of prompts—that is, mastery andrecognition—were designed to focus student attention ondifferent elements of the worked examples. The promptsinvolving single concept mastery explicitly targeted theapplication of the individual physics concepts within theworked examples, while the recognition prompts focusedon concept recognition and their combination within theworked examples. Examples of each type of comparisonprompt are included in Table I; the full set of questions areprovided in Appendix A.In addition to investigating different types of comparison

questions, we varied the type of worked example providedto the students, namely either single-concept or synthesisproblems. To this, the combination of the single-concept–mastery and synthesis-mastery conditions were designed tomeasure the effect of the type of worked example utilized foranalogical comparison. These conditions used the sameprompts for comparison with only minor changes to accountfor different line numbers in the solutions. In addition, thephysical contexts of the solutions, diagrams, and problemstatements were kept as similar as possible between the

(c) Target Synthesis Problem A 15.0 kg cart rolls down an incline into a circular loop of radius R = 7.0m. What is the minimum height, h, from which the cart can be released so that it will travel all the way around the loop without falling off? Show your work.

(b) Single Concept Worked Example #1 You are tasked with designing a rollercoaster. As part of the design, the track gradually descends until it comes to a small semi-circular hill of height R. You know that the speed of the rollercoaster cart will be 15 m/s when the cart is 20 m above the height of the oncoming hill. What is the velocity at the top of the oncoming hill?

( ) S i

b) Single Concept Worked Example #1

You are tasked with designing a rollercoaster. Aspart of the design, the track gradually descends untilit comes to a small semi-circular hill of height R.You know that the speed of the rollercoaster cartwill be 15 m/mm s when the cart is 20 m above the height of the oncoming hill. What is the velocity atthe top of the oncoming hill?

(a) Synthesis Problem – Worked Example

You are tasked with designing a rollercoaster. As part of the design, the track gradually descends until it comes to a small semi-circular hill of height R. You know that the speed of the rollercoaster cart will be 18.5 m/s when the cart is 20 m above the height of the oncoming hill. What is the minimum possible hill height for which the cart does not leave the surface of the track at the top? Ignore friction.

Single Concept Worked Example #2

You are tasked with designing a rollercoaster. As part of the design, the track includes a semi-circular hill of height R. You know that the speed of the rollercoaster cart will be 27 m/s at the top of the hill. What is the minimum possible hill height for which the cart does not leave the surface of the track at the top? Ignore friction.

Single Concept Worked Example #2

You are tasked with designing a rollercoaster. Aspart of the design, the trtt ack includes a semi-circularaahill of height R. You know that the speed of therollercoaster cart will be 27 m/mm s at the top of the hill.What is the minimumm m possible hill height foff r whichthe cart does not leave the surfaff ce of the trtt ack at thetop? Ignore frff iction.

FIG. 2. An example synthesis worked example (a), the corresponding single concept worked examples (b), and the target synthesisproblem (c).


020112-4

synthesis and single-concept worked examples. In order tokeep time on task as similar as possible across interventions,the analogical comparison conditions with synthesis prob-lems included a comparison of only a single pair of workedexamples. Students in the single-concept analogical com-parison condition compared two pairs ofworked examples: apair of worked examples involving circular motion, and apair of energy conservation examples.In principle, there are compelling reasons to expect both

methods to be successful. On the one hand, the single-concept problems are the embodiment of a reductionistapproach: break the overall problem solution structure intoits component parts and minimize cognitive load at eachstage of comparison. As a result, the reduced complexitymay assist students to recognize how to apply the individ-ual concepts to a following novel problem. On the otherhand, the synthesis worked examples are structurally moresimilar to the target synthesis problem, and include thestructural step of joining the two concepts.Alongwith the three analogical comparison interventions,

there were two additional conditions. A no-training controlwas included to establish a baseline of student performancesolely from course instruction. The final condition was thepriming intervention. The priming interventionwas includedto provide a comparison for effects from recent single-concept practice with the relevant physics concepts, namely,increased concept availability. Rather than having studentsexplicitly compare the worked examples, the priming con-dition asked students to solve two of the single conceptproblems—one of each concept—used as worked examplesin the analogical comparison conditions.

2. Participants

Tasks were administered during Fall 2015. Participantswere students in a calculus-based introductory mechanicscourse at The Ohio State University who participated aspart of a 1-hr-long flexible homework assignment forcourse credit. Students earned full credit for the assignment

based on participation. A total of 196 students wererandomly assigned into one of the five study conditions.

3. Administration

Students completed the training tasks and target synthesisproblem in individual carrels in a quiet room. An equationsheet similar to those used in the course was provided toall students. Tasks were administered and collected by theproctor one at a time, and students were allowed to work attheir own pace. Students first completed their selectedtraining, followed by 10–15 min of unrelated physics tasks,and then the target synthesis problem. The intervening,unrelated physics tasks were included to reduce short termmemory and priming effects from recent activation of thephysical concepts relative to control. Students in the controlcondition also completed a set of unrelated physics tasks andthe target synthesis problem, as shown in Fig. 1.

4. Method of analysis

A rubric for assessing student solutions of the targetsynthesis problem was determined by two of the authors. Inan effort to provide an authentic measure of student perfor-mance, the rubric was designed to mirror a grading schemethat could be applied in the students’ introductory course.The rubric is shown in Table II. After discussing the rubric,the authors coded all of the target student responses inde-pendently with an intercoder agreement of 80%. Conceptrecognition was coded generously (for example, a studentearned credit for recognizing energy conservation if theytried to apply a ½ mv2 term), but required the student tocommit to using the concept as part of their solution.Assessment of concept recognition was in complete agree-ment between the two coders. All disagreements werediscussed leading to the agreed upon scores presented here.

B. Results

Student final course grades in their introductory mechan-ics class were collected and compared across experimentalconditions. To eliminate outliers, two cuts were uniformly

TABLE I. Examples of the written prompts provided to students along with the worked examples.

Single-concept mastery prompts Recognition prompts

Consider the diagram in line 1 of solution 1 and line 6 ofsolution 2. Explain any similarities and differencesbetween the identified forces in terms of the two physicalsituations.

What are the main physical concept(s) used inboth of the students’ solutions?

Consider line 3 of solution 1 and line 8 of solution 2. Is thedirection of the mv2=R term the same or different in thetwo solutions? Explain your reasoning.

For problem 1, identify the elements of the problemstatement or diagram which indicate the need to use eachof the physical concept(s) you mentioned in part A. Listspecific elements and explain your reasoning.

In problem 1 line 3, the student substitutes in zero for FN.Explain why. Is the situation in problem 2 similar ordifferent to problem 1? Explain your reasoning.

Are the elements you identified above similar between thetwo problems? Explain your answer highlighting anydifferences and similarities.


020112-5

conducted across all conditions: studentsmust have completedthe course (removing 1 student), and have scored no lowerthan 2 standard deviations below the coursemean (removing atotal of 5 students, ranging from 0 to 3 students per condition).Given that the synthesis problem represents the combinationof single concepts covered as part of the students’ introductorymechanics course, these cuts were conducted to minimizeuninformative student difficulties in synthesizing those con-cepts that may have been due to simple unfamiliarity with therelated physics coursematerial. A one-wayANOVAof coursegrade showed no significant differences across conditions[Fð4;185Þ¼ 1.228, p ¼ 0.301].The mean score on the target synthesis problem (out

of a maximum of 9) and the number of students per

condition are shown in Table III, with corresponding scoredistributions included in Fig. 3. The distributions aredistinctly non-normal and roughly clustered into twodistinct groups: one group centered near a score of 3–4and the other at a total score of 8–9.A Kruskal-Wallis H test was conducted to determine the

effectiveness of the 4 interventions versus control (courseinstruction only). The mean rank of scores on the targetsynthesis problem was significantly different between con-ditions [χ2ð4Þ ¼ 36.2, p < 0.001]. Pairwise comparisons ofeach intervention to control were conducted using Dunn’sprocedure with a Bonferroni correction for multiple compar-isons (n ¼ 4). The adjusted p values are presented. Resultsindicated that while the priming condition was not signifi-cantly different from control (z ¼ 1.97, p ¼ 0.196), allthree analogical comparison conditions were significantlyhigher than control: Single-concept–mastery (z ¼ 2.632,p ¼ 0.034), synthesis-mastery (z ¼ 5.436, p < 0.001),and synthesis-recognition (z ¼ 4.379, p < 0.001).To test for hypothesized differences (H-1) between

analogical comparison and priming, (H-2) between single-concept and synthesis worked examples, and (H-3)between mastery and recognition prompts, we conducteda Kruskal-Wallis H test across only the interventionconditions. There was a significant difference in meanrank of scores on the target synthesis problem between theinterventions [χ2ð3Þ ¼ 16.72, p ¼ 0.001]. Five pairwisecomparisons were conducted using Dunn’s procedure witha Bonferroni correction for multiple comparisons. Theadjusted p values are presented. To test H-1, comparisonsshowed there were significant differences between primingand both the synthesis-mastery (z ¼ 3.718, p < 0.001) andsynthesis—recognition conditions (z ¼ 2.602, p ¼ 0.045),but not with single-concept–mastery (z¼0.721, p¼1.000).To test H-2, pairwise comparison showed there weresignificant differences between single-concept–masteryand synthesis-mastery (z¼ 2.772, p¼ 0.03). To test H-3,there were no significant differences between synthesis-mastery and synthesis-recognition (z ¼ 1.187, p ¼ 1.000).Taken together, these results demonstrate that the effective-ness of analogical comparison extends beyond just single-concept practice and concept activation (as evidenced by

TABLE II. Scoring rubric for the target synthesis problem. 1point was awarded for each item, for a total of 9 points.

Recognition

þ1 Energy conservationþ1 Centripetal acceleration

Application

þ1 ac applied correctly with Newton’s second lawþ1 Identify normal force ¼ 0 (minimum criteria)þ1 Correct initial potential energyþ1 Correct final potential energyþ1 Included final kinetic energy (velocity top of loop ≠ 0)þ1 Substitute correct final velocity from centripetal motionconstraint

Calculation

þ1 Correct final answer and no mathematical mistakes

FIG. 3. Distributions of student score on target synthesisproblem by treatment condition.

TABLE III. Mean score on target synthesis problem out of amaximumscore of 9 points by condition. Errors shown are standarderrors. Analogical comparison conditions are labeled AC.

Condition Score� SE N

Control (No training) 4.15� 0.37 40Priming 5.12� 0.39 43AC: Single concept exampleswith mastery prompts

5.69� 0.40 32

AC: Synthesis examples withmastery prompts

7.23� 0.36 35

AC: Synthesis examples withrecognition prompts

6.83� 0.40 40


020112-6

comparison to priming via single-concept problem solvingexercises). Moreover, while varying the type of promptshad no significant effect on student performance with thetarget synthesis problem (d ¼ 0.17), students who com-pared synthesis worked examples performed significantlybetter on the target problem than those who comparedexamples highlighting the component concepts in isolation(d ¼ 0.70).In addition to considering total scores on the target

synthesis problem, we analyzed the proportion of studentsrecognizing each of the two component concepts. Theproportion of students recognizing a concept is shown inFig. 4. Almost all students (≥95%) recognized and utilizedenergy conservation as part of their solution.The proportion of students recognizing centripetal

acceleration on the target synthesis problem varied con-siderably. A chi-squared test across treatment conditionsshowed there was a significant difference in the proportionof students recognizing centripetal acceleration whensolving the target synthesis problem [χ2ð3Þ ¼ 29.899,p < 0.001]. Post hoc comparisons between treatmentswere conducted using pairwise chi-squared tests with aBonferroni correction for multiple comparisons (n ¼ 5).The proportion of students recognizing centripetal accel-eration in both the synthesis-mastery and synthesis-recognition conditions was significantly different frompriming, [χ2ð1Þ¼18.059, p¼0.001] and [χ2ð1Þ¼19.267,p < 0.001], respectively. The proportion of students inthe single-concept–mastery condition was not significantly

different than priming [χ2ð1Þ ¼ 1.099, p ¼ 1.000]. Com-parison between the single-concept–mastery and synthesis-mastery conditions also showed a significant difference[χ2ð1Þ¼ 9.600, p¼0.01]. There was no difference betweenthe synthesis-mastery and synthesis-recognition condi-tions. Taken together these results support the trendsuggested by the total score on the target synthesisproblem: analogical comparison significantly increasedrecognition of centripetal acceleration, but only whenstudents compared synthesis examples.

C. Discussion

There are four important findings from this experiment.First and foremost, training via analogical comparison ofworked examples was effective in improving student scoreson a target synthesis problem relative to control. Second,training via analogical comparison behaved markedly betterthan priming (via problem solving exercises). Third, theeffectiveness of analogical comparison depended signifi-cantly on the type of worked examples to be compared, butnot the specific nature of the comparison prompts.The fourth finding is that student success on the problem

was bottlenecked primarily by their ability to recognize thepresence of the centripetal motion constraint. Given theanalysis of student concept recognition across control andthe four treatment conditions, the non-normal distributionsof student total scores are telling: without interventionstudents primarily solved the target synthesis problem as ifit were a single-concept problem. Once they were able toconceptually recognize conservation of energy and cen-tripetal acceleration, they almost universally shifted fromonly applying one concept to correctly applying both. As aresult, one of the strongest potential gains from training viaanalogical comparison—at least, specifically in the contextof synthesis problems—may be the improvement of studentconceptual recognition.Moreover, the finding that comparisons of synthesis

worked examples was significantly more effective thancomparisons of single-concept examples (using the sameprompts) supports the importance of this full structuraltransfer [38]. Combined with the fact that priming studentsvia explicit problem-solving practice was not significantlybetter than course instruction alone, these results are alsoprescriptive. Namely, these results weaken the often-heldassumption that students can repeatedly practice physicsconcepts in isolation, and simultaneously expect success onproblems that combine them. Instead, these results suggestthat integration does not happen spontaneously, at least notfor solving complex physics problems. As such, in contrastto the vast majority of introductory homework problemsand end-of-chapter exercises, success with synthesis prob-lems may best be facilitated by explicit practice withsynthesis problems.We consider two potential explanations for the finding of

no significant difference between the comparison prompts

FIG. 4. Proportion of students in each condition demonstratingrecognition of centripetal acceleration and energy conservation.Error bars are standard errors.


020112-7

focused on individual concept application and those focusedon overall concept recognition and structure. First, notingthat both conditions were quite successful relative to addi-tional practice solving single-concept problems, it is possiblethat ceiling effects are limiting the possibility for anydifference in overall effectiveness. Second, given thatstudents tend to switch from applying a one-concept onlysolution on the target synthesis problem in the control andpriming conditions to providing fully correct solutions aftercompleting the analogical comparison tasks, it is possiblethat the greatest benefit of the analogical comparisonprompts is that they force the student to sufficiently encodethe two synthesis examples; because the students are sosuccessful with the recognition of the individual conceptsonce they have encoded the combined concept structure, thespecific comparisons themselves are less important.Overall, the results of this experiment suggest that

analogical comparison can be an effective technique fortraining students to solve synthesis problems—at least forproblems that demonstrate the level of conceptual andmathematical complexity represented by the target synthe-sis problem. However, even considering the vast array ofpotential physics concepts and combinations, the successhere is promising. After all, this target synthesis problemand the corresponding base worked examples are alreadymore involved than other previous, successful examples ofanalogical comparison [38,39]. Still, the results heresuggest natural follow-up questions: Is analogical com-parison effective for a synthesis problem with increasedcomplexity? Is analogical comparison as effective asanother known effective problem solving interventionmethod, namely, self-explanation?

III. EXPERIMENT 2: ANALOGICALCOMPARISON VS EXPLANATION

A. Method

1. Design

The goals of experiment 2 were to explore the effective-ness of analogical comparison in the context of morecomplicated introductory-level synthesis problems [see thetarget problem in Fig. 5(c)] and to compare the effectivenessof analogical comparison to the method of self-explanation.A full solution of the target problem requires three mainconceptual components: simple circuit analysis (Ohm’s law),induced EMF (Faraday’s law), and magnetic force. Inparticular, the concepts of magnetic force and Faraday’slaw represent distinct and documented challenges for stu-dents [47,48]. In the case of magnetic force, students mustsuccessfully interpret cross products. Faraday’s law requiresan implicit understanding of magnetic flux and the consid-eration of direction via Lenz’s law. Correspondingly, acompact solution of the target synthesis problem utilizesnot only three basic physics equations, but considerablymore algebraic manipulations than that required to solve

experiment 1. Along with the target synthesis problem, wedesigned a corresponding set of single-concept and synthesisworked examples [see Figs. 5(a) and 5(b)].Experiment 2 had six conditions (see Fig. 1). Four of the

conditions (control and three analogical comparison con-ditions) were similar in structure to experiment 1. The fifthcondition was aimed at avoiding a potential shortcoming ofthe other analogical comparison conditions. This conditionrepresented a “best-effort” attempt to scaffold comparisonsthrough a mix of recognition and single-concept focusedprompts. In particular, the prompts explicitly addressed theconcept of induced emf and its application within the twosynthesis worked examples. For example, one prompt in thiscondition states “One of the important concepts is that of aninduced emf due to a changing magnetic flux. Compare thephysical reason for a changing magnetic flux in each of theproblems. Explain your answer highlighting any differencesbetween the two problems.” In contrast, the other conditionsdid not explicitly invoke the concept of Faraday’s law.Further, the combined “best-attempt” condition tried toscaffold comparison that followed the worked examples:starting with recognition of the relevant concepts, throughconsideration of the physical context for Faraday’s law, andthen subsequent application of the component concepts inthe worked examples. The other analogical comparisonconditions were restricted to focus on either recognitionof the concepts or their application as in experiment 1. Fullversions of all worked examples and corresponding com-parison prompts are included in Appendix B.Finally, the sixth condition was the self-explanation

condition. This condition simply prompted students toexplain (write) both of the synthesis worked examples asif to a friend, but did not invoke explicit comparison betweenthe two. Ample space was provided for the explanation.

2. Participants

Tasks were administered during Spring 2016. Partici-pants were students in the second semester of a calculus-based introductory electromagnetism course at The OhioState University. Students participated as part of a 1-hr-longflexible homework assignment for course credit. Studentscompleted the flexible assignment over a three weekwindow, after course instruction on Faraday’s law, andin close proximity to a course midterm covering therelevant material. A total of 254 students were randomlyassigned into one of the six study conditions.

3. Administration

Students completed the training tasks and target synthe-sis problem in individual carrels in a quiet room. Anequation sheet similar to those used in the course wasprovided to all students. Tasks were administered andcollected by the proctor one at a time, and students wereallowed to work at their own pace. Whereas students in theanalogical comparison conditions were given all relevant


020112-8

worked examples and prompts together, students in theself-explanation condition were given one synthesisworked example to summarize at a time. Students firstcompleted their selected training, followed by 10–15 minof unrelated physics tasks, and then the target synthesisproblem. Students in the control condition completed a setof unrelated physics tasks and the target synthesis problem.


A rubric for assessing student solutions of the targetsynthesis problem was determined by two of the authors.The rubric is shown in Table IV. As in experiment 1,recognition of component physics concepts was codedgenerously, but required the student to commit to using the

concept as part of their solution. A random sample of 25student solutions was coded by two researchers with anintercoder agreement of 84%. Disagreements were dis-cussed and resolved.

B. Results

Student final course grades in their introductory electro-magnetism class were collected and compared acrossconditions. The same cuts conducted in experiment 1 wereapplied to eliminate outliers. Students must have completedthe course (removing no students), and have scoredno lower than 2 standard deviations below the mean(removing a total of 9 students, ranging from 1 to 4students per condition). A one-way ANOVA of course

(c) Target Synthesis Problem

A conducting bar of negligible resistance and length 30 cm is sliding along a pair of frictionless rails at a speed of v = 4.0 m/s as shown in the figure. A uniform magnetic field of B = 11.0 T is directed into the page. The battery voltage is 24V and the resistances are R1= 10Ω and R2= 15Ω.

In order to keep the rod moving at a constant speed, an additional force needs to be applied on it. Find the magnitude and direction of this applied force.

(b) Single Concept Example #1 A square loop of conducting wire, length 40 cm is being pulled at a constant speed 10.0 m/s through a region with a uniform magnetic field B = 2.3 T directed out of the page as shown. At the instant shown, when part of the circuit is still in the region with a magnetic field, find the direction and magnitude of the induced emf in the loop.

b) Single Concept Example #1

A square loop of condf ucting wire, length 40 cm is being pulled at aconstant speed 10.0 m/mm s through aregion with a unifoff rm magneticfiff eld B = 2.3 T directed out of thepage as shown. At the instantshown, when part of the circuit isstill in the region with a magneticfiff eld, fiff nd the direction andmagnitude of the induced emf inthe loop.

(a) Synthesis Problem – Worked Example

A square circuit of length L = 40cm and unknown mass m is falling (gravity directed as shown) through a region with a uniform and perpendicular magnetic field B = 2.3 T directed out of the page. The resistors have resistances of R1 2= 10Ω and R = 25Ω, and the voltage of the battery is 9V. At the instant shown, when part of the circuit is still in the region with a magnetic field, the velocity of the object is constant and equal to 10.0 m/s downward. Find the mass of the circuit.

Single Concept Example #2 A current carrying wire of length L = 40cm is falling (gravity directed as shown) at a constant velocity through a region with a uniform and perpendicular magnetic field B = 2.0 T directed out of the page. If the current is 0.5A, find the mass of the wire.

Single Concept Example #2

A current carrrr ying wire of lengthL = 40cm is faff lling (gravity directed as shown) at a constantvelocity through a region with aunifoff rm and perpendicularaamagnetic fiff eld B = 2.0 T directedout of the page. If the currrr ent is0.5A, fiff nd the mass of the wire.


Given V1=9V, V2=5V, R1=10Ω and R2=15Ω, find the magnitude of the current in the circuit shown.


Given V1=9V, V2=5V, R1=10Ωand R2=15Ω, fiff nd the magnitudeof the current in the circuitshown.

FIG. 5. An example synthesis problem provided as a worked example (a), corresponding single concept problems (b), and the targetsynthesis problem (c).


020112-9

grade showed no significant differences across conditions[Fð5; 239Þ ¼ 0.554, p ¼ 0.735].Mean scores on the target synthesis problem are shown

in Table V. We first compared the interventions to control.A one-way ANOVA showed significant differencesbetween conditions [Fð5; 239Þ ¼ 5.961, p < 0.001]. ATukey HSD post hoc showed significant differencesbetween self-explanation and control (d ¼ 1.14, p <0.001) and synthesis–combined prompts and control(d ¼ 0.67, p ¼ 0.018). The other analogical comparisonconditions were not significantly different from control:single-concept–mastery (d ¼ 0.18, p ¼ 0.959), synthesis-mastery (d ¼ 0.44, p ¼ 0.363), and synthesis-recognition(d ¼ 0.44, p ¼ 0.296).A one-way ANOVA was conducted between inter-

vention conditions to test for hypothesized differencesbetween single-concept worked examples and synthesisworked examples, betweenmastery and recognition prompts,and between self-explanation and the combined, best-effortanalogical comparison condition. The one-way ANOVAshowed significant differences between interventions

[Fð4; 198Þ ¼ 4.697, p ¼ 0.001]. A Tukey HSD post hocshowed no significant difference in total score on the targetsynthesis problem between single-concept–mastery andsynthesis-mastery (d ¼ 0.25, p ¼ 0.776). There was nosignificant difference between synthesis-mastery and synthe-sis-recognition (p ¼ 1.00). Finally, although there were nosignificant differences between self-explanation and synthesis-combined (d ¼ 0.38, p ¼ 0.48), the self-explanation treat-ment significantly outperformed all of the other analogicalcomparison conditions: Single-concept–mastery (d ¼ 0.95,p ¼ 0.001), synthesis-mastery (d ¼ 0.70, p ¼ 0.042), andsynthesis-recognition (d ¼ 0.66, p ¼ 0.042).To sum up, only the best-effort attempt at analogical

comparison (synthesis worked examples using a combina-tion of scaffolded comparison prompts) and the self-explanation intervention were significantly better thancourse instruction alone (control) in terms of overall studentperformance on the target synthesis problem. In addition,there were no statistical differences in the effectiveness ofanalogical comparison based on either the type of workedexample or the type of comparison prompts. Finally,summarization via self-explanation was the most effectiveintervention, with significant differences in student perfor-mance versus all analogical comparison conditions exceptfor the highly scaffolded best-attempt condition.In addition to overall performance on the target synthesis

problem, we compared student conceptual recognitionacross conditions. The proportion of students recognizingeach component physics concept is shown in Fig. 6. Thevast majority of students recognized and utilized Ohm’slaw (≥93%) and magnetic force due to a current carryingwire (≥83%) across all conditions.A chi-squared test across treatment conditions showed

therewas a significant difference in the proportionof studentsrecognizing and utilizing Faraday’s law on the target syn-thesis problem [χ2ð4Þ ¼ 25.543, p < 0.001]. To test forhypothesized differences between the interventions, post hoccomparisons between treatments were conducted usingpairwise chi-squared tests with a Bonferroni correction formultiple comparisons (n ¼ 3). The adjusted p values arereported. To test for the hypothesized difference due to typeofworked example, a comparisonof single-concept–masteryand synthesis-mastery showed no significant differencein the proportion of students recognizing Faraday’s law[χ2ð1Þ ¼ 0.029, p ¼ 1.00]. To test for the hypothesizeddifference due to type of comparison prompt, a comparisonof synthesis-mastery and synthesis-recognition also showedno significant difference [χ2ð1Þ ¼ 1.749, p ¼ 0.558].Finally, a comparison between synthesis-combined andself-explanation showed a significant difference in theproportion of students recognizing Faraday’s law on thetarget synthesis problem [χ2ð1Þ ¼ 6.316, p ¼ 0.036].Overall, there was a statistically significant difference

in the proportion of students identifying Faraday’s lawbetween the self-explanation condition and the best

TABLE IV. Scoring rubric for the target synthesis problem.1 point was awarded for each item, for a total of 10 points.

Recognition

þ1 Faraday’s law or induced emfþ1 Ohm’s lawþ1 Magnetic force

Application

þ1 Correct induced emfþ1 Correct induced emf direction or Lenz’s lawþ1 Ohm’s law: series circuit or combined resistors in seriesþ1 Ohm’s law: combined voltage sourcesþ1 Correct magnetic force equationþ1 Correct applied force direction

Calculation

þ1 Correct calculations and no mathematical mistakes

TABLE V. Mean score on target synthesis problem out of amaximum score of 10 points by condition. Errors shown arestandard errors. Analogical comparison conditions are labeled AC.


Control 5.19� 0.35 42AC: Single concept examples withmastery prompts

5.60� 0.35 42

AC: Synthesis examples withmastery prompts

6.16� 0.35 38

AC: Synthesis examples withrecognition prompts

6.19� 0.35 42

AC: Synthesis examples with combinedprompts

6.75� 0.37 40

Self-explanation 7.54� 0.28 41


020112-10

attempt, combined analogical comparison treatment withthe self-explanation group outperforming the combinedanalogical comparison group. However, there was nosignificant effect from either the type of worked exampleor the type of comparison prompts on student recognitionof Faraday’s law for the target synthesis problem.In order to further examine the effect of training on

how students approached the target synthesis problem, wecompared the proportion of students across conditions whorecognized and applied Faraday’s law, Ohm’s law, and themagnetic force on a current carrying wire, and alsoexplicitly calculated a total current in the circuit thatcombined the voltage due to the battery with the inducedemf (but not necessarily with correct directions or magni-tudes). This combination was used to represent the mini-mum structure necessary for a correct approach to the targetsynthesis problem. The proportion of students successfullymeeting this threshold is shown in Fig. 7.

A chi-squared test across treatment conditions showedthere was a significant difference in the proportion ofstudents meeting this structural threshold [χ2ð4Þ ¼ 32.482,p < 0.001]. To test for hypothesized differences betweenthe interventions, post hoc comparisons between treatmentswere conducted using pairwise chi-squared tests with aBonferroni correction for multiple comparisons (n ¼ 3).Tests for hypothesized differences due to type of workedexample and type of prompt showed no significant dif-ferences (i.e., comparison between Single-concept–masteryand synthesis-mastery and between synthesis-mastery andsynthesis-recognition, respectively). However, comparisonbetween synthesis-combined and self-explanation didshow a significant difference in the proportion of studentswho applied all component concepts and calculated thecombined total current [χ2ð1Þ¼ 8.320, p ¼ 0.012]. Simplyput, the difference does not come from the first four treatmentgroups in Fig. 7; instead, it comes from the last group.

FIG. 6. Proportion of students in each condition demonstrating recognition of Faraday’s law, Ohm’s law, and magnetic force on thetarget synthesis problem. Error bars are standard errors.

FIG. 7. Proportion of students in each condition who employed the correct solution structure on the target synthesis problem. Errorbars are standard errors.


020112-11

C. Discussion

There are two main takeaways from experiment 2. First,of the analogical comparison interventions, only the scaf-folded best-attempt analogical comparison condition per-formed significantly better than control in terms of overallperformance on the target synthesis problem. Moreover,with other factors controlled, single-concept and synthesisworked examples produced similar results. The casewas the same comparing prompts focused on mastery orrecognition. Second, summarization via self-explanation ofsynthesis worked examples alone was not only the mosteffective intervention in improving students’ overall per-formance on the target problem, but it also significantlyimproved the recognition and use of Faraday’s law com-pared to every other intervention, including the best-attempt analogical comparison condition.The increased complexity of the target problem in

experiment 2 (compared to experiment 1) manifested itselfin the details of student performance on the problem. Inexperiment 1, student conceptual recognition was thedominant bottleneck to correctly solving the target prob-lem; once that difficulty was successfully overcome bythe analogical comparison interventions, students correctlyapplied the component physics concepts. In experiment 2,recognition alone was not enough to guarantee a com-pletely correct solution. The summarization conditionresulted in 90% of students recognizing and applyingFaraday’s law (and even more recognizing the other twocomponent concepts), but the mean score was still only7.54=10, in part due to remaining difficulties applyingLenz’s law and cross-product-based reasoning.The synthesis worked examples were also more com-

plex. We suggest that this manipulation of the synthesisworked example complexity is a likely reason for thesubsequent lack of statistically significant gains for thoseanalogical comparison conditions. The importance of theincreased complexity seems even more likely consideringthat the only statistically different analogical comparisoncondition versus control was the combined analogicalcomparison treatment. The combined prompts providedmore information to the student (explicitly identifyingFaraday’s law) and scaffolded the comparison of the twoworked examples more extensively than any of the otheranalogical comparison interventions. In order for theanalogical comparison to be effective with more compli-cated base examples, additional scaffolding was necessary.The necessity for additional scaffolding may not be too

surprising given the cognitive load and increased demandsfrom the more complicated worked examples. However,it is important to note that once again comparisons ofconsiderably simpler single-concept examples did notsignificantly improve student performance on the targetproblem. It appears that the structure of a synthesis problemcannot simply be broken into parts, even when studentsare asked to compare the set of parts one after another.

Unfortunately, this also suggests that analogical compari-son with single-concept examples alone is unlikely to be aneffective way to help students reduce the cognitive loadinherent to complicated synthesis problems.On the other hand, despite the lack of instructional

scaffolding or information beyond the worked example,students in the self-explanation condition performed sig-nificantly better than the control group on the targetsynthesis problem. Further, students in the self-explanationcondition performed significantly better in recognizing theneed for Faraday’s law compared to the best analogicalcomparison condition—despite the fact that the best-effortanalogical comparison condition explicitly pointed out theconcept as one of its comparison prompts. This result wasfurther supported by the proportion of students whogenerated the correct solution structure on the targetproblem—using all three identified physics concepts andexplicitly combining both the induced emf and the batteryvoltage (cf. Fig. 7).Given these results, there are several possible explan-

ations for the relative success of the self-explanationcondition. First, self-explanation might have been moresuccessful because the task of explaining each problem,one at a time, allowed students to better encode the entirestructure of each independent worked example. In contrast,students in the analogical comparison condition may havemade sense of smaller-grain component concepts across thebase examples, but without encoding the full structure ofeither example. This lack of encoding the full structure inthe analogical comparison conditions could be due to asimple failure of students to satisfactorily read throughthe base worked examples—that is, beyond what wasnecessary to answer each specific invoked comparison—or more nuanced differences in how the students extractedthe structure of the provided solutions. Experiment 3 wasdesigned to help exclude the first possibility. Two mech-anisms proposed by Chi [49] to explain the success ofself-explanation, namely, inference generation and mental-model revision, may account for other underlyingdifferences in student encoding.The mechanism of inference generation suggests that

summarization via self-explanation may have led to higherperformance by prompting students to fill in necessaryinformation and reasoning steps missing in the workedexamples. There is some evidence for such an effect: moststudents included not only the relevant physics conceptsin their summaries, but also additional justifications. Forexample, Faraday’s law and induced emf were addressed in97% of student summaries and 57% of students describedthe physical reason for the two cases of changing magneticflux. Moreover, student summaries seemed to follow thereasoning of the provided solutions. The majority ofstudent summaries discussed the concepts in the order thatthey were presented in the worked solutions while alsoidentifying important intermediate quantities. In particular,


020112-12

students often included explicit identification of the electriccurrent as a crucial unknown, e.g., “The next step is to solvefor the current since L, B, g are all known….however, toproperly solve this we need to solve for the E created by theB field.” Although that recognition was probably driven inpart by students relying on a given-unknown problemsolving heuristic, it may have allowed students to identifythe electric current as a structural connection between thephysics concepts in the problem. In contrast, the connec-tions between the analogical comparisons prompts may nothave been as strongly internalized by students, even thoughthe combined analogical comparison condition invokedcomparisons that explicitly targeted those exact elements.The second mechanism, mental-model revision, could

also account for the relative success of the summarizationcondition. The target synthesis problem—and its poten-tially novel combination of an induced emf and a battery—represented a significant challenge for students. It ispossible that students had an incomplete or disjointed priorunderstanding of electromotive force. If that was the case,summarization via self-explanation may have helped stu-dents to reconcile their mental model with the workedexamples. Future work is needed to differentiate betweenthese possible mechanisms.

IV. EXPERIMENT 3: ANALOGICALCOMPARISON WITH EXPLANATION

A. Method

1. Design

The third experiment was built upon the previous studyto both replicate the relative success of the self-explanationintervention and test whether analogical comparison andself-explanation of the individual worked examples can becombined for further benefit. As such, this study used thesame target problem employed in experiment 2. In additionto a no-training control, 4 treatments were included in theexperimental design. To replicate the results of experiment2, we once again included a best-effort analogical com-parison condition and self-explanation condition, using thesame worked examples as before. The prompts for theanalogical comparison condition were similar to thoseused previously and explicitly invoked the concept ofFaraday’s law. Given student’s prior difficulty with apply-ing the individual concepts—in particular, difficulties withdirection-based considerations due to Lenz’s law and crossproducts—we made adjustments to the prompts to encour-age further comparison of the important directions identi-fied in the worked examples. The full set of prompts isincluded in Appendix C.The other two conditions were designed to test whether

an increased emphasis on encoding the individual workedexamples before analogical comparison would increasestudent performance on the target synthesis problem. First,we included an “annotation” condition that asked students

to very briefly label the two individual worked examples.The prompts were intended to be brief checks to verify thatstudents had successfully read through the example sol-utions. As such, they simply asked students to identify boththe concepts used and the basic goal of sections of theworked examples (i.e., “Ohm’s law” and “find the totalcurrent”). In contrast to the self-explanation prompt,students were only asked to label the individual solutionsrather than provide additional justifications or elaborations.The presentation of these reading annotations is shown inAppendix C.The last condition sought to test the hypothesized idea

that self-explanation and analogical comparison could becombined for additional benefit. Given the increasedcomplexity of the base worked examples, we posited thatinviting students to first summarize the individual workedexamples independently would facilitate subsequent ana-logical comparison. Consequently, students may have abetter holistic understanding of the base examples and howindividual comparisons fit within the two overall solutionstructures, rather than viewing them as a set of uncon-nected, piecewise comparisons. As such, the combinedself-explanation and analogical comparison conditionasked students to first briefly summarize each workedexample before prompting them to compare across the twoworked examples.

2. Participants

Tasks were administered during Fall 2016. Participantswere students in an off-sequence calculus-based introduc-tory electromagnetism course at The Ohio State Universitywho participated as part of a 1-hr-long flexible homeworkassignment for course credit. The flexible homeworkassignment was administered over a three week periodnear the end of the semester, approximately one month onaverage after course instruction on Faraday’s law. A total of232 students were randomly assigned into one of the fivestudy conditions.

3. Administration

Students completed the training tasks and target synthe-sis problem in individual carrels in a quiet room. Taskswere administered and collected by the proctor one at atime and an equation sheet was provided to all students.Students first completed their selected training, followedby 10–15 minutes of unrelated physics tasks, and then thetarget synthesis problem, as shown in Fig. 1. Whereasstudents in the analogical comparison and annotationconditions were given all relevant worked examples andprompts together, students in both the self-explanation andcombined analogical comparison and self-explanation con-ditions were given only a single synthesis worked exampleto summarize at a time. Students in the control conditiononce again completed a set of unrelated physics tasks andthe target synthesis problem.


020112-13

Student time on task was digitally recorded by theproctor. Because of time constraints from the testingformat, students in the combined self-explanation andanalogical comparison condition were given a time limitof approximately 7 min per individual summary. Time ontask during the training interventions was collected for thevast majority of students (95%).


Student solutions to the target synthesis problem weregraded with the same rubric used in experiment 2 (shown inTable IV). A random sample of 25 student solutions wascoded independently by two researchers. Any differencesin coding were discussed and resolved leading to anintercoding agreement of 88%.

B. Results

Student final course grades in their introductory electro-magnetism class were collected and compared acrossconditions. The same cuts conducted previously wereapplied to eliminate outliers. Students must have completedthe course (removing no students), and have scored nolower than 2 standard deviations below the mean (removinga total of 6 students, ranging from 0 to 2 students percondition). Almost all students satisfactorily completed thetraining tasks. One student who did not complete thetraining task was removed from the combined analogicalcomparison and self-explanation condition. A one-wayANOVA of course grade showed no significant differencesacross conditions [Fð4; 220Þ ¼ 0.522, p ¼ 0.719].Mean scores on the target synthesis problem are shown

in Table VI. First, interventions were compared to control.A one-way ANOVA showed significant differences in totalscore on the target synthesis problem between conditions[Fð4; 220Þ ¼ 11.351, p < 0.001]. A Tukey HSD post hocshowed significant differences for all treatment conditionscompared to control: Comparison-only (d ¼ 0.77, p ¼0.003), comparison and annotations (d ¼ 0.92, p ¼0.001), self-explanation (d ¼ 1.05, p < 0.001), and com-parison and self-explanation (d ¼ 1.44, p < 0.001).A one-way ANOVAwas conducted between intervention

conditions to test for hypothesized differences in student

performance on the target synthesis problem. The one-wayANOVA showed significant differences between the treat-ments [Fð3; 176Þ ¼ 3.002, p ¼ 0.032]. In order to testfor hypothesized differences, a Tukey HSD post hoc wasconducted. The post hoc tests showed no significantdifference in total score on the target synthesis problembetween comparison-only and comparison and annotations(d ¼ 0.08, p ¼ 0.977), nor between comparison-only andself-explanation (d ¼ 0.23, p ¼ 0.653), but there was asignificant difference between comparison-only and com-parison and self-explanation (d ¼ 0.58, p ¼ 0.030).In summary, the combination of self-explanation and

analogical comparison was significantly better than ana-logical comparison alone. However, there was no signifi-cant difference between students who completed only theanalogical comparison prompts and the students who wereexplicitly asked to read through each example and providebrief annotations before making comparisons. There wasalso no significant difference between self-explanation andanalogical comparison (d ¼ 0.23), though the trend was inthe same direction as in experiment 2 (d ¼ 0.38), with self-explanation performing nominally better than analogicalcomparison. Taken together, this replication may suggest asmall, but potentially significant effect (d ≈ 0.3).The proportion of students recognizing each component

physics concept and employing it as part of their solutionon the target synthesis problem is shown in Fig. 8. Acrossall conditions, the majority of students successfully recog-nized and utilized Ohm’s law (≥80%) and magnetic forcedue to a current carrying wire (≥93%). A chi-squaredtest across all conditions showed there was a significantdifference in the proportion of students recognizing andutilizing Faraday’s law on the target synthesis problem[χ2ð4Þ ¼ 39.140, p < 0.001]. However, such a differencewent away if only the treatment groups were compared[χ2ð3Þ ¼ 5.658, p ¼ 0.129]. This suggests that while thefour treatment conditions all significantly improved con-cept recognition versus control, they did not differ signifi-cantly from one another.In addition, we compared the proportion of students who

met a minimum threshold for a correct approach on thetarget problem, namely, recognize and apply all threeconcepts and calculate a total current using both theinduced emf and battery voltage. The results are shownin Fig. 9. Along with clear differences between theinterventions and control, a chi-squared test was used tocompare students who did not self-explain the individualworked examples with those who did as part of theirtraining. There was a significant difference in the propor-tion of students who met the proposed threshold on thetarget synthesis problem [χ2ð1Þ ¼ 24.360, p < 0.001].Students who self-explained the individual worked exam-ples as a part of their training were significantly more likelyto recognize all three concepts and combine the two sourcesof emf.

TABLE VI. Mean score on target synthesis problem out of amaximum score of 10 points by condition. Errors shown arestandard errors.


Control 4.04� 0.30 45Analogical comparison 5.74� 0.35 46Analogical comparison andreading annotations

5.93� 0.32 45

Self-explanation 6.29� 0.34 45Analogical comparison and self-explanation 7.07� 0.34 44


020112-14

Time on task during training was recorded for the vastmajority of students (95%) and compared across the fourintervention conditions. Students without timing data wereremoved from the subsequent analysis, resulting in thecorresponding boxplots presented in Fig. 10. A median testshowed significant differences in time spent during trainingacross the conditions. In particular, comparison and anno-tation and comparison and self-explanation represent anincrease of approximately 20% and 35% (5 and 8 min,respectively) in training time over the comparison-onlycondition.To assess the effect of time on task and student aptitude

in determining student performance on the target synthesisproblem, we compared total score on the target synthesisproblem between the comparison and self-explanation andcomparison-only conditions using a general linear model,accounting for a main effect (condition) and two covariates(course grade and time on task). Outliers in time on taskwere removed based on inspection of the boxplot presentedin Fig. 10. Both course grade (partial η2 ¼ 0.14, p¼ 0.001)

and time on task (partial η2 ¼ 0.05, p ¼ 0.048) were foundto significantly predict student performance on the targetsynthesis problem. Condition was marginally significant(partial η2 ¼ 0.04, p ¼ 0.093). These results suggest thatstudent aptitude, as measured by course grade, is thestrongest predictor of subsequent performance on the targetsynthesis problem. Moreover, though the model lacked thestatistical power to observe the effect at the 0.05 level, itsuggests some evidence that combining analogical com-parison and summarization may provide a small, positiveeffect beyond that accounted only by the additional timeon task.

C. Discussion

Taken together, the results from experiment 3 supportthree broad conclusions. First, both analogical comparison

FIG. 8. Proportion of students in each condition demonstrating recognition of Faraday’s law, Ohm’s law, and magnetic force on thetarget synthesis problem. Error bars are standard errors.

FIG. 9. Proportion of students in each condition demonstratingthe correct general solution structure on the target problem. Errorbars are standard errors.

FIG. 10. Box plot of student time spent on training bycondition.


020112-15

and self-explanation were effective in improving studentperformance on the target synthesis problem versus control,though there were differences between analogical compari-son and self-explanation in the proportion of studentsemploying the correct solution structure on the targetproblem. Second, the combination of self-explanation andanalogical comparison was significantly more effective thananalogical comparison alone. Third, student annotations ofthe worked examples were accurate, but did not significantlyimprove student performance on the subsequent targetproblem relative to analogical comparison alone.While the mean score on the target synthesis problem for

the control condition in this study was 4.04=10, with only16% of students recognizing and applying Faraday’s law, themean score on the same target problem in experiment 2 was5.19=10, with 38% of students recognizing and applyingFaraday’s law. In order to provide context for these results inlight of previous findings, we note that the population sampleused in this study differed in two potentially important waysfrom the sample in experiment 2. First, though the twosamples were drawn from different semesters of the sameintroductory electromagnetics course, students in this studycompleted the course off sequence. Second, students in thisstudy completed the training task and target problem over athree week period near the end of the semester, approx-imately one month on average after course instruction onFaraday’s law. In contrast, students in experiment 2 com-pleted the training over a similar period that began closelyfollowing in-course instruction, and in proximity to an in-course exam on the relevant topics. Although we cannotexclude differences in the on- and off-course sequence,previous research tracking student understanding over theduration of an introductory course suggests timingdifferences between in-course instruction and administrationof the training and target synthesis problem are a potentialexplanation for the observed differences in baseline studentperformance between the two studies [50,51].This difference in baseline performance suggests an

important distinction when discussing the relative effec-tiveness of the analogical comparison and self-explanationconditions. This study found no significant differences ineither total score on the target synthesis problem or theproportion of students recognizing and applying Faraday’slaw between the self-explanation and analogical compari-son conditions. In contrast, experiment 2 found that theself-explanation condition resulted in significantly morestudents recognizing and applying Faraday’s law than inthe analogical comparison condition. Moreover, the overallproportion of students in the self-explanation conditionwho were able to construct the correct solution structurefor the target synthesis problem was considerably lower inexperiment 3 than in experiment 2.At the same time, these findings support the hypothesis

that there is a meaningful difference in how successfullystudents encoded the base worked examples between the

self-explanation and analogical comparison conditions: inparticular, the replicated finding that significantly morestudents in the self-explanation condition constructed thecorrect solution structure for the target synthesis problem thanin the analogical comparisoncondition.Moreover, the findingof no significant difference between analogical comparisonalone and analogical comparison with annotations suggeststhat these differences are likely not due to students simplyfailing to sufficiently read through the worked examples inthe comparison conditions. In other words, students in theannotation condition satisfactorily labeled physical conceptsand key steps within both worked examples with no differ-ence in student performance on the target synthesis problemcompared with just analogical comparison alone.As a result, it is more likely that the success of analogical

comparison of the two worked examples was limited bycognitive load and not a failure of students to appropriatelyattend to the task—here, and potentially in experiment 2.There are several additional pieces of evidence. First,student performance on the target problem once againindicated significant and persistent student difficulty withapplications of the single concepts—in particular, deter-mining physical directions associated with Lenz’s law andcross products. Unlike experiment 1 where students dem-onstrated a high degree of mastery of the two componentconcepts after recognizing the need for their simultaneousapplication, students continued to struggle with thesesingle-concept difficulties regardless of intervention.Second, there was a significant difference between ana-logical comparison alone and analogical comparison afterself-explanation, in terms of both total score and theproportion of students demonstrating the correct solutionstructure on the target synthesis problem, suggesting thatthe initial self-explanation did aid comparison.One limitation of this study is that it was unable to make

a definitive distinction between the value added by thecombination of analogical comparison and self-explanationand the corresponding additional time on task. Thislimitation was a consequence of constraints involvingthe administration of the task, available contact time,and number of participants. However, the overall signifi-cant difference between analogical comparison alone andthe combination of self-explanation and analogical com-parison is still of particular value, as it suggests at least onepedagogically relevant way to help students analyze sim-ilar, complicated synthesis problems. In other words, theadditional approximate 8 min to time on task from thecombined intervention was well spent, engaging, andresulted in significant gains on the target synthesis prob-lem; in contrast, the additional time spent required by thereading annotations was not inherently productive.

V. CONCLUSION

Taken as a whole, the three experiments demonstrate thatthe instructional methods of analogical comparison and


020112-16

self-explanation of worked examples can successfully beextended to improve student performance on target syn-thesis problems. As such, this work represents a novelcontribution to the study of these techniques beyondprevious work predominantly studying their applicationwith single-concept examples. Moreover, the results ofthese experiments suggest several principles regarding theconditions under which the two instructional methods arelikely to be effective.First, analogical comparison only resulted in significant

increases in student performance over baseline when thecomparisons were invoked between synthesis examples.There were no significant improvements when studentswere asked to compare corresponding sets of single-concept problems, despite the same prompts and surfacefeatures in the worked examples. This is particularlyimportant given the increase in the target synthesis problemcomplexity from experiment 1 to 2—in terms of both theconceptual difficulty represented by the involved conceptsand the requisite number of algebraic manipulations nec-essary for a complete solution. The finding that breakingthe base worked examples into component parts was notsignificantly different than either unguided problem solvingpractice (experiment 1) or course instruction alone (experi-ments 1 and 2) emphasizes the importance of the combinedstructure and joint application of concepts within synthesisproblems. Even with explicit and sequential comparisonsof the component parts, students cannot be expected tosuccessfully transfer those parts to a novel synthesisproblem without extensive efforts to explicitly scaffoldthe missing structure.Second, these results show that while analogical com-

parison and self-explanation of worked examples can beeffective in improving student conceptual recognition andthe use of the correct solution structure on a target synthesisproblem, pervasive difficulties associated with single-concept mastery—such as how to apply Lenz’s law—may not be as successfully remedied. In part, this suggeststhat synthesis problems and the instructional methodsused here can best be employed to help students explicitlypractice concept recognition, as opposed to the plug-and-chug heuristics often associated with single-concept prob-lems. Meanwhile, issues of single-concept mastery maybenefit from complementary and targeted practice in single-concept settings. As such, synthesis problems representanother type of tool for physics instructors, similar to otherclasses of physics problems like context-rich problems andjeopardy problems.Third, there is evidence that self-explanation and ana-

logical comparison can be combined for additional gains.Although it is not completely clear whether those gainsare solely due to increased time on task or represent anadditional inherent difference between the treatments, thisfinding is of pedagogical value. The addition of < 10 minof total time on task necessary for students to briefly

self-explain the individual worked examples before com-parison resulted in significant improvements in total scoreon the target synthesis problem.There are several important limitations to the studies

presented here. First, the three experiments studied syn-thesis problems involving only two sets of physics con-cepts. Further study with an increased variety of conceptsand combinations is necessary. As a corollary, the problemsstudied here share a subtle but important commonality: theyare both structured so that students can arrive at an answer,albeit incorrect, without considering at least one of thecomponent concepts. In experiment 1, students can forgo(and frequently did) the circular motion constraint andsubsequent application of centripetal acceleration if theynaively assumed that the velocity of the cart was zero at thetop of the loop. In experiments 2 and 3, students couldarrive at an answer by only considering the battery voltage.As such, students were not blocked from the final answeronly because of a missing unknown, but rather conceptualconsideration of the physical situation. It is possible that thelarge gains in concept recognition reflect this inability ofstudents to successfully rely only on an equation-huntingheuristic. Mathematically sequential synthesis problemsthat require students to use one concept to first solve foran unknown necessary for the application of the secondconcept may not represent as significant a challenge forstudents (cf. Ref. [19]).A second limitation of this series of experiments is that

they do not account for potential interactions with feedbackduring training. In particular, the effectiveness of analogicalcomparison might increase when students are providedimmediate feedback on their comparisons, either throughpeer-mediated feedback in a group work setting or viaindividualized tutoring in computer-based instruction. In asimilar vein, these experiments did not explicitly manipu-late the presentation of the worked example prompts andsolutions. It is possible that the inclusion of expertlikeexplanations or highlighted presentation of the workedexamples may help reduce cognitive load, and sub-sequently improve the effectiveness of analogical compari-son with more demanding worked examples. However,even without such additional support, the fact that theseinterventions proved successful in supporting studentperformance suggests that such methods may be productiveas part of individual assignments to scaffold studentproblem solving, particularly in large classes with limi-tations for one-on-one interaction with an instructor.A final key limitation is that our current study focused

on interventions to support student problem solving on atarget problem that shared the same physical concepts andgeneral solution structure as the provided worked exam-ples. Previous research on student categorization tasks withintroductory-level single-concept physics problems hasshown that students are much more likely to rely on suchsurface features in their classification of problem solving


020112-17

approaches compared to the principle-based methods oftenemployed by experts [52–55]. As such, one potentialcritique is that this study is merely documenting trivial,surface-feature transfer. That is, successful students maynot have created a deep underlying problem representationof the target problem, but instead superficially mapped theworked examples to the target synthesis problem basedonly on the similarity of surface features (i.e., in experiment1, all problems contained hills and circular loops).While this study was not directly designed to investigate

this possibility, there are a number of reasons to suspect thatstudent performance improvements were driven by morethan surface feature matching and direct replication of thework flow in the provided solutions. First, while we wantedto limit the confound of far transfer difficulties between theworked example and the target task, the experimental designincluded efforts to ensure that the target task was not toosimilar to the worked examples. For example, there aresignificant superficial differences between the diagrams andphysical context for theworked examples and the target tasks(see Figs 2 and 5), and the order of the work flow within theworked-example solutions is varied and different from thetypical solutions for the target tasks. Second, if success onthe target problem was primarily modulated by exposure tothe same collective bag of physics concepts, basic physicsequations, and other, problem-specific surface features (forexample, circular tracks and changes in height), one mighthave expected the single-concept analogical comparisons toalso result in significant improvements, since the solutionsto the single concept problems included the same conceptsand basic physics equations, just in different combinations.However, experiments 1 and 2 both found that comparisonsof single-concept worked examples did not significantlyimprove student performance relative to control. Third, forexperiment 2, only the self-explanation and most scaffoldedanalogical comparison condition significantly improvedstudent concept recognition and execution on the targetsynthesis problem—providing synthesis worked examplesand solutions was not enough to increase performancewithout additional scaffolding or an opportunity for studentsto elaborate on the structure of theworked examples. In otherwords, on this more complicated problem, only the twoconditions that most emphasized the underlying structure oftheworked exampleswere successful. To further support thisthird point, we discussed in Sec. III. C howmany students inthe self-explanation condition produced summaries thatwerericher than simple reproductions of solutions, and that thereare two proposed theoretical explanations for this, includingmental-model revision.At its core, the key instructional issue is that transfer with

single-concept problems is qualitatively different from trans-fer with synthesis problems. In this study, we find thatsynthesis brings a dimension of complexity to a transfer taskthat must be addressed. In particular, unlike single-conceptproblems where the goal is often to get students to use

problem cues to identify a particular principle for use (ratherthan match an equation to the corresponding variable set ofthe problem), synthesis problems have multiple, potentiallycompeting conceptual cues within a single-problem state-ment. As such, the interaction between these conceptual cuesand the identification and joint application of the concepts,can be relatively more complicated. This difficulty is likelyexacerbated by the fact that students are likely biased toconsider only a single concept given their previous exposureto more traditional physics problems.With these issues in mind, it remains an open question

to what extent the improvements in student performanceobserved here for these relatively isomorphic training andtarget examples will transfer to problems with less similarsurface features or altogether different combinations ofphysics concepts. However, there are reasons to be opti-mistic on both accounts. First, we note that the targetproblem in the second and third experiments—in part as aresult of its complexity—arguably differed more from itsworked example counterparts than the target problem usedin the first experiment. Whereas the first experimentconsisted of the same physical entities (hills and circularloops) repeated in the worked examples and target problem,the second and third experiments included more variedphysical situations representing different contexts forFaraday’s law (a circuit falling out of a field, a stationarycircuit in a changing field, and a moving rod). Althoughmore explicit scaffolding was necessary, students were stillable to successfully compare and transfer the training to thetarget problem. This may suggest that transfer to yet evenmore dissimilar problems and examples may be possiblewith appropriate scaffolding. As such, future work toexplore the effectiveness of these techniques as modulatedby transfer distance would be interesting and useful.The second reason for optimism is that there is evidence

that a primary benefit of these interventions for synthesisproblems is that they did in fact shift a significant numberstudents from applying only a single concept (or conceptsubset) to correctly applying the full set of required physicsconcepts. Although training on a single problem involvinga particular pair of physics concepts may be unlikely totransfer directly to another involving a completely differentcombination of concepts, it is possible that repeatedexposure to synthesis problems may represent a usefulway to help students to recognize deep problem structure,consider conceptual cues, and de-emphasize the use ofequation-hunting heuristics typically associated withsingle-concept end-of-chapter problems. As such, synthesisworked examples—combined with either analogical com-parisons or self-explanations—may represent a useful toolto help students develop complex problem solving skills.

ACKNOWLEDGMENTS

This work is supported by the National ScienceFoundation (Grant No. DRL—1252399).


020112-18

APPENDIX A: WORKED EXAMPLES ANDPROMPTS FOR EXPERIMENT 1

1. Synthesis worked examples

Problem 1.A block with massM ¼ 2.0 kg slides from ahorizontal surface into a vertical circular track with a radiusof R ¼ 3.0 m. Assume that friction between the block andthe track is negligible. What is the minimum speed theblock must have at the bottom of the loop that will permit itto slide all the way around the circular track without leavingthe track at the top?

Student solution 1At the top of the loop:

ðA1Þ

FNet ¼ Fg þ FN ¼ mac ðA2Þ

FNet ¼ mgþ 0 ¼ mv2

RðA3Þ

g ¼ v2

RðA4Þ

v ¼ffiffiffiffiffiffigR

pðA5Þ

1

2mvb2 ¼

1

2mv2 þmgh ðA6Þ

1

2mvb2 ¼

1

2mðgRÞ þmgð2RÞ ðA7Þ

vb2 ¼ gRþ 4gR ðA8Þ

vb ¼ffiffiffiffiffiffiffiffi5gR

p¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi5�9.81

ms2

�ð3.0 mÞ

rðA9Þ

vb ¼ 12.1 m=s ðA10Þ

Problem 2. You are tasked with designing arollercoaster. As part of the design, the track graduallydescends until it comes to a small semicircular hill ofheight R.You know that the speed of the rollercoaster cart will be

18.5 m=s when the cart is 20 m above the height of the

oncoming hill. What is the minimum possible hill heightfor which the cart does not leave the surface of the track atthe top? Ignore friction.

Student solution 2

1

2mvi2 þmgh ¼ 1

2mvf2 ðA11Þ

vf2 ¼ vi2 þ 2gh ðA12Þ

vf ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffið18.5 m=sÞ2 þ 2ð9.81 m=s2Þð20 mÞ

qðA13Þ

vf ¼ 27.1 m=s ðA14Þ

At the top of the hill∶ ðA15Þ

ðA16Þ

FNet ¼ FN − Fg ¼ − mac ðA17Þ

FNet ¼ 0 −mg ¼ −mvf2

RðA18Þ

R ¼ vf2

g¼ ð27.1 m=sÞ2

9.81 m=s2ðA19Þ

R ¼ 75 m ðA20Þ

2. Single-concept worked examples

Problem 1. A block with mass M ¼ 2.0 kg slidesaround a vertical circular track with a radius ofR ¼ 3.0 m. Assume that friction between the block andthe track is negligible. What is the minimum speed theblock must have at the top of the track in order to ensurethat it does not leave the track at the top?


020112-19

Student solution 1At the top of the loop:

ðA21Þ

FNet ¼ Fg þ FN ¼ mac ðA22Þ

FNet ¼ mgþ 0 ¼ mv2

RðA23Þ

g ¼ v2

RðA24Þ

v ¼ffiffiffiffiffiffigR

pðA25Þ

v ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffið9.8 m=s2Þð3.0 mÞ

qðA26Þ

vb ¼ 5.4 m=s ðA27Þ

Problem 2. You are tasked with designing a rollercoas-ter. As part of the design, the track includes a semi-circularhill of height R.

You know that the speed of the rollercoaster cart will be27 m=s at the top of the hill. What is the minimum possiblehill height for which the cart does not leave the surface ofthe track at the top? Ignore friction.Student solution 2At the top of the hill:

ðA28Þ

FNet ¼ FN − Fg ¼ −mac ðA29Þ

FNet ¼ 0 −mg ¼ −mvf2

RðA30Þ

R ¼ v2fg

ðA31Þ

R ¼ ð27 ms Þ2

9.81 ms2

ðA32Þ

R ¼ 74 m ðA33Þ

Problem 1.A block with massM ¼ 1.5 kg slides from ahorizontal surface into a circular track with a radius ofR ¼ 2.0 m. Assume that friction between the block and thetrack is negligible. If the speed of the block at the top ofthe track is 3.0 m=s, what was the speed of the block at thebottom before entering the loop?

Student solution 1

1

2mvb2 ¼

1

2mvt2 þmgh ðA34Þ

1

2mvb2 ¼

1

2mvt2 þmgð2RÞ ðA35Þ

vb2 ¼ vt2 þ 4gR ðA36Þ

vb ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi�3.0

ms

�2 þ 4ð9.81 m=s2Þð2.0 mÞ

rðA37Þ

vb ¼ 9.4 m=s ðA38Þ

Problem 2. You are tasked with designing a rollercoas-ter. As part of the design, the track gradually descends untilit comes to a small semicircular hill of height R.

You know that the speed of the rollercoaster cart will be15 m=s when the cart is 20 m above the height of theoncoming hill. What is the velocity at the top of theoncoming hill?Student solution 2

1

2mvi2 þmgh ¼ 1

2mvf2 ðA39Þ

vf2 ¼ vi2 þ 2gh ðA40Þ


020112-20

vf ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffið15 m=sÞ2 þ 2ð9.81 m=s2Þð20 mÞ

qðA41Þ

vf ¼ 24.8 m=s ðA42Þ

3. Mastery prompts (synthesis version)

Each of the following questions refers to the twodifferent problem-solution pairs on page 1.(A) Consider the diagram in line 1 of solution 1 and

line 6 of solution 2. Explain any similarities anddifferences between the identified forces in terms ofthe two physical situations.

(B) Consider line 3 of solution 1 and line 8 of solution 2.Is the direction of the mv2

R term the same or differentin the two solutions? Explain your reasoning.

(C) In Problem 1 line 3, the student substitutes in zerofor FN. Explain why.

Is the situation in problem 2 similar or different to problem1? Explain your reasoning.(D) Consider line 6 in solution 1 and line 1 in solution 2.

Identify the sources of energy considered in the twosolutions.Why is the term mgh on the left side of solution 2, but onthe right side in solution 1?Where did the student in solution 1 assign y ¼ 0? Explainyour reasoning.(E) A friend in an introductory physics class at another

institution asks for help in understanding and tack-ling similar problems. What will you tell your friendso that s/he can fully understand the problem?

4. Recognition prompts (synthesis version)

Each of the following questions refers to the twodifferent problem-solution pairs on page 1.(A) What are the main physical concept(s) used in both

of the students’ solutions?(B) For problem 1, identify the elements of the problem

statement or diagram which indicate the need touse each of the physical concept(s) you mentionedin Part A. List specific elements and explain yourreasoning.

(C) For problem 2, identify the elements of the problemstatement or diagram which indicate the need to useeach of the physical concept(s) you mentioned inPart A. List specific elements and explain yourreasoning.

(D) Are the elements you identified above similarbetween the two problems? Explain your answerhighlighting any differences and similarities.

(E) Another student compared these two problems andsolutions, and remarked:

“I get why the initial and final velocity in problem 2are both nonzero, but I don’t understand why the finalvelocity in problem 1 must be nonzero.”

• Using the physical concept(s) you identified in part A,what would you say to help this student?

• What elements of problem 1 indicate that the finalvelocity should be nonzero?

(F) A friend in an introductory physics class at anotherinstitution asks for help in understanding and tack-ling similar problems. What will you tell your friendso that s/he can fully understand the problem?

APPENDIX B: WORKED EXAMPLES ANDPROMPTS FOR EXPERIMENT 2

1. Synthesis worked examples

Problem I. A square circuit of length L ¼ 40 cm andunknown mass m is falling (gravity directed as shown)through a region with a uniform and perpendicular mag-netic field B ¼ 2.3 T directed out of the page. The resistorshave resistances of R1 ¼ 10 Ω and R2 ¼ 25 Ω, and thevoltage of the battery is 9 V. At the instant shown, whenpart of the circuit is still in the region with a magnetic field,the velocity of the object is constant and equal to 10.0 m=sdownward Find the mass of the circuit.

Student solution I

ðB1Þ

FNet ¼ FB − Fg ¼ ma ðB2Þ

ILB −mg ¼ 0 ðB3Þ

m ¼ ILBg

ðB4Þ

jεj ¼ dΦB

dtðB5Þ

jεj ¼ dðBAÞdt

¼ dðBLyÞdt

¼ BLdydt

¼ BLv ðB6Þ

ε ¼ BLv counterclockwise ðB7Þ


020112-21

XV ¼ ItotalRtotal ðB8Þ

Vþ ε ¼ IðR1 þ R2Þ ðB9Þ

I ¼ Vþ ε

R1 þ R2

ðB10Þ

m ¼ LBðVþ BLvÞgðR1 þ R2Þ

ðB11Þ

m¼ð0.4mÞð2.3TÞ½9Vþð2.3TÞð0.4mÞð10.0 ms Þ�

9.81 ms2ð35ΩÞ

ðB12Þ

m ¼ 0.049 kg ðB13Þ

Problem II. A square circuit of length L ¼ 3.0 m is in aregion with a changing magnetic field as shown. Themagnetic field is directed into the page, and varies withtime as BðtÞ ¼ Bot, where Bo ¼ 0.125 T=s. The batteryvoltage is 5 V, R1 ¼ 25 Ω and R2 ¼ 75 Ω. What is themagnitude and direction of the force on the top wire at timet ¼ 30.0 s?

Student solution II

jεj ¼ dΦB

dtðB14Þ

jεj ¼ dðBAÞdt

¼ L2dBdt

¼ L2B0 ðB15Þ

ε ¼ L2B0 counterclockwise ðB16ÞX

V ¼ ItotalRtotal ðB17Þ

V − ε ¼ IðR1 þ R2Þ ðB18Þ

I ¼ V − ε

R1 þ R2

ðB19Þ

At t ¼ 30.0∶ BðtÞ ¼ Bot → B ¼ 3.75 T ðB20Þ

jFj ¼ ILB ¼ ðV − L2B0ÞLBR1 þ R2

ðB21Þ

jFj ¼ ½5V − ð3.0 mÞ2ð0.125 TsÞ�ð3.0 mÞð3.75 TÞ

100 ΩðB22Þ

F ¼ 0.44 N towards the top of the page ðB23Þ

2. Single-concept worked examples

Problem I. A square loop of conducting wire, length40 cm is being pulled at a constant speed 10.0 m=s througha region with a uniform magnetic field B ¼ 2.3 T directedout of the page as shown. At the instant shown, when partof the circuit is still in the region with a magnetic field, findthe direction and magnitude of the induced emf in the loop.

Student solution I

jεj ¼ dΦB

dtðB24Þ

jεj ¼ dðBAÞdt

¼ dðBLyÞdt

¼ BLdydt

ðB25Þ

jεj ¼ BLv ðB26Þ

jεj ¼ ð2.3 TÞð0.4 mÞ�10.0

ms

�ðB27Þ

ε ¼ 9.2 Vcounterclockwise ðB28Þ

Problem II. A square loop of conducting wire, length3.00 m is in a region with a changing magnetic field asshown. The magnetic field is into the page and varies withtime as BðtÞ ¼ Bot, where Bo ¼ 0.125 T=s. What is themagnitude and direction of the induced emf in the loop?


020112-22

Student solution II

jεj ¼ dΦB

dtðB29Þ

jεj ¼ dðBAÞdt

¼ L2dBdt

ðB30Þ

jεj ¼ L2B0 ðB31Þ

jεj ¼ ð3.00 mÞ2�0.125

Ts

�ðB32Þ

ε ¼ 1.13 V counterclockwise ðB33Þ

Problem I. A current carrying wire of length L ¼ 40 cmis falling (gravity directed as shown) at a constant velocitythrough a region with a uniform and perpendicular mag-netic field B ¼ 2.0 T directed out of the page. If the currentis 0.5 A, find the mass of the wire.

Student solution I

ðB34Þ

FNet ¼ FB − Fg ¼ ma ðB35Þ

ILB −mg ¼ 0 ðB36Þ

m ¼ ILBg

ðB37Þ

m ¼ ð0.5 AÞð0.4 mÞð2.0 TÞ9.81 m=s2

¼ 0.041 kg ðB38Þ

Problem II. A square loop of conducting wire, lengthL ¼ 3.0 m is in a region with a uniform magnetic fieldB ¼ 1.2 T as shown. If there is a current I ¼ 0.25 A in theloop, what is the magnitude and direction of the force onthe top wire?

Student solution II

F ¼ ILB ðB39Þ

F ¼ ð0.25 AÞð3.0 mÞð1.2 TÞ ðB40Þ

F ¼ 0.9 N towards the top of the page ðB41Þ

Problem I. Given V1 ¼ 9 V, V2 ¼ 5 V, R1 ¼ 10 Ω andR2 ¼ 15 Ω, find the magnitude of the current in the circuitshown.

Student solution IX

V ¼ ItotalRtotal ðB42ÞV1 þ V2 ¼ IðR1 þ R2Þ ðB43Þ

I ¼ V1 þ V2

R1 þ R2

ðB44Þ

I ¼ 14

35A ¼ 0.4 A ðB45Þ

Problem II. There is a clockwise current I ¼ 0.04 Ain the circuit shown. Given V1 ¼ 5.0 V, R1 ¼ 25 Ω andR2 ¼ 75 Ω, find the voltage of the unknown battery V2.

Student solution IIX

V ¼ ItotalRtotal ðB46ÞV1 − V2 ¼ IðR1 þ R2Þ ðB47ÞV2 ¼ V1 − IðR1 þ R2Þ ðB48Þ

V2 ¼ 5 V − ð0.04 AÞð100 ΩÞ ðB49ÞV2 ¼ 1.0 V ðB50Þ


020112-23

3. Mastery prompts (synthesis version)

Each of the following questions refers to the twodifferent problem-solution pairs on page 1.(A) What physical quantity is calculated in line 6 in

solution I and line 2 in solution II?(B) Explain any similarities and differences between line

6 in solution I and line 2 in solution II. Hint: As partof your answer, explain the dy

dt term in problem I andthe dB

dt term in problem II in light of the two physicalsituations.

(C) Consider line 7 in solution I and line 3 in solution II.Explain why both have a counterclockwise direc-tion, even though the direction of the magnetic fieldis different in problem I and problem II.

(D) Consider line 9 in solution I and line 5 in solution II.Explain why solution I substitutes in Vþ ε whilesolution II subsitutes in V − ε.

(E) Why is the force due to the magnetic field on theloop in problem I directed towards the top of thepage (solution I line 1)? Explain your reasoning.

(F) The magnetic field in Problem II is in the oppositedirection of problem I. Why is the magnetic force onthe top wire in problem II also directed towards thetop of the page (solution II line 9)? Explain yourreasoning.

(G) A friend in an introductory physics class at anotherinstitution asks for help in understanding and tack-ling similar problems. What are the most importantideas or information that you can tell your friend sothat s/he can solve a similar problem?

4. Recognition prompts


of the students’ solutions?(B) For problem I, identify the elements of the problem

statement and/or diagram which indicate the need touse each of the physical concept(s) you mentioned inPart A. List the elements and explain your reasoningfor each of them.

(C) For problem II, identify the elements of the problemstatement and/or diagram which indicate the need touse each of the physical concept(s) you mentioned inPart A. List the elements and explain your reasoningfor each of them.

(D) Are the elements you identified in part B and Csimilar or different between the two problems?Explain your answer highlighting any similaritiesand/or differences.

(E) In problem I line 9, the student substitutes in for thetotal emf for both circuits. Explain why the student

includes more than just the battery voltage, usingthe physical concepts you identified in Part A. Inwhat ways is the situation in problem II line 5 similaror different from problem I line 9? Explain yourreasoning.

(F) A friend in an introductory physics class at anotherinstitution asks for help in understanding and tack-ling similar problems. What are the most importantideas or information that you can tell your friend sothat s/he can solve a similar problem?

5. Combined prompts


of the students’ solutions?(B) For problem I, identify the elements of the problem

statement and/or diagram which indicate the need touse each of the physical concept(s) you mentioned inPart A. List the elements and explain your reasoningfor each of them.

(C) For problem II, identify the elements of the problemstatement and/or diagram which indicate the need touse each of the physical concept(s) you mentioned inPart A. List the elements and explain your reasoningfor each of them.

(D) One of the important concepts in both problems isthat of an induced emf due to a changing magneticflux. Compare the physical reason for a changingmagnetic flux in each problem. Explain your answerhighlighting any differences between the twoproblems.

(E) A changing magnetic flux induces an emf. Explainany similarities and differences between line 6 insolution I and line 2 in solution II. Hint: As part ofyour answer, explain the dy


dt term in problem II in light of the two physicalsituations and your answers to part D.

(F) Consider line 7 in solution I and line 3 in solution II.Explain why both induced emfs have a counter-clockwise direction, even though the direction of themagnetic field is different in problem I and prob-lem II.

(G) The battery is not the only source driving the currentin both problems. Consider line 9 in solution I andline 5 in solution II. Explain why solution I sub-stitutes in Vþ ε while solution II subsitutesin V − ε.

(H) A friend in an introductory physics class at anotherinstitution asks for help in understanding and tack-ling similar problems. What will you tell your friendso that s/he can fully understand the problem?


020112-24

APPENDIX C: WORKED EXAMPLES AND PROMPTS FOR EXPERIMENT 3

1. Synthesis worked examples and annotation

2. Combined prompts

(A) What are the main physical concept(s) used in both of the students’ solutions?(B) Compare the main solution steps used to solve both problems. Explain in what ways they are similar and/or different.(C) One of the important concepts in both problems is that of an induced emf due to a changing magnetic flux. Compare

the physical reason for a changing magnetic flux in each problem. Explain your answer highlighting any differencesbetween the two problems.


020112-25

(D) A changing magnetic flux induces an emf. Explainany similarities and differences between line 6 inSolution I and line 2 in Solution II. Hint: As part ofyour answer, explain the dy


dt term in problem II in light of the two physicalsituations and your answers to part C.

(E) Consider line 7 in solution I and line 3 in solution II.Explain why both induced emfs have a counter-clockwise direction, even though the direction of themagnetic field is different in problem I and problemII. Note: It is not enough to only state the name of arule or physical law; you must explain and comparethe application of that rule or law in the two cases.

(F) The battery is not the only source driving the currentin both problems. Consider line 9 in solution I andline 5 in solution II. Explain why solution I substitutesin Vþ ε while solution II subsitutes in V − ε.

(G) Consider line 1 in solution I and line 10 in solutionII. Explain why the force of the magnetic field onboth current carrying wires is directed towards thetop of the page in both problems, even though thedirection of the magnetic field is different in problemI and problem II. Note: It is not enough to only statethe name of a rule or physical law; you must explainand compare the application of that rule or law inthe two cases.

[1] J. L. Docktor and J. P. Mestre, Synthesis of discipline-based education research in physics, Phys. Rev. ST Phys.Educ. Res. 10, 020119 (2014).

[2] L. Hsu, E. Brewe, T. M. Foster, and K. A. Harper, Resourceletter RPS-1: Research in problem solving, Am. J. Phys.72, 1147 (2004).

[3] D. P. Maloney, An overview of physics education researchon problem solving, in Getting Started in PER, edited byC. Henderson and K. A. Harper (American Association ofPhysics Teachers, College Park, MD, 2011), Reviews inPER Vol. 2.

[4] J. H. Larkin, Processing information for effective problemsolving, Eng. Educ. 70, 285 (1979).

[5] J. H. Larkin and F. Reif, Understanding and teachingproblem-solving in physics, Int. J. Sci. Educ. 1, 191 (1979).

[6] F. Reif and J. Heller, Knowledge structure and problemsolving in physics, Educ. Psychol. 17, 102 (1982).

[7] A. Van Heuvelen and D. P. Maloney, Playing physicsjeopardy, Am. J. Phys. 67, 252 (1999).

[8] P. T. Hardiman, R. Dufresne, and J. P. Mestre, The relationbetween problem categorization and problem solvingamong experts and novices, Memory Cogn. 17, 627 (1989).

[9] P. B. Kohl and N. D. Finkelstein, Patterns of multiplerepresentation use by experts and novices during physicsproblem solving, Phys. Rev. ST Phys. Educ. Res. 4,010111 (2008).

[10] D. E. Meltzer, Relation between students’ problem-solvingperformance and representational format, Am. J. Phys. 73,463 (2005).

[11] L. Ding, N. W. Reay, A. Lee, and L. Bao, Exploring therole of conceptual scaffolding in solving synthesis prob-lems, Phys. Rev. ST Phys. Educ. Res. 7, 020109 (2011).

[12] P. Heller, R. Keith, and S. Anderson, Teaching problemsolving through cooperative grouping. Part 1: Group versusindividual problem solving, Am. J. Phys. 60, 627(1992).

[13] P. Heller and M. Hollabaugh, Teaching problem solvingthrough cooperative grouping. Part 2: Designing problemsand structuring groups, Am. J. Phys. 60, 637 (1992).

[14] C. A. Ogilvie, Changes in students’ problem-solving strat-egies in a course that includes context-rich, multifacetedproblems, Phys. Rev. ST Phys. Educ. Res. 5, 020102 (2009).

[15] P. D. Antonenko, C. A. Ogilvie, D. S. Niederhauser, J.Jackman, P. Kumsaikaew, R. R. Marathe, and S. M. Ryan,Understanding student pathways in context-rich problems,Educ. Inf. Technol. 16, 323 (2011).

[16] Q. X. Ryan, E. Frodermann, K. Heller, L. Hsu, and A.Mason, Computer problem-solving coaches for introduc-tory physics: Design, and usability studies, Phys. Rev.Phys. Educ. Res. 12, 010105 (2016).

[17] L. Ding, N. W. Reay, A. Heckler, and L. Bao, Sustainedeffects of solving conceptually scaffolded synthesis prob-lems, AIP Conf. Proc. 1289, 133 (2010).

[18] L. Ding, N. W. Reay, A. Lee, and L. Bao, Using conceptualscaffolding to foster effective problem solving, AIP Conf.Proc. 1179, 129 (2009).

[19] B. Ibrahim, L. Ding, A. F. Heckler, D. R. White, andR. Badeau, Students’ conceptual performance on syn-thesis physics problems with varying mathematicalcomplexity, Phys. Rev. Phys. Educ. Res. 13, 010133(2017).

[20] D. R. White, R. Badeau, A. F. Heckler, and L. Ding,Bottlenecks in solving synthesis problems, in Proceedingsof the Physics Education Research Conference, Minne-apolis, MN, 2014, edited by P. V. Engelhardt, A. D.Churukian, and D. L. Jones (AIP, New York, 2015).

[21] D. E. Allen, B. J. Duch, and S. E. Groh, The power ofproblem-based learning in teaching introductory sciencecourses, New Directions for Teaching and Learning 1996,43 (1996).

[22] B. J. Duch, Writing problems for deeper understanding,in The Power of Problem-Based Learning: A Practical“How To” for Teaching Undergraduate Courses in anyDiscipline, edited by B. J. Duch, S. E. Groh, and D. E.Allen (Stylus Publishing Sterling, VA, 2001), p. 47.

[23] M. S. Sabella and E. F. Redish, Knowledge organizationand activation in physics problem solving, Am. J. Phys. 75,1017 (2007).


020112-26

https://doi.org/10.1103/PhysRevSTPER.10.020119


https://doi.org/10.1119/1.1763175

https://doi.org/10.1119/1.1763175

https://doi.org/10.1080/0140528790010208

https://doi.org/10.1080/00461528209529248

https://doi.org/10.1119/1.19233

https://doi.org/10.3758/BF03197085



https://doi.org/10.1119/1.1862636

https://doi.org/10.1119/1.1862636


https://doi.org/10.1119/1.17117

https://doi.org/10.1119/1.17117

https://doi.org/10.1119/1.17118


https://doi.org/10.1007/s10639-010-9132-x



https://doi.org/10.1063/1.3515179

https://doi.org/10.1063/1.3266695

https://doi.org/10.1063/1.3266695



https://doi.org/10.1002/tl.37219966808

https://doi.org/10.1002/tl.37219966808

https://doi.org/10.1119/1.2746359

https://doi.org/10.1119/1.2746359

[24] J. Sweller and G. A. Cooper, The use of worked examplesas a substitute for problem solving inlearning algebra,Cognit. Instr. 2, 59 (1985).

[25] X. Zhu and H. A. Simon, Learning mathematics fromexamples and by doing, Cognit. Instr. 4, 137 (1987).

[26] R. K. Atkinson, S. J. Derry, A. Renkl, and D. Wortham,Learning from examples: Instructional principles from theworked examples research, Rev. Educ. Res. 70, 181(2000).

[27] J. Sweller, Cognitive load during problem-solving: Effectson learning, Cogn. Sci. 12, 257 (1988).

[28] A. Renkl and R. K. Atkinson, An example order forcognitive skill acquisition, in In Order to Learn: Howthe Sequence of Topics Influences Learning, edited by F. E.Ritter, J. Nerb, E. Lehtinen, and T. M. O’Shea (OxfordUniversity Press New York, 2007), pp. 95–105.

[29] M. T. H. Chi, M. Bassok, and M. Lewis, Self-explanations:How students study and use examples in learning to solveproblems, Cogn. Sci. 13, 145 (1989).

[30] M. T. H. Chi, N. De Leeuw, M.-H. Chiu, and C. Lavancher,Eliciting Self-Explanations Improves Understanding,Cogn. Sci. 18, 439 (1994).

[31] M. T. H. Chi and K. VanLehn, The content of physics self-explanations, J. Learn. Sci. 1, 69 (1991).

[32] V. Aleven and K. Koedinger, An effective metacognitivestrategy: Learning by doing and explaining with a com-puter-based Cognitive Tutor, Cogn. Sci. 26, 147 (2002).

[33] K. VanLehn, R. Jones, and M. T. H. Chi, A model of theself-explanation effect, J. Learn. Sci. 2, 1 (1992).

[34] R. Catrambone and K. J. Holyoak, Overcoming contextuallimitations on problem-solving transfer, J. Exp. Psychol.Learn. Mem. Cognit. 15, 1147 (1989).

[35] D. Gentner, Structure mapping: A theoretical frameworkfor analogy, Cogn. Sci. 7, 155 (1983).

[36] D. Gentner, The mechanisms of analogical learning,Similarity and Analogical Reasoning, edited by S. Vosniadouand A. Ortony (Cambridge University Press London, 1989),pp. 199–241.

[37] D. Gentner and A. B. Markman, Structure mapping inanalogy and similarity, Am. Psychol. 52, 45 (1997).

[38] D. Gentner, J. Loewenstein, and L. Thompson, Learningand transfer: A general role for analogical encoding, J.Educ. Psychol. 95, 393 (2003).

[39] R. W. Ferguson and K. D. Forbus, Telling juxtapositions:Using repetition and align-able difference in diagramunderstanding, in Advances in Analogy Research, editedby K. Holyoak, D. Gentner, and B. Kokinov (NewBulgarian University Sofia, Bulgaria, 1998), pp. 109–117.

[40] D. Gentner and D. R. Gentner, Flowing waters or teemingcrowds: Mental models of electricity, in Mental Models,

edited by D. Gentner and A. L. Stevens (LawrenceErlbaum Hillsdale, NJ, 1983), pp. 99–129.

[41] D. E. Brown and J. Clement, Overcoming misconceptionsvia analogical reasoning: abstract transfer versus explan-antory model construction, Instr. Sci. 18, 237 (1989).

[42] J. Clement, Using bridging analogies and anchoringintuitions to deal with students’ preconceptions in physics,J. Res. Sci. Teach. 30, 1241 (1993).

[43] L. Ding, N. W. Reay, A. Lee, and L. Bao, Exploring therole of conceptual scaffolding in solving synthesis prob-lems, Phys. Rev. ST Phys. Educ. Res. 7, 020109 (2011).

[44] N. S. Podolefsky and N. D. Finkelstein, Use of analogy inlearning physics: The role of representations, Phys. Rev.ST Phys. Educ. Res. 2, 020101 (2006).

[45] N. S. Podolefsky and N. D. Finkelstein, Analogical scaf-folding and the learning of abstract ideas in physics: Anexample from electromagnetic waves, Phys. Rev. ST Phys.Educ. Res. 3, 010109 (2007).

[46] S.-Y. Lin and C. Singh, Using isomorphic problems tolearn introductory physics, Phys. Rev. ST Phys. Educ. Res.7, 020104 (2011).

[47] D. P. Maloney, O’T. L. Kuma, C. J. Hieggelke, and A. VanHeuvelen, Surveying students’ conceptual knowledge ofelectricity and magnetism, Am. J. Phys. 69, S12 (2001).

[48] K. Zuza, J.-M. Almudí, A. Leniz, and J. Guisasola,Addressing students’ difficulties with Faraday’s law: Aguided problem solving approach, Phys. Rev. ST Phys.Educ. Res. 10, 010122 (2014).

[49] M. Chi, Self-explaining expository texts: The dual proc-esses of generating inferences and repairing mental mod-els, Adv. Instr. Psychol. 5, 161 (2000).

[50] A. F. Heckler and E. C. Sayre, What happens between pre-and post-tests: Multiple measurements of student under-standing during an introductory physics course, Am. J.Phys. 78, 768 (2010).

[51] L. Ding, N. W. Reay, A. Lee, and L. Bao, Effects of testingconditions on conceptual survey results, Phys. Rev. STPhys. Educ. Res. 4, 010112 (2008).

[52] M. T. Chi, P. J. Feltovich, and R. Glaser, Categorizationand representation of physics problems by experts andnovices, Cogn. Sci. 5, 121 (1981).

[53] J. L. Docktor, J. P. Mestre, and B. H. Ross, Impact of ashort intervention on novices’ categorization criteria, Phys.Rev. ST Phys. Educ. Res. 8, 020102 (2012).

[54] A. Mason and C. Singh, Assessing expertise in introduc-tory physics using categorization task, Phys. Rev. ST Phys.Educ. Res. 7, 020110 (2011).

[55] E. R. Savelsbergh, T. de Jong, and M. G. Ferguson-Hessler,Choosing the right solution approach: The crucial role ofsituational knowledge in electricity and magnetism, Phys.Rev. ST Phys. Educ. Res. 7, 010103 (2011).


020112-27

https://doi.org/10.1207/s1532690xci0403_1

https://doi.org/10.3102/00346543070002181

https://doi.org/10.3102/00346543070002181

https://doi.org/10.1207/s15516709cog1202_4



https://doi.org/10.1207/s15327809jls0101_4


https://doi.org/10.1207/s15327809jls0201_1

https://doi.org/10.1037/0278-7393.15.6.1147

https://doi.org/10.1037/0278-7393.15.6.1147


https://doi.org/10.1037/0003-066X.52.1.45

https://doi.org/10.1037/0022-0663.95.2.393

https://doi.org/10.1037/0022-0663.95.2.393

https://doi.org/10.1007/BF00118013

https://doi.org/10.1002/tea.3660301007








https://doi.org/10.1119/1.1371296



https://doi.org/10.1119/1.3384261

https://doi.org/10.1119/1.3384261










What works with worked examples: Extending self ...heckler/Published papers...What works with worked...

Documents

Transcript of What works with worked examples: Extending self ...heckler/Published papers...What works with worked...