Nurse Education Today 34 (2014) 971–977

Contents lists available at ScienceDirect

Nurse Education Today

j ourna l homepage: www.e lsev ie r .com/nedt

Three heads are better than one: A mixed methods study examiningcollaborative versus traditional test-taking with nursing students

Donna Martin a,⁎, Eunice Friesen b, Antonina De Pau a

a Faculty of Nursing, University of Manitoba, Winnipeg, Canadab Centre for Advancement in Teaching/Learning, University of Manitoba, Winnipeg, Canada

⁎ Corresponding author at: Mailing address: Universityfor Nursing, 57 Curry Place,Winnipeg,Manitoba R3T 2N2,fax: +1 204 474 7682.

E-mail address: donna.martin@umanitoba.ca (D. Mart

0260-6917/$ – see front matter © 2014 Elsevier Ltd. All rihttp://dx.doi.org/10.1016/j.nedt.2014.01.004

s u m m a r y

a r t i c l e i n f o

Article history:Accepted 8 January 2014

Keywords:Educational assessmentCross-over studiesFocus groupsNursing students

Patient care requires a collaborative teamwork approach; therefore, nursing programs must provide studentswith ample opportunities to hone teamwork skills. Although collaborative testing (CT) has been utilized inhighereducation for decades, its uptake in nursing education has only recently surfaced. The purpose of this mixedmethods study was to examine CT versus traditional test-taking with undergraduate nursing students in aWestern Canadian university. Using a cross-over design, surveys, and focus groups, nursing students' experiencesof learning, retention of course content, and teamwork were explored. A convenience sample of 70 students wasrecruited. In this paper, we present the results along with recommendations for future research. The treatmenteffect was highly significant with CT scores being an average of 7.99 units higher than traditional test-takingscores when the test was allocated a total score of 100. However, the two formative tests were weighted mini-mally resulting in participating students' overall grades being increased by an average of 1.2 out of 100. Of prac-tical significance, students perceived that CT helped them understand the course material better and theyappreciated the opportunity to practice effective communication within a group to prepare for future practice.

© 2014 Elsevier Ltd. All rights reserved.

Introduction

Currently, nurse educators are being challenged to revise curriculaand pedagogy (Eggertson, 2013; Institute of Medicine, 2010). Sincethe provision of high quality nursing care in the practice setting is de-pendent upon strong collaboration between health care professionals,it is essential that nursing educational programs provide opportunitiesfor students to hone teamwork skills (Eggertson, 2013; Lusk andConklin, 2003; Pollard and Miers, 2008; Pollard et al., 2004; Sandahl,2009, 2010; Wiggs, 2011).

Collaborative testing (CT) is an educational strategy that engagesgroups of students in formative assessments designed to foster deeperlearning and refine students' teamwork skills (Bloom, 2009; Wiggs,2011). CT is also referred to as group testing group, cooperative testing,double testing, or dyad testing (Centrella-Nigro, 2012). Students workin teams while taking a formative examination to facilitate a group dis-cussion about the exam questions and formulate appropriate answers;hence, they work together towards a common goal (Keselyak et al.,2009; Wiggs, 2011). “An appealing aspect of this type of testing isthat it potentially transforms the necessary process of assessment intoa greater opportunity for learning” (Pandey and Kapitanoff, 2011,

ofManitoba, Helen Glass CentreCanada. Tel.: +1 204 474 6716;

in).

ghts reserved.

p. 163). CT may be an effective strategy to foster teamwork skillsamongst nursing students.

In this paper, results are presented from amixedmethods study thatutilized a cross-over design, survey and focus groups to examine CT ver-sus traditional test-taking (TT). The research site was a four-year bacca-laureate nursing program located in a Western Canadian university.Based upon the study's findings, we provide recommendations aboutusing CT and identify the need for future research.

Background

Sandahl (2009) identified that a large body of research supported CTacross disciplines. Since 1985, CT has been implemented and evaluatedin higher education in various programs such as biology, chiropracticmedicine, dental hygiene, physiology, psychology, medicine and sociol-ogy (Cortright et al., 2005; Cortright et al., 2003; Durrant et al., 1985;Giuliodori et al., 2008, 2009; Haberyan and Barnett, 2010; Leight et al.,2012; Meseke et al., 2009; Rao et al., 2002; Slusser and Erickson, 2006;Zipp, 2007).

Johnson et al. (1998) conducted a meta-analysis of educational re-search about cooperative learning, locating 305 studies that comparedefficacy of cooperative, competitive and individualistic learning onstudent achievement in adult learners. They classified the evidence ac-cording to academic success, quality of relationships, and psychologicaladjustment to college life. Cooperative learning promoted higher indi-vidual student achievement than competitive and individualistic

Fig. 1. Cross-over design used in CT versus TT.

972 D. Martin et al. / Nurse Education Today 34 (2014) 971–977

approaches with significant effect sizes. Cooperative learning enhancedstudents' relationships with peers and faculty and tended to promotehigher self-esteem and more socially skilled students.

Heglund andWink (2011) argued that research on CT has primarilyexamined two main outcomes: student and faculty perceptions of CTand the impact of CT on grades. Student and faculty's perceptions ofCT have been consistently positive (Cortright et al., 2003; Gallagher,2009; Giuliodori et al., 2008, 2009; Haberyan and Barnett, 2010;Hickey, 2006; Jones and Lishman, 2011; Lusk and Conklin, 2003; Raoet al., 2002; Sandahl, 2009, 2010). The impact of CT on course gradeshas been inconsistent— ranging from no impact to a statistically signif-icant improvement (Haberyan and Barnett, 2010; Heglund and Wink,2011; Rao and DiCarlo, 2000; Zimbardo et al., 2003).

Another outcome of CTwas determined to be the retention of coursematerial. Some researchers identified better learning associatedwith CT(Bloom, 2009; Cortright et al., 2003; Jones and Lishman, 2011), althoughother researchers reported that improved performance did not carryover to final examinations (Leight et al., 2012; Meseke et al., 2009;Molsbee, 2013; Slusser and Erickson, 2006; Sandahl, 2010; Wiggs,2011;Woody et al., 2008). Other outcomes associated with CTwere en-hanced critical thinking (Gallagher, 2009), less anxiety (Kapitanoff,2009; Keselyak et al., 2009; Haberyan and Barnett, 2010; Mitchell andMelton, 2003; Pandey and Kapitanoff, 2011), improved peer relations(Heglund and Wink, 2011; Kapitanoff, 2009; Pandey and Kapitanoff,2011; Sandahl, 2010) and increased motivation to learn (Slusser andErickson, 2006).

Since CT “has been used in a number of different formats and designsfor many years” (Jones and Lishman, 2011, p. 176), there is difficulty indiscerning the evidence (Pandey and Kapitanoff, 2011). Most researchhas been descriptive in nature with some studies using a quasi-experimental design (Sandahl, 2010). Group composition has varied insize from pairs to larger numbers of students (Pandey and Kapitanoff,2011). In some studies, CT groups were assigned by the instructor(Mitchell and Melton, 2003; Molsbee, 2013), while other researchersallowed students to choose their own groups (Zimbardo et al., 2003).Another variation involved the submission of answer sheets. Some re-searchers allowed students to submit separate answer sheets and didnot require a group consensus (Breedlove et al., 2004; Lusk andConklin, 2003; Sandahl, 2010; Wiggs, 2011), while others established aprotocol that CT groups submit one answer sheet for the group (Duaneand Satre, 2013).

The uptake and evaluation of CT by nurse educators have onlyrecently surfaced (Centrella-Nigro, 2012; Duane and Satre, 2013;Heglund and Wink, 2011; Hickey, 2006; Jones and Lishman, 2011;Lusk and Conklin, 2003; Mitchell and Melton, 2003; Molsbee, 2013;Sandahl, 2009, 2010; Wiggs, 2011). Similar to Sandahl's (2010) study,we discerned that a quasi-experimental study using a cross-over designwith cross-sectional surveys and focus groups was warranted to exam-ine CT versus TT.

This study was guided by three inter-related frameworks: social in-terdependence theory, cognitive elaboration perspective, and neurosci-ence. Social interdependence theory suggests that the way in whichgroup goals are structured determine how individuals interact andthat interaction pattern subsequently creates outcomes (Johnson et al.,1998). The practical application of social interdependence theory is co-operative and collaborative learning. Outcomes of cooperative and col-laborative learning can be divided into three categories. Firstly, thereis an improvement in individual's effort to achieve using higher levelcognitive and moral reasoning strategies (Johnson, 2003). Secondly,there is an increase in perceived peer support (Johnson, 2003). Thirdly,“cooperative experiences tend to be related to beliefs that one is intrin-sically worthwhile, others see one in positive ways, one's attributescompare favorably with those of one's peers, and one is a capable, com-petent, and successful person” (Johnson, 2003, p. 938).

The second guiding framework was the cognitive elaboration per-spective, which contends that peers learn better together (Ashman

and Gillies, 2003). Students learn in a context whereby more effectivelearning processes are used. Working in a group allows individuals toquestion each other's thinking thereby promoting metacognition.Group members are able to assist each other to maintain focus on thetask. This perspective contends that group learning promotes studentachievement, critical thinking skills, and transfer of learning (Brandonand Hollingshead, 1999; Ashman and Gillies, 2003). Furthermore,group learning aids in the development of social skills such as commu-nication, presentation of ideas, problem-solving, leadership, delegationof tasks and organizational skills (Butcher et al., 1995).

The third guiding framework is neuroscience. Neurobiological re-search has determined that social learning interactions “are capable of al-tering the way the synaptic connections in our brain cells grow andconnect to other brain cells” (Caine et al., 2009, p. 60). During grouplearning, complex and intricate cognitive processes are stimulated.Group learning develops areas of the brain located primarily in the pre-frontal cortex, which is responsible for executive functions includingthe ability to plan and organize thinking, make sense of ideas and behav-ior, multitask, moderate emotions, critically think and reflect. These neu-robiological changes lead to better learning and enhanced memory(Carpenter, 2012; Damasio, 2005; Immordino-Yang andDamasio, 2007).

Methods

A mixed methods study, consisting of a cross-over design (seeFig. 1), surveys, and focus groups, examined nursing students' experi-ences related to teamwork and learning (i.e. retention of knowledge)with CT. The research questionswere: (a) DoCT and TT approaches pro-duce statistically significant differences in formative test scores amongstgroups of undergraduate nursing students?; (b) Does CT enhance stu-dents' retention of course material?; and (c) What are the educationaland group process experiences of undergraduate nursing studentswhen CT is implemented?

Sampling

Following approval from the university's ethical review board, stu-dents were recruited from two sections that were taught by the sameinstructor with the same learning resources and instructional tech-niques in the same term. All students received an email invitation toparticipate. The research assistant (RA) visited the first class to describethe purpose of the research and its design. Students who had previouslytaken the course were not eligible to participate. A biostatistician wasconsulted prior to implementing the study and it was determined thata minimal sample size of 50 would be required.

973D. Martin et al. / Nurse Education Today 34 (2014) 971–977

Data Collection Procedure

Participating students completed two formative tests and a summa-tive test. One formative testwas in a CT format and the other used the TTapproach. All students (study subjects and non-participants) wrote thesummative test in the TT format. Students that chose not to participatein the study wrote all tests individually.

Recruitment of subjects, data collection and analysis procedures en-sured that the instructor was “blind” to the identification of studentswho were participating. Following the acquisition of informed consent,the RA collected demographic data using a short, anonymous survey.Shortly thereafter, the RA randomly assigned participants in eachsection into Group A or B by drawing their student numbers at random;the first student number was assigned to Group A, the second wasallocated to Group B, and so on. Group A completed the first formativetest using CT and the second formative test using TTwhile Group B com-pleted the first formative test using TT and the second using CT (seeFig. 1). The two groups wrote their formative tests in two separaterooms — one room for the CT group and one for those completingthe formative test in the TT format. Tests were invigilated by membersof the research team to ensure that the instructor was unaware ofparticipants' identities.

Immediately prior to CT, the RA randomly assigned participants intosmall groups of three, similar to the format utilized by Sandahl (2010).The first three names drawn were assigned to the first small group, thenext three names to the second group, and so on. The formative testswere comprised of 45 test questionsworth 15% towards thefinal coursegrade. The majority of questions were posed with multiple choice for-mats along with several short answer questions. According to Wiggs(2011), to accomplish a 50-item examination, 2 min per item shouldbe allotted rather than the traditional 1min per test question. Therefore,rather than 50min for a 50-item test, 110 min would be a more appro-priate time frame to foster discussion, information-sharing, negotiation,and collaboration. Two hours were allotted for ample opportunities tocomplete the test as each formative test was a 45-item test. Eachsmall group submitted one answer sheet as this procedure fit moreaptly with the realm of teamwork (Duane and Satre, 2013).

Upon completion of CT, the RA collected anonymous responses fromparticipants through the administration of a CT evaluation form, whichincluded 14 items rated on a five-point Likert scale (see Table 1). Thisform also provided space for additional comments. Students thatchose not to participate in the study wrote both formative tests in theTT format and their test marks were excluded from the study's data.

In the final class, the RA provided participants with a $10 gift card tothe university bookstore. The RA arranged a date, time, and location fortwo focus groups (one for each section). One focus group had three

Table 1Collaborative Testing Evaluation Form.

Do not write your name on this survey.

Survey item Strongly

I liked collaborative testing.Collaborative testing encouraged me to study.I feel that collaborative testing helped me understand the material better.I feel that collaborative testing helped me to critically think.I feel that collaborative testing helped me develop communication skills.Collaborative testing allowed me to study less.I feel collaboration is an important skill for nurses.Collaborative testing allowed me to feel confident in the knowledge that I hadabout a certain subject so I could explain it to my peers.

Members of my collaborative testing group stayed on task.Members of my group encouraged all group members to contribute.Members of my group contributed ideas that helped the group.Members of my group came prepared for the test.Members of my group listened respectfully to other group members' ideas.Members of my group asked group members for clarification.Additional comments about collaborative testing:

participants and the second was comprised of nine students. The co-investigator facilitated each one-hour audio-taped focus group using asemi-structured interview guide (see Table 2) and the RA served asthe recorder. At the completion of the interview, participants were pro-vided with another $10 gift card to the bookstore. A transcriptionisttranscribed all audiotapes verbatim and the RA removed all identifyinginformation from the transcripts.

Data Analysis Procedures

Students' demographics and CT evaluation scores were uploadedinto a quantitative data software program to generate descriptive statis-tics. Participants' test scores were de-identified by the RA using subjectnumbers (1, 2, 3, etc.) and transferred into an Excel spreadsheet anduploaded into SAS Version 9.3 for inferential statistical analysis. Qualita-tive data, including students' comments on the CT evaluation tool, tran-scripts from both focus groups, and researchers' field notes, wereuploaded into NVivo9 and two research teammembers independently,and then collaboratively, analyzed this qualitative data to identify codes,categories, and themes using a process described by Creswell (2013).

To answer research question #1, a crossover analysis using SASVersion 9.3 was implemented. To answer research question #2, severalchallenging questions were identified from the results of the formativetests. On the summative test, these challenging questions were posedagain. An Excel spreadsheet was formulated that recorded students' re-sponses to the challenging questions on the formative test (CT, TT) andthe summative test with notations about the testing format. Chi-squaretests were conducted to determine if the proportion of correct re-sponses was the same between groups (CT, TT, final test). To addressresearch question #3, we amalgamated the responses of the CT evalua-tion formwith textual data from the transcripts andfield notes. By read-ing and rereading textual data and identifying codes, clustering codesinto categories, and then clustering the categories into a major theme,students' experiences with CT were described.

Results

Description of the Sample

Initially, 74 (out of 98 enrolled students) volunteered to participate.Four students withdrew from the study prior to data collection, yieldinga convenience sample of 70 students (36 from one section and 34 fromthe other section). Approximately 71% of enrolled students participatedin the study. Most participants were female (60 females, 8 males, and 2individuals who did not specify), under 30 years of age with 62.9% be-tween 20 and 24 years of age and 21.4% between 25 and 29 years.

agree Agree I don't agree or disagree Disagree Strongly disagree

Table 2Semi-structured interview guide — Focus group.

1. First of all, please tell me why you decided to participate in this study.2. Why do you think that collaborative testing format was offered in this course?3. What is the purpose of testing?4. How did you feel about the testing format?5. How did the testing format influence your studying?6. How did the testing format influence your abilities to express your ideas?7. What were the benefits/pitfalls for you participating in this study?8. Tell me how you reacted to the random assignment of you to a group with

two other students.9. Please share a story about the group's method of choosing one best answer.10. Did your group designate a leader?11. If so, how did this designation occur?12. How did you decide what answers to choose?13. What happened when there was no consensus on the right answer?14. How was consensus achieved?15. Is there anything else that you would like to share?

Table 4Percentage of correct and incorrect responses to a test question on the formativeand summative tests. Chi-square analysis was performed with no statistically significantdifference.

Formative Summative Formative Summative

Testing type Wrong answer Correct answer Total

CT 10 17 21 14 3132.26% 58.84% 67.74% 45.16%

Traditional 21 12 15 24 3658.33% 33.33% 41.67% 66.67%

Total 29 38 67

974 D. Martin et al. / Nurse Education Today 34 (2014) 971–977

Almost 83% of participants had completed two years of course work innursing. Although themajority of participants had no prior experienceswith CT, 70% identified that they had worked on collaborative assign-ments with other students, providing examples such as group papersand group presentations. The two sections were similar in all measuredcategories. Although the demographic survey did not gather informa-tion about students' cultural identities, we observed that this studentpopulation was culturally diverse.

Test Scores

Using a cross-over analysis and SAS Version 9.3, the treatment effect(CT) was highly significant, with CT scores being an average of 7.99units higher than formative scores (5.21–10.77, p value b 0.0001)when the testwas allocated a total score of 100 (see Table 3). The periodeffect was not significant (p value of 0.9721), indicating that studentsdid not perform better in the second formative test. With the two for-mative tests each allocated a total score of 15, participants' overallgrades increased by an average of 1.2 points out of 100. From the CTEvaluation Survey responses, most students perceived that their gradeswere enhanced by the CT process.

Retention of Course Material

Following each formative test, students were provided with the an-swer key. To determine if students retained information over the dura-tion of the course, challenging questions (i.e. poor rate of correctresponses) were identified from each of the two formative tests andposed again on the summative test. If students learned from theirprior mistakes, the researchers hypothesized that the proportion of cor-rect responses would increase by at least 5% in the final test. For exam-ple, the correct response rate for one question on a formative test was67.74% (CT) and 61.67% (TT). However, the correct response rate forthat same question dropped to 45.16% on the final test for students inCT. In contrast, the response rate was slightly elevated on the final testfor students that were asked that question during TT (see Table 4).Chi-square analysis indicated that there was no statistically significantdifference between CT, TT, and final test groups in the proportion of cor-rect responses on all repeated questions. Based upon this statisticalfind-ing, CT failed to enhanced retention of course material.

Table 3Comparison of test scores from CT and traditional testing (out of 100).

Variable Mean Median

CT score 88.6176812 89.9000000Traditional Test score 80.6269565 80.6700000

Students' Experiences: “Three Heads are Better Than One”

The main theme, “three heads are better than one,” emerged fromthe qualitative data. This theme was comprised of five inter-relatedcategories: (a) studying more — studying differently; (b) cognitivecollectivism; (c) “it stuck in my head better”; (d) confidence; and(e) practicing how to share knowledge and negotiate.

Studying More — Studying DifferentlyApproximately three quarters of participants indicated that CT en-

couraged them to study, while 20% did not agree or disagree as towhether or not CT encouraged them to study for the formative test.Most students shared that they studied differently for the CT experienceas they perceived that they were now responsible to others.

Because considering thatwe need it to discusswith other people, I didn'twant to look like a fool because I didn't know something or if somebodyelse was counting on me to know something, which is the real worldand in reality, people expect you to know things and they expect youto answer questions and know your stuff. I think it's more of a realisticscenario to take a test with other people and showothers that you knowyour stuff rather than sitting in front of a piece of paper and showing thepiece of paper that you know something.

Not only were the students trying to learn new material, but theyalso had to ponder how they could present the content to others. Thefollowing quote demonstrates how the student was contemplatingways to share knowledge and positively influence others, while prepar-ing for the test.

I felt that [CT] increased the stress to study harder… I felt that I had tocome prepared to argue for my answer that I wanted to choose, whichmight make me a control freak … I had to study harder so that I hadthe words and the vocabulary to argue for my answer.

Participants quickly adapted to the notion that their classmateswererelyingupon their knowledge and their abilities to effectively communi-cate that knowledge. One student related this notion to nursingpractice,which requires effective communication techniques.

I feel like when others are dependent on me, I would want to be able toprovide the answer … I just think that I wanted, not wanting to disap-point my group.

Minimum Maximum Standard deviation

66.7000000 97.3300000 5.6444629253.3000000 100.000000 8.9149030

975D. Martin et al. / Nurse Education Today 34 (2014) 971–977

Cognitive CollectivismParticipants shared their experiences of having a newfound trust in

their peers, arising from the need to rely upon fellow students to con-tribute to their learning. Interdependence and cognitive teamworkwere identified as new experiences.

With this experience, I learned that I can trust others with, youknow, the knowledge that they bring to the table and not feel like theresponsibility is all on me and I, you know, need to come through forthe group. I realized that there's people who are more capable of,definitely more capable than I, with the material that was presented.So that was, I mean, an opportunity that just helped me build my truston others.

Most students shared that they were typically focused on achievinghigh grades. However, participants expressed that CT changed theirfocus as they were more concerned about promoting group harmonyand ensuring respectful communication.

I think I was more concerned that everybody felt included and that theyhad their say than the mark itself.

It Stuck in my Head BetterApproximately 79% of students agreed (38.6%) or strongly agreed

(40%) that CT helped them understand the course material better.Eighty percent of participants reported that CT assisted them in the pro-cess of critical thinking. The following quote represents a participant'sinsights into how CT enhanced their retention of course material:

I feel like I learn a lot better when I discuss things or talk them out thenthey stick in my head better.

ConfidenceApproximately 83% of participants indicated that CT allowed them to

feel confidentwith respect to their own knowledge. Participants sharedthat the CT format facilitated affirmation of their knowledge, which in-creased their confidence. One participant stated:

It strengthens your knowledge coming out [of the test] that you felt goodthat you knew yourmaterial down pat because someone else knew thattoo.

Practicing How to Share Knowledge and NegotiateParticipants shared that they negotiated the correct answers with

their group members using one of two methods. Some groups decidedto go through the test andmark answers individually and then compare,contrast, and discuss the best answer for each question. Other groupsappointed onemember to lead them through each test question, discussthe best answer, seek consensus, and then proceed to the next question.If consensus was not achieved, the group revisited the question againlater.

In my group, I think we're probably around 3/4 of the questions we allhad the exact same answer. We all agreed. And then for some of them,we had to convince others. So wewould share our ideas and if someonehad good reasoning for it then that was usually good enough to con-vince the others in the group.

Nearly 79% of participants indicated that CT helped them developstronger communication skills.

You need to understand the materials in order to communicate and youneed to be able to communicate that. So in away, it's a good strategy for

team working to be able to get everybody's input. Take it all apart andput it all back together and make sure that you have the right answeron the paper at least in the end.

Approximately 93% of participants reported that members of the CTgroup stayed on task throughout the test and 90% found that all mem-bers of the groupwere encouraged to contribute equally. Almost all par-ticipants found that everyone was well prepared for the test. Ninety-three percent reported that their peers listened respectfully to allgroup members' ideas while 90% reported that group members askedeach other for clarification during the CT process. Ninety-four percentof participants perceived that collaboration is an important skill fornurses. The following quote illustrates how a student believed CTwould assist them in their future nursing practice:

I think that CT would get people to really develop their communicationskills and social skills and bedside manner, which is extremelyimportant.

Discussion

Strengths of this study included the mixed methods approach,which allowed for a cross-over design and multiple data sources. By in-tegrating three guiding frameworks, the study was grounded in educa-tional theories. An additional strength was the sample size of 70 (pre-determined n of 50was identified by a biostatistician as being appropri-ate for this study).

The study had several limitations. Firstly, the method in which re-tention of course material was evaluated was inappropriate. In hind-sight, repeating challenging questions to determine learning lacked arationale and did not stem from the grounding frameworks. It wouldbe more appropriate to evaluate retention of coursematerial by repeat-ing questions on core concepts (throughout the course) by posing thesame questions that reflected significant points on both formativetests and then the summative test. One would anticipate that theproportion of correct responses would increase over time and be statis-tically significant. More appropriate methods need to be identified as tohow to measure retention of course material. Furthermore, in keepingwith the guiding frameworks, a formal test review may have facilitatedstudents' understanding of the course material with classroom discus-sions rather than simply providing an answer key. Secondly, there wasup to 13% of “missing” data to answer research question #2,which com-promised the accuracy of the analysis. Thirdly, the first focus group wascomprised of three participants and this size may not have accuratelydepicted how that class experienced CT. Krueger (2006) identifiedthat the size of a focus group can range from as small as four to anupper limit of 12 people with a key feature of an academic focusgroup being that it includes five to eight members. Perhaps, initiatingrecruitment of participants into the focus groups earlier with a greaterincentive may have boosted the number of participants in the firstfocus group. Focus groups were scheduled towards the end of theterm and students may have been overwhelmed with the impendingfinal exam, resulting in recruitment difficulties. Scheduling the focusgroups during the second last week of the termmay have enhanced re-cruitment. Despite the low number of participants in the first focusgroup, the second focus group yielded an ample number of members,according to Krueger (2006). Although the number of focus groupmembers ranged from three to 12, rich data was gathered and similarthemes emerged from both transcripts. Furthermore, qualitative datawas collected on the CT Evaluation Survey and incorporated into theanalysis.

Similar to findings from recent studies about CT in nursing education(Sandahl, 2010; Wiggs, 2011), CT was an effective learning strategy,particularly when the formative tests were weighted appropriatelyand written in a multiple choice format. The differences between CT

976 D. Martin et al. / Nurse Education Today 34 (2014) 971–977

and TT scores were statistically significant, supporting the participants'perceptions that “three heads are better than one.” Because the two for-mative tests were eachweighted 15 out of 100, the overall course gradewas elevated by amean of 1.2 out of 100with participation in CT. Gradeinflation was tempered by this weighting.

This sample indicated that they were more confident in theirlearning and they appreciated the opportunity to hone their team-work skills through effective communication and negotiation.Future research, similar to Pollard and Miers' (2008) work in theUnited Kingdom, is warranted to test the longer term retention ofcourse material with the implementation of CT. Future researchabout the effectiveness of CT with culturally diverse studentswould be beneficial.

Conclusion

To facilitate an education outcome of providing safe and competentnursing care in practice settings, it is essential that nursing educationalprograms provide ample opportunity for students to hone teamworkskills. Our findings indicated that “three heads are better than one” asCT produced statistically significant higher test scores. Because forma-tive tests were weighted appropriately, students participating in theCT process scored an average of 1.2 out of 100 higher than the traditionaltesting group in their final course grades. In this study, inappropriatemethods were utilized to test retention of course material; therefore,CT failed to enhance students' long-term retention of course materialfrom a statistically significant standpoint. Perhaps of more practical sig-nificance is the finding that CT provided undergraduate nursing stu-dents with an opportunity to practice effective communication withina group. These students perceived that “three heads are better thanone.”

Acknowledgments

This study was funded by the Professional Foundations ResearchAward provided by the Manitoba Centre for Nursing & Health Researchvia the Nursing Endowment Fund, University of Manitoba. The authorswish to thank the nursing students who volunteered to share theirexperiences with collaborative testing.

References

Ashman, R., Gillies, A., 2003. Co-operative Learning: The Social and Intellectual Outcomesof Learning in Groups. RoutledgeFalmer, New York.

Bloom, D., 2009. Collaborative test taking: benefits for learning and retention. Coll. Teach.57 (4), 216–220. http://dx.doi.org/10.1080/87567550903218646.

Brandon, D.P., Hollingshead, A.B., 1999. Collaborative learning and computer‐supported groups. Commun. Educ. 48 (2), 109–126. http://dx.doi.org/10.1080/03634529909379159.

Breedlove, W., Burkett, T., Winfield, I., 2004. Collaborative testing and test performance.Acad. Exch. Q. 8 (3), 36–40 (Retrieved from http://www.rapidintellect.com/AEQweb/mo2609j4.htm).

Butcher, A.C., Stefani, L.A.J., Tariq, V.N., 1995. Analysis of peer‐, self‐ and staff‐assessment ingroup project work. Assess. Educ. Princ. Policy Pract. 2 (2), 165–185. http://dx.doi.org/10.1080/0969594950020204.

Caine, R., Caine, G., McClintic, C., Klimek, K., 2009. 12 Mind/Brain Learning Principles inAction. Corwin Press, Thousand Oaks, CA.

Carpenter, S.K., 2012. Testing enhances the transfer of learning. Curr. Dir. Psychol. Sci. 21(5), 279–283. http://dx.doi.org/10.1177/0963721412452728.

Centrella-Nigro, A., 2012. Collaborative testing as posttest review. Nurs. Educ. Perspect. 33(5), 340–341.

Cortright, R., Collins, H., Rodenbaugh, D., DiCarlo, S., 2003. Student retention of coursecontent is improved by collaborative-group testing. Adv. Physiol. Educ. 27 (3),102–108. http://dx.doi.org/10.1152/advan.00041.2002.

Cortright, R., Collins, H., DiCarlo, S., 2005. Peer instruction enhanced meaningful learning:ability to solve novel problems. Adv. Physiol. Educ. 29 (2), 107–111. http://dx.doi.org/10.1152/advan.00060.2004.

Creswell, J., 2013. Qualitative Design and Inquiry: Choosing Among the Five Approaches,3rd ed. Sage, Thousand Oaks, CA.

Damasio, A., 2005. Descartes' Error: Emotion, Reason, and the Human Brain. PenguinBooks, New York.

Duane, B., Satre, M., 2013. Utilizing constructivism learning theory in collaborativetesting as a creative strategy to promote essential nursing skills. Nurse Educ. Today.http://dx.doi.org/10.1016/j.nedt.2013.03.005 (Retrieved from).

Durrant, L., Pierson, G., Allen, E., 1985. Group testing and its effectiveness in learning se-lected nursing concepts. J. R. Soc. Promot. Heal. 105 (3), 107–111. http://dx.doi.org/10.1177/146642408510500306.

Eggertson, L., 2013. The gap between clinical practice and education. Can. Nurse 109 (7),23–26 (Retrieved from http://www.canadian-nurse.com/images/pdf/2013/cnj-sep-2013/files/assets/downloads/publication.pdf).

Gallagher, P., 2009. Collaborative essay testing: group work that counts. Int. J. Nurs. Educ.Scholarsh. 6 (1). http://dx.doi.org/10.2202/1548-923X.1845.

Giuliodori, M., Lujan, H., DiCarlo, S., 2008. Collaborative group testing benefits high- andlow-performing students. Adv. Physiol. Educ. 32 (4), 274–278. http://dx.doi.org/10.1152/advan.00101.2007.

Giuliodori, M., Lujan, H., DiCarlo, S., 2009. Student interaction characteristics during col-laborative group testing. Adv. Physiol. Educ. 33 (1), 24–29. http://dx.doi.org/10.1152/advan.90161.2008.

Haberyan, A., Barnett, J., 2010. Collaborative testing and achievement: are two heads re-ally better than one? J. Instr. Psychol. 37 (1), 32–41 (Retrieved from http://web.ebscohost.com.proxy2.lib.umanitoba.ca/ehost/pdfviewer/pdfviewer?vid=2&sid=63d9bd62-5eba-492f-9c25-e18d5c97eca3%40sessionmgr11&hid=12).

Heglund, S., Wink, D., 2011. Impact of double testing on student knowledge in a profes-sional issues course. J. Nurs. Educ. 50 (5), 278–280. http://dx.doi.org/10.3928/01484834-20110131-06.

Hickey, B.L., 2006. Lessons learned from collaborative testing. Nurse Educ. 31 (2),88–91.

Immordino-Yang, M.H., Damasio, A., 2007. We feel, therefore we learn: the relevance of af-fective and social neuroscience to education.Mind Brain Educ. 1, 3–10. http://dx.doi.org/10.1111/j.1751-228X.2007.00004.x.

Institute of Medicine, 2010. The Future of Nursing: Leading Change, Advancing Health.The National Academies Press, Washington, D.C.

Johnson, D., 2003. Social interdependence: interrelationships among theory, research, andpractice. Am. Psychol. 58 (11), 934–945.

Johnson, D., Johnson, R., Smith, K., 1998. Cooperative learning returns to college —

what evidence is there that it works? Change Mag. High. Learn. 30 (4), 26–35.http://dx.doi.org/10.1080/00091389809602629 (Retrieved from).

Jones, J., Lishman, K., 2011. Collaborative posttesting in an adult health nursing course.Teach. Learn. Nurs. 6, 176–180. http://dx.doi.org/10.1016/j.teln.2011.05.003.

Kapitanoff, S.H., 2009. Collaborative testing: cognitive and interpersonal processes relatedto enhanced test performance. Act. Learn. High. Educ. 10 (1), 56–70. http://dx.doi.org/10.1177/1469787408100195.

Keselyak, N., Saylor, C., Simmer-Beck, M., Bray, K., 2009. Examining the role of collabora-tive assessment in a didactic dental hygiene course. J. Dent. Educ. 73 (8), 980–990(Retrieved from http://www.jdentaled.org.proxy2.lib.umanitoba.ca/content/73/8/980.full.pdf+html).

Krueger, R., 2006. Is it a focus group? J. Wound Ostomy Continence Nurs. 33 (4),363–366.

Leight, H., Saunders, C., Calkins, R., Withers, M., 2012. Collaborative testingimproves performance but not content retention in a large-enrollment introduc-tory biology class. CBE Life Sci. Educ. 11, 392–401. http://dx.doi.org/10.1187/cbe.12-04-0048.

Lusk, M., Conklin, L., 2003. Collaborative testing to promote learning. J. Nurs. Educ. 42 (3),121–124 (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12661712).

Meseke, C., Bovee, M., Gran, D., 2009. Impact of collaborative testing on student perfor-mance and satisfaction in a chiropractic science course. J. Manip. Physiol. Ther. 32(4), 309–314. http://dx.doi.org/10.1016/j.jmpt.2009.03.012.

Mitchell, N., Melton, S., 2003. Collaborative testing: an innovative approach to test taking.Nurse Educ. 28 (2), 95–97 (Retrieved February 3, 2011, from http://ovidsp.tx.ovid.com.proxy1.lib.umanitoba.ca/sp-3.3.0b/ovidweb.cgi?&S=ECIEFPKNGIDDELDFNCCLAELBPCKPAA00&Link+Set=S.sh.15.16.19.22%7c13%7csl_10).

Molsbee, C.P., 2013. Collaborative testing and mixed results. Teach. Learn. Nurs. 8, 22–25.http://dx.doi.org/10.1016/j.teln.2012.09.001 (Retrieved from).

Pandey, D., Kapitanoff, S., 2011. The influence of anxiety and quality of interaction on col-laborative test performance. Act. Learn. High. Educ. 12, 163–174. http://dx.doi.org/10.1177/1469787411415077.

Pollard, K., Miers, M., 2008. From students to professionals: results of a longitudinalstudy of attitudes to pre-qualifying collaborative learning and working inhealth and social care in the United Kingdom. J. Interprof. Care 22 (4), 399–414(Retrieved from, http://informahealthcare.com.proxy1.lib.umanitoba.ca/doi/abs/10.1080/13561820802190483).

Pollard, K., Miers, M., Gilchrist, M., 2004. Collaborative learning for collaborativeworking? Initial findings from a longitudinal study of health and social care students.Health Soc. Care Community 12, 346–358. http://dx.doi.org/10.1111/j.1365-2524.2004.00504.x.

Rao, S.P., DiCarlo, S.E., 2000. Peer instruction improves performance on quizzes. Adv.Physiol. Educ. 24 (1), 51–55 (Retrieved from http://advan.physiology.org/cgi/content/abstract/24/1/S51).

Rao, S.P., Collins, H.L., DiCarlo, S.E., 2002. Collaborative testing enhances student learning.Adv. Physiol. Educ. 26, 37–41. http://dx.doi.org/10.1152/advan.00032.2001.

Sandahl, S., 2009. Collaborative testing as a learning strategy in nursing education: a re-view of the literature. Nurs. Educ. Perspect. 30 (3), 171–175.

Sandahl, S., 2010. Collaborative testing as a learning strategy in nursing. Nurs. Educ.Perspect. 31 (3), 142–147.

Slusser, S.R., Erickson, R.J., 2006. Group quizzes: an extension of the collaborativelearning process. Teach. Sociol. 34 (3), 249–262. http://dx.doi.org/10.1177/0092055X0603400304.

977D. Martin et al. / Nurse Education Today 34 (2014) 971–977

Wiggs, C., 2011. Collaborative testing: assessing teamwork and critical thinkingbehaviors in baccalaureate nursing students. Nurs. Educ. Today 31 (3), 279–282.http://dx.doi.org/10.1016/j.nedt.2010.10.027.

Woody, W.D., Woody, L.K., Bromley, S., 2008. Anticipated group versus individualexaminations. Teach. Psychol. 35, 13–17. http://dx.doi.org/10.1080/00986280701818540.

Zimbardo, P.G., Butler, L.D., Wolfe, V.A., 2003. Cooperative college examinations: moregain, less pain when students share information and grades. J. Exp. Educ. 71 (2),101–125 (Retrieved from http://www.jstor.org/stable/20152702).

Zipp, J.F., 2007. Learning by exams: the impact of two-stage cooperative tests. Teach.Sociol. 35 (62), 62–76. http://dx.doi.org/10.1177/0092055X0703500105.